A text training set determination method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a text training set determination method and device, electronic equipment and storage medium, the method comprising: obtaining a first quantity of first corpus from a corpus, and training a first model using the first corpus; randomly obtaining a second quantity of second corpus from the corpus, scoring the accuracy of the first model to obtain the score of each second corpus; constructing a training sample based on the second corpus whose score meets the low score condition, and retraining the first model; iteratively performing the process of randomly obtaining the second corpus to the retraining until the first model meets the preset accuracy requirement; after the first model meets the preset accuracy requirement, using the corpus used for training the first model as the script of the sample used for training the second model. The embodiment of the present disclosure reduces the data amount of the script, reduces the information redundancy in the script, and improves the efficiency of training the second model.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for determining a text training set, an electronic device, and a storage medium. Background Technology

[0002] Speech synthesis timbre models can generate speech with a specific timbre based on text. Their training data is in the form of "speech-text" data; the input to the model is text, and the expected output (label) is speech. The generation process (training process) of this model heavily relies on high-quality training text.

[0003] In related technologies, the method of collecting training texts for this model involves a three-stage process: 1) constructing a set of audio transcripts; 2) recording in a recording studio using the transcripts; and 3) filtering and annotating the audio transcripts. A key step is obtaining suitable audio transcripts. Choosing appropriate transcripts is a technically challenging problem. Only suitable transcripts can provide training data highly relevant to the model's application scenario. Ultimately, this training data can be used to obtain a speech synthesis timbre model with high generalization ability and high reliability in specific application scenarios.

[0004] In related technologies, in order to obtain a speech synthesis timbre model with high generalization ability, it is often trained with a large amount of training data, which consumes a lot of resources and time, resulting in low training efficiency of the speech synthesis timbre model. Summary of the Invention

[0005] This disclosure proposes a technical solution for determining the text training set.

[0006] According to one aspect of this disclosure, a method for determining a text training set is provided, comprising:

[0007] Obtain a first amount of first data from the corpus, and use the first data to train the first model;

[0008] A second number of second corpora are randomly selected from the corpus, and the accuracy of the first model is scored to obtain the score for each second corpus.

[0009] Training samples are constructed based on the second corpus whose scores meet the low score criteria, and the first model is trained again.

[0010] The process of randomly acquiring the second corpus and retraining is performed iteratively until the first model meets the preset accuracy requirements.

[0011] After the first model meets the preset accuracy requirements, the corpus used to train the first model is used as the audio transcript of the sample used to train the second model. The first model and the second model are used to output speech based on the text, and the timbre of the speech output by the first model and the second model is different.

[0012] In one possible implementation, training the first model using the first corpus includes:

[0013] The text in the first corpus is converted into first speech using a general speech synthesis service;

[0014] The first model is trained by using the text in the first corpus as input and the first speech obtained by conversion as the expected output.

[0015] In one possible implementation, obtaining a first quantity of the first corpus from the corpus includes:

[0016] By traversing and combining the phonemes in Chinese Pinyin according to the grammar of Pinyin, a set consisting of the combined Pinyin sequences is obtained;

[0017] Based on the set, a first number of first corpora are selected from the corpus, wherein the pinyin sequences in the first corpus have a higher coverage of the pinyin sequences in the set than a coverage threshold.

[0018] In one possible implementation, the second model is applied to the target scenario. After traversing and combining the phonemes in Chinese Pinyin according to the Pinyin grammar to obtain a set composed of the combined Pinyin sequences, the method further includes:

[0019] Extract target scene words from the text that appear more frequently than a set frequency threshold in the target scene;

[0020] Increase the priority weight of the pinyin sequence of the target scene words in the set, or delete words other than the pinyin sequence of the target scene words in the set.

[0021] In one possible implementation, scoring the accuracy of the first model includes:

[0022] The first model is used to convert the text in the second corpus into speech, resulting in the second speech.

[0023] The second speech is used to perform text recognition using a speech recognition model to obtain the text recognition result;

[0024] The accuracy of the text recognition result relative to the text in the second corpus is determined and used as the score for the second corpus.

[0025] In one possible implementation, training samples are constructed based on a second corpus containing data that meets the low-score criteria, including:

[0026] Based on the scores of the second corpus from low to high, select the third number of second corpus samples to construct training samples.

[0027] In one possible implementation, the first model has the same structure as the second model.

[0028] According to one aspect of this disclosure, a text training set determination apparatus is provided, comprising:

[0029] The first training module is used to obtain a first amount of first corpus from the corpus and use the first corpus to train the first model;

[0030] The scoring module is used to randomly select a second number of second corpora from the corpus, score the accuracy of the first model, and obtain the score of each second corpus.

[0031] The second training module is used to construct training samples based on the second corpus whose scores meet the low score condition, and to retrain the first model.

[0032] An iterative control module is used to iteratively execute the process of randomly acquiring the second corpus until the first model meets the preset accuracy requirements.

[0033] The corpus determination module is used to use the corpus used to train the first model as the audio transcript of the sample used to train the second model after the first model meets the preset accuracy requirements. The first model and the second model are used to output speech based on the text, and the timbre of the speech output by the first model and the second model is different.

[0034] In one possible implementation, the first training module includes:

[0035] The first speech conversion module is used to convert the text in the first corpus into first speech through a general speech synthesis service;

[0036] The first training submodule is used to train the first model by taking the text in the first corpus as the input of the first model and the converted first speech as the expected output.

[0037] In one possible implementation, the first training module includes:

[0038] The Pinyin construction module is used to traverse and combine the phonemes in Chinese Pinyin according to the grammar of Chinese Pinyin, and obtain a set composed of the combined Pinyin sequences;

[0039] A filtering module is used to filter out a first number of first corpora from the corpus based on the set, wherein the coverage of the pinyin sequences in the first corpus to the pinyin sequences in the set is higher than a coverage threshold.

[0040] In one possible implementation, the second model is applied to the target scene, and the device further includes:

[0041] The vocabulary acquisition module is used to acquire target scene words in the text that appear more frequently than a set frequency threshold in the target scene.

[0042] The vocabulary adjustment module is used to increase the priority weight of the pinyin sequence of target scene vocabulary in the set, or to delete vocabulary other than the pinyin sequence of target scene vocabulary in the set.

[0043] In one possible implementation, the scoring module includes:

[0044] The second speech conversion module is used to convert the text of the second corpus into speech using the first model to obtain the second speech.

[0045] The text recognition module is used to perform text recognition on the second speech using a speech recognition model to obtain the text recognition result;

[0046] The numerator module is used to determine the accuracy of the text recognition result relative to the text in the second corpus, which is then used as the score for the second corpus.

[0047] In one possible implementation, the second training module includes:

[0048] The training sample construction module is used to select a third number of second corpora in order of their scores from low to high to construct training samples.

[0049] In one possible implementation, the first model has the same structure as the second model.

[0050] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0051] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0052] In this embodiment, both the first and second models are used to output speech from text, differing only in the timbre of the output speech. During training, the corpus does not carry timbre information; it is only in the audio obtained from recording the corpus that timbre information is carried. Therefore, the corpus used to train the first and second models can be universal. If all the corpus used to train the first model meets the accuracy requirements of the first model, it can also meet the accuracy requirements of the second model and can be used as the recorded transcripts for training the second model. By incrementally iterating, selecting a batch of corpus that the first model cannot accurately generate speech to expand the recorded transcripts used for training the second model improves the efficiency of obtaining recorded transcripts. This method can specifically compensate for the shortcomings in the recorded transcripts, minimizing the inclusion of corpus that cannot effectively improve the accuracy of the second model, reducing the data volume of the recorded transcripts, reducing information redundancy in the recorded transcripts, and improving the efficiency of training the second model.

[0053] Furthermore, the accuracy of the first model is scored during this process, which can evaluate the effect of the speech synthesis timbre model (first model) trained by combining the current recording transcripts, reducing the risk of synthesis quality defects in the final model to be trained (second model). Before training the second model, the corresponding timbre is recorded using the determined recording transcript, and then the second model is trained using the transcript and the recording to obtain the second model used to output the speech with that timbre.

[0054] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0056] Figure 1 A flowchart illustrating a method for determining a text training set according to an embodiment of this disclosure is shown.

[0057] Figure 2 A block diagram of a text training set determination apparatus according to an embodiment of the present disclosure is shown.

[0058] Figure 3 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0059] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0060] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0061] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0062] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0063] A speech synthesis timbre model can generate speech with a specific timbre based on text, and then read the text aloud using that timbre. This model can be trained using samples, which contain text and corresponding speech with the specific timbre. The speech with the specific timbre used during training can be obtained by recording the text. The selection of the recorded text used for training is crucial to improving the training efficiency of the speech synthesis timbre model.

[0064] In this embodiment, a first corpus is obtained from a corpus, and the first model is trained using the first corpus; a second corpus is randomly obtained from the corpus, and the accuracy of the first model is scored to obtain a score for each second corpus; training samples are constructed based on the second corpus whose scores meet the low score condition, and the first model is trained again; the process of randomly obtaining the second corpus until the retraining is performed iteratively is executed until the first model meets the preset accuracy requirement; after the first model meets the preset accuracy requirement, the corpus used to train the first model is used as the recording transcript of the samples used to train the second model, wherein the first model and the second model are used to output speech based on text, and the timbre of the speech output by the first model and the second model is different.

[0065] Since both the first and second models output speech from text, the difference lies in the timbre of the output speech. During training, the corpus itself doesn't carry timbre information; it's only in the audio recordings obtained from the corpus that timbre information is generated. Therefore, the corpus used to train the first and second models can be interchangeable. If all the corpus used to train the first model meets its accuracy requirements, it will also meet the accuracy requirements of the second model and can be used as the training transcripts for the second model. By incrementally iterating, selecting a batch of corpus data that the first model cannot accurately generate speech to expand the training transcripts for the second model, the efficiency of obtaining transcripts can be improved. This method can specifically address the shortcomings in the transcripts, avoiding the inclusion of corpus data that cannot effectively improve the accuracy of the second model, reducing the amount of data in the transcripts, minimizing information redundancy, and improving the efficiency of training the second model.

[0066] Furthermore, the accuracy of the first model is scored during this process, which can evaluate the effect of the speech synthesis timbre model (first model) trained by combining the current recording transcripts, reducing the risk of synthesis quality defects in the final model to be trained (second model). Before training the second model, the corresponding timbre is recorded using the determined recording transcript, and then the second model is trained using the transcript and the recording to obtain the second model used to output the speech with that timbre.

[0067] In one possible implementation, the text training set determination method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable instructions stored in memory.

[0068] Figure 1 A flowchart illustrating a text training set determination method according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the method for determining the text training set includes:

[0069] In step S11, a first amount of first data is obtained from the corpus, and the first model is trained using the first data.

[0070] The corpus contains text data. Therefore, training the first model using the first corpus can be achieved by obtaining the text's label (i.e., the expected speech corresponding to the text), then using the text as the input to the first model to obtain the speech output by the first model. By adjusting the parameters of the first model, the output speech can be made closer to the expected speech, thus realizing the training of the first model.

[0071] The corpus can be a general corpus. In one possible implementation, the generation process of a general corpus includes: segmenting sentences based on sentence-ending punctuation from a large number of general texts; obtaining the pinyin sequence of each sentence using a commonly used Chinese pinyin conversion tool (such as pypinyin); associating the pinyin sequence with the text to obtain the corpus, whose data format is text and pinyin, such as (Mayday's concert, wu3 yue4 tian1 de5 yan3 chang4 hui4).

[0072] It should be noted that the corpus can also be data from a specific application scenario. The text topics can be selected according to the specific application scenario of the actual speech synthesis timbre model (second model), without limitation here. For example, if the application scenario of the second model is a news scenario, the text in the corpus can also be from the news scenario.

[0073] When acquiring the first corpus, it can be acquired randomly or according to a specific rule. For details on the acquisition method, please refer to the possible implementation methods provided in this disclosure, which will not be elaborated here. The number of corpora acquired can be determined according to actual needs. For example, the first number can be 100.

[0074] It should be noted that both the first and second models here are speech synthesis timbre models used to output speech based on text. In one possible implementation, the first and second models have the same structure. In one example, the initial structure and parameters of the first and second models are the same, but after training, they become two models with different parameters, and the timbre of their output speech is also different. Therefore, the recording text selected based on the first model can maintain the good generalization ability of the second model when used to train the second model.

[0075] In step S12, a second number of second corpora are randomly obtained from the corpus, and the accuracy of the first model is scored to obtain the score of each second corpus.

[0076] The second corpus is randomly obtained from the corpus. The second corpus may or may not include the first corpus.

[0077] When scoring the accuracy of the first model, its accuracy can be evaluated by the degree of difference between the output speech of the first model and the label (expected speech). In one example, the accuracy of the first model is directly proportional to the score.

[0078] The specific value of the second quantity can be greater than that of the first quantity so that more second corpora that meet the low score criteria can be obtained after filtering. However, the specific value of the second quantity should not be too large to avoid consuming too many processing resources. Based on practical experience, the specific value of the second quantity can be between 500 and 2000, which can better balance the processing efficiency and power consumption.

[0079] For each second corpus, a score can be obtained to characterize the accuracy of the audio generated by the first model for that second corpus.

[0080] In this process, the effectiveness of the speech synthesis timbre model (first model) trained by combining the current recording transcripts can be evaluated, reducing the risk of synthesis quality defects in the final model to be trained (second model).

[0081] In step S13, training samples are constructed based on the second corpus whose scores meet the low score condition, and the first model is trained again.

[0082] The low-score condition is used to filter second corpora that the first model cannot accurately synthesize speech. In one possible implementation, training samples are constructed based on the second corpora whose scores meet the low-score condition. This includes selecting a third number of second corpora in ascending order of their scores to construct training samples. The third number can be the same as or different from the first number. For example, the low-score condition can be the last 100 second corpora with the lowest scores.

[0083] In another example, a low score condition could also be a second corpus with a score below a score threshold.

[0084] The first model cannot accurately convert the second corpus that meets the low score criteria into speech. However, if the first model is trained with the second corpus that meets the low score criteria, it can accurately convert the second corpus into speech, which can make up for the shortcomings of the first model in speech synthesis and specifically compensate for the recording transcripts used to train the second model.

[0085] After obtaining the second corpus with scores meeting the low-score criteria, the second speech of the second corpus with low scores can be identified as a label to constitute training samples. The specific process of determining the second speech is the same as the process of generating the first speech using the first corpus, and can be found in the possible implementations provided in this disclosure, which will not be elaborated here.

[0086] When retraining the first model using the constructed samples, the training process is the same as the training process in step S11, and will not be repeated here.

[0087] In step S14, the process of randomly acquiring the second corpus and retraining is performed iteratively until the first model meets the preset accuracy requirements.

[0088] After retraining the first model, steps S12-S13 can be executed iteratively until the first model meets the preset accuracy requirements. The preset accuracy requirements can be, for example, that the accuracy improvement after multiple iterations is not significant, or that the optimized accuracy has reached the set accuracy threshold.

[0089] In step S15, after the first model meets the preset accuracy requirements, the corpus used to train the first model is used as the audio transcript of the sample used to train the second model.

[0090] The first model and the second model are used to output speech based on text, and the timbre of the speech output by the first model and the second model is different.

[0091] Since both the first and second models are used to generate speech from text corpora, the corpora themselves do not carry timbre information during training. Timbre information is only present in the audio recordings obtained from the corpora. Therefore, the corpora used to train the first and second models can be interchangeable. During later training, recordings with the desired timbre are used as labels based on the corpora, and the trained models can then generate speech that matches the recorded timbre. Thus, the second corpus, which meets the low-score criteria, can also compensate for the second model's weakness in speech synthesis and can be used as the training transcript for the second model.

[0092] Therefore, the corpus used to train the first model can be used as the audio transcripts of the samples used to train the second model. Then, using these transcripts, speech with the required timbre can be recorded to obtain training samples for training the second model, ultimately resulting in a second model that meets the requirements.

[0093] Since both the first and second models output speech from text, the difference lies in the timbre of the output speech. During training, the corpus itself doesn't carry timbre information; it's only in the audio recordings obtained from the corpus that timbre information is generated. Therefore, the corpus used to train the first and second models can be interchangeable. If all the corpus used to train the first model meets its accuracy requirements, it will also meet the accuracy requirements of the second model and can be used as the training transcripts for the second model. By incrementally iterating, selecting a batch of corpus data that the first model cannot accurately generate speech to expand the training transcripts for the second model, the efficiency of obtaining transcripts can be improved. This method can specifically address the shortcomings in the transcripts, avoiding the inclusion of corpus data that cannot effectively improve the accuracy of the second model, reducing the amount of data in the transcripts, minimizing information redundancy, and improving the efficiency of training the second model.

[0094] Furthermore, the accuracy of the first model is scored during this process, which can evaluate the effect of the speech synthesis timbre model (first model) trained by combining the current recording transcripts, reducing the risk of synthesis quality defects in the final model to be trained (second model). Before training the second model, the corresponding timbre is recorded using the determined recording transcript, and then the second model is trained using the transcript and the recording to obtain the second model used to output the speech with that timbre.

[0095] In one possible implementation, training the first model using the first corpus includes: converting the text in the first corpus into first speech using a general speech synthesis service; using the text in the first corpus as input to the first model and the converted first speech as the desired output to train the first model.

[0096] In this embodiment of the disclosure, the aim is to select suitable training transcripts for training the second model. The timbre of the audio output by the second model depends mainly on the recording, not on the timbre of the audio during the training of the first model. Therefore, in order to facilitate the quick selection of the recording transcripts for the second model, when training the first model, it is not necessary to record the transcripts. Instead, the transcripts in the first corpus can be directly converted into the first speech through a general speech synthesis service, which is convenient and fast and does not affect the quality of the selected recording transcripts.

[0097] The general speech synthesis service mentioned here can be any service capable of synthesizing text into speech, and this disclosure does not specifically limit it.

[0098] Therefore, when training the first model, the text in the first corpus can be used as the input to the first model, and the converted first speech can be used as the expected output to train the first model. Specifically, using the text in the first corpus as the input to the first model will produce the simulated speech output by the first model. Then, the loss between the simulated speech and the first speech is determined, and the parameters of the first model are modified using this loss so that the simulated speech output by the first model is close to the first speech.

[0099] In one possible implementation, obtaining a first quantity of first corpus from the corpus includes: traversing and combining phonemes in Chinese Pinyin according to the Pinyin grammar to obtain a set composed of combined Pinyin sequences; and based on the set, selecting a first quantity of first corpus from the corpus, wherein the coverage of the Pinyin sequences in the first corpus to the Pinyin sequences in the set is higher than a coverage threshold.

[0100] Phonemes in Hanyu Pinyin are the elements that constitute Pinyin; they are the smallest units or the smallest speech segments that make up Pinyin. In this embodiment of the disclosure, phonemes are combined to form Pinyin sequences, which can be composed of single or multiple phonemes.

[0101] For example, the general composition of a combination to generate a pinyin sequence includes: a single initial consonant audio (such as d, f, q, etc.); a single vowel plus tone combination (such as i3, an1, etc.); a single pinyin syllable (such as di3, fan1, etc.); and a combination of two pinyin syllables appearing in sequence (such as di3-fan1, etc.).

[0102] The combined pinyin sequence can include at least one of the pinyin sequences of characters, words, and sentences. Since there are many combinations of pinyin sequences that constitute a sentence, in this embodiment of the disclosure, the pinyin sequence includes the pinyin of a single character and the pinyin of a word composed of 2-3 characters.

[0103] It should be noted that, as one possible implementation, the combined pinyin sequence is obtained by enumerating and traversing the combination. Therefore, the obtained pinyin sequence can be a sequence of words that already exist in the current Chinese language, or a sequence of words that do not yet exist in the Chinese language. These non-existent sequences may appear in the Chinese language in the future. Therefore, the set of combined pinyin sequences can be reused for a considerable period of time, which can save processing resources.

[0104] Moreover, by traversing and combining the phonemes in Chinese Pinyin according to the grammar of Pinyin, a set of Pinyin sequences with good coverage of various application scenarios can be obtained. This set of Pinyin sequences is a universal set applicable to any application scenario.

[0105] After obtaining the set, when selecting the first set from the corpus, if the first number of corpora are not repeated and can broadly cover the set, the first model / second model trained from the first number of selected corpora will have better generalization ability. Therefore, when selecting the first number of corpora, the standard will be that the coverage of the pinyin sequences in the first corpora to the pinyin sequences in the set is higher than the coverage threshold.

[0106] The first corpus might consist of pinyin sequences of sentences. A single sentence's pinyin sequence might contain (i.e., cover) pinyin sequences of multiple characters or words. Coverage can be understood as the proportion of pinyin sequences in the first corpus (a first set of sentences) to the total number of pinyin sequences in that set. For example, if the first corpus consists of 100 sentences, these 100 sentences might cover 20% of the pinyin sequences in the set (coverage of 20%), while another 100 sentences might cover 30% of the pinyin sequences (coverage of 30%). Assuming a coverage threshold of 25%, the 100 sentences with 30% coverage would be selected for training the first model.

[0107] The coverage threshold is preset and can be set empirically. Alternatively, as a possible implementation, a greedy algorithm can be used to select the first corpus with the highest global coverage. Similarly, when obtaining the second corpus, a greedy algorithm can be used to select the second corpus with the highest global coverage of pinyin sequence combinations, and this second corpus can be used as the corpus to be evaluated.

[0108] In this embodiment, phonemes in Chinese Pinyin are traversed and combined according to the Pinyin grammar to obtain a set composed of combined Pinyin sequences. Based on the set, a first quantity of first corpus is selected from the corpus. Since the Pinyin sequences in the selected first corpus have a coverage of more than a coverage threshold for the Pinyin sequences in the set, the first model / second model trained from the selected first quantity of first corpus has good generalization ability. Therefore, it can save the number of subsequent iterations, improve the efficiency of selecting audio transcripts, and save processing resources for selecting audio transcripts.

[0109] In one possible implementation, the second model is applied to the target scenario. After traversing and combining the phonemes in the Chinese Pinyin according to the Pinyin grammar to obtain a set composed of the combined Pinyin sequences, the method further includes: obtaining target scenario words in the text under the target scenario whose frequency is higher than a set frequency threshold; increasing the priority weight of the Pinyin sequences of the target scenario words in the set, or deleting words other than the Pinyin sequences of the target scenario words in the set.

[0110] The target scenarios here can be, for example, scenarios such as news, education, sports, etc. For specific target scenarios, they can be selected according to actual needs, and the present disclosure does not limit this.

[0111] When obtaining the vocabulary in the target scenario, the manuscript in the target scenario can be obtained, and then the word frequency of the words in the manuscript can be counted, and the target scenario vocabulary with a word frequency higher than the set frequency threshold in the manuscript can be selected. For example, 1000 words with relatively high word frequencies are selected, and then the pinyin sequences of these 1000 words are obtained.

[0112] After obtaining the target scenario vocabulary, the priority weight of the pinyin sequence of the target scenario vocabulary in the set composed of the combined pinyin sequences can be increased, or alternatively, the words other than the pinyin sequence of the target scenario vocabulary in the set can be deleted. In the updated set like this, the importance degree of the vocabulary in the target scenario is highlighted. Thus, the first corpus selected, after being used to train the first model / second model, will perform better and have higher accuracy when applied to the target scenario.

[0113] In one possible implementation, the scoring of the accuracy of the first model includes: using the first model to perform speech conversion on the text of the second corpus to obtain a second speech; performing text recognition on the second speech through a speech recognition model to obtain a text recognition result; determining the accuracy rate of the text recognition result relative to the text of the second corpus as the score of the second corpus.

[0114] After inputting the text of the second corpus into the first model, the first model will output the generated speech. For the convenience of description, it is referred to as the second speech here. When the second speech generated by the first model is accurate, the content pronounced by the second speech should be the text of the second corpus. Then, a speech recognition model can be used here to recognize the text in the second speech to obtain a text recognition result, and by judging the consistency between the text recognition result and the text input into the first model, the accuracy rate of the first model for speech generation of the text of the second corpus can be determined.

[0115] Exemplarily, the proportion of the correctly recognized text in the total number of texts can be determined as the score of the second corpus. For example, the text of the second corpus is specifically "The weather is nice today". This text is converted into the second speech by the first model, and then the text in the second speech is recognized. The obtained text recognition result is "The weather is not good today". The number of misrecognized words is 1, and the total number of words is 6. Then, the accuracy score corresponding to the second corpus "The weather is nice today" is 83.3%.

[0116] In this embodiment, the text of the second corpus is converted into speech using the first model to obtain second speech; the second speech is then used to perform text recognition using a speech recognition model to obtain a text recognition result; the accuracy of the text recognition result relative to the text of the second corpus is determined as the score of the second corpus. This allows for a rapid and accurate evaluation of the score of the second corpus, improving the efficiency of constructing the audio transcript and the effectiveness of the obtained audio transcript in training the second model.

[0117] It should be noted that this disclosure does not specifically limit the use of the first and second models. The first and second models are speech synthesis timbre models. As an example, the model frameworks that can be selected include: (1) commonly used acoustic models, such as FastSpeech, Tacotron, etc.; (2) commonly used vocoder models, such as HifiGan, WaveNet, WaveRNN, etc. As one optional embodiment, the basic model based on FastSpeech2 trained in a large-scale open-source database can be used as the acoustic model, and HifiGan can be used as the vocoder model. The acoustic model can be fine-tuned using the latest candidate simulation data of the current version to obtain the speech synthesis timbre model. The specific scheme can be designed according to actual needs, and this application does not specifically limit it.

[0118] The following describes an application scenario of an embodiment of this disclosure. In this application scenario, the speech synthesis timbre model is a general-purpose model. In this embodiment, phonemes in Chinese Pinyin are combined to obtain a set of Pinyin sequence combinations. Then, based on a greedy algorithm, 100 sentences with optimal coverage of this set are selected from a general corpus. These 100 sentences are used to train the first model. From the general corpus, 1000 sentences are randomly selected as second-order corpora, and the pronunciation accuracy of the first model is scored. The 100 sentences with the lowest scores are selected to construct training samples and retrain the first model. The process of selecting 1000 sentences as second-order corpora and retraining the first model is iteratively executed until the first model meets the accuracy requirements. Then, the corpus used to train the first model is used as the recording transcript for training the second model. After obtaining the recording transcript for the second model, the corresponding timbre is recorded using the recording transcript as a label for the training transcript, and then the second model is trained.

[0119] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0120] In addition, this disclosure also provides a text training set determination device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any of the text training set determination methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section and will not be repeated here.

[0121] Figure 2 A block diagram of a text training set determination apparatus according to an embodiment of the present disclosure is shown, such as Figure 2 As shown, the device 20 includes:

[0122] The first training module 21 is used to obtain a first amount of first corpus from the corpus and use the first corpus to train the first model;

[0123] The scoring module 22 is used to randomly obtain a second number of second corpora from the corpus, score the accuracy of the first model, and obtain the score of each second corpus.

[0124] The second training module 23 is used to construct training samples based on the second corpus whose scores meet the low score condition, and to retrain the first model.

[0125] The iterative control module 24 is used to iteratively execute the process of randomly acquiring the second corpus until the first model meets the preset accuracy requirements.

[0126] The corpus determination module 25 is used to use the corpus used to train the first model as the recorded transcript of the sample used to train the second model after the first model meets the preset accuracy requirements. The first model and the second model are used to output speech based on the text, and the timbre of the speech output by the first model and the second model is different.

[0127] In one possible implementation, the first training module includes:

[0128] The first speech conversion module is used to convert the text in the first corpus into first speech through a general speech synthesis service;

[0129] The first training submodule is used to train the first model by taking the text in the first corpus as the input of the first model and the converted first speech as the expected output.

[0130] In one possible implementation, the first training module includes:

[0131] The Pinyin construction module is used to traverse and combine the phonemes in Chinese Pinyin according to the grammar of Chinese Pinyin, and obtain a set composed of the combined Pinyin sequences;

[0132] A filtering module is used to filter out a first number of first corpora from the corpus based on the set, wherein the coverage of the pinyin sequences in the first corpus to the pinyin sequences in the set is higher than a coverage threshold.

[0133] In one possible implementation, the second model is applied to the target scene, and the device further includes:

[0134] The vocabulary acquisition module is used to acquire target scene words in the text that appear more frequently than a set frequency threshold in the target scene.

[0135] The vocabulary adjustment module is used to increase the priority weight of the pinyin sequence of target scene vocabulary in the set, or to delete vocabulary other than the pinyin sequence of target scene vocabulary in the set.

[0136] In one possible implementation, the scoring module includes:

[0137] The second speech conversion module is used to convert the text of the second corpus into speech using the first model to obtain the second speech.

[0138] The text recognition module is used to perform text recognition on the second speech using a speech recognition model to obtain the text recognition result;

[0139] The numerator module is used to determine the accuracy of the text recognition result relative to the text in the second corpus, which is then used as the score for the second corpus.

[0140] In one possible implementation, the second training module includes:

[0141] The training sample construction module is used to select a third number of second corpora in order of their scores from low to high to construct training samples.

[0142] In one possible implementation, the first model has the same structure as the second model.

[0143] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0144] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0145] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described method.

[0146] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0147] Figure 3 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal device. (Refer to...) Figure 3 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0148] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (Mac OSX) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0149] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0150] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0151] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0152] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0153] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0154] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0155] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0156] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0157] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0158] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0159] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0160] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0161] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0162] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for determining a text training set, characterized in that, include: Obtain a first amount of first data from the corpus, and use the first data to train the first model; A second number of second corpora are randomly selected from the corpus, and the accuracy of the first model is scored to obtain the score for each second corpus. Training samples are constructed based on the second corpus whose scores meet the low score criteria, and the first model is trained again. The process of randomly acquiring the second corpus and retraining is performed iteratively until the first model meets the preset accuracy requirements. After the first model meets the preset accuracy requirements, the corpus used to train the first model is used as the audio transcript of the sample used to train the second model. The first model and the second model are used to output speech based on the text, and the timbre of the speech output by the first model and the second model is different.

2. The method according to claim 1, characterized in that, The first model is trained using the first corpus, including: The text in the first corpus is converted into first speech using a general speech synthesis service; The first model is trained by using the text in the first corpus as input and the first speech obtained by conversion as the expected output.

3. The method according to claim 1, characterized in that, The step of obtaining a first quantity of first data from the corpus includes: By traversing and combining the phonemes in Chinese Pinyin according to the grammar of Pinyin, a set consisting of the combined Pinyin sequences is obtained; Based on the set, a first number of first corpora are selected from the corpus, wherein the pinyin sequences in the first corpus have a higher coverage of the pinyin sequences in the set than a coverage threshold.

4. The method according to claim 3, characterized in that, The second model is applied to the target scenario. After traversing and combining the phonemes in Chinese Pinyin according to the Pinyin grammar to obtain a set composed of the combined Pinyin sequences, the method further includes: Extract target scene words from the text that appear more frequently than a set frequency threshold in the target scene; Increase the priority weight of the pinyin sequence of the target scene words in the set, or delete words other than the pinyin sequence of the target scene words in the set.

5. The method according to claim 1, characterized in that, The scoring of the accuracy of the first model includes: The first model is used to convert the text in the second corpus into speech, resulting in the second speech. The second speech is used to perform text recognition using a speech recognition model to obtain the text recognition result; The accuracy of the text recognition result relative to the text in the second corpus is determined and used as the score for the second corpus.

6. The method according to claim 1, characterized in that, Training samples were constructed based on a second corpus containing text that met the low-score criteria, including: Based on the scores of the second corpus from low to high, select the third number of second corpus samples to construct training samples.

7. The method according to claim 1, characterized in that, The first model has the same structure as the second model.

8. A text training set determination device, characterized in that, include: The first training module is used to obtain a first amount of first corpus from the corpus and use the first corpus to train the first model; The scoring module is used to randomly select a second number of second corpora from the corpus, score the accuracy of the first model, and obtain the score of each second corpus. The second training module is used to construct training samples based on the second corpus whose scores meet the low score condition, and to retrain the first model. An iterative control module is used to iteratively execute the process of randomly acquiring the second corpus until the first model meets the preset accuracy requirements. The corpus determination module is used to use the corpus used to train the first model as the audio transcript of the sample used to train the second model after the first model meets the preset accuracy requirements. The first model and the second model are used to output speech based on the text, and the timbre of the speech output by the first model and the second model is different.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images