A pause position prediction method, speech synthesis method and related equipment

Abstract

The present application provides a pause position prediction method, a speech synthesis method and related equipment, wherein the pause position prediction method includes: obtaining a target text; segmenting the target text based on a pre-established first dictionary to obtain a segmentation result of the target text, wherein the first dictionary includes a plurality of first basic language units and a plurality of second basic language units, the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words; predicting the pause position of the target text according to the segmentation result of the target text. The pause position prediction method provided by the present application can predict a more reasonable pause position, and speech synthesis can be performed according to the pause position predicted by the pause position prediction method provided by the present application for the target text, so as to obtain a better synthesized speech.

Description

Technical Field

[0001] The present application relates to the field of speech synthesis technology, and in particular to a pause position prediction method, a speech synthesis method and related equipment. Background Art

[0002] Pause position prediction is the front-end text processing process of speech synthesis, that is, predicting the pause positions in the text used for speech synthesis, so that in the back-end speech synthesis process, the pause positions in the text can be used to synthesize speech that is as consistent as possible with the pause habits of human speech. It can be seen that the rationality of the predicted pause positions directly affects the quality of speech synthesis, and how to predict reasonable pause positions is an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present application provides a pause position prediction method, device, equipment and storage medium for predicting a reasonable pause position, and its technical solution is as follows:

[0004] A pause position prediction method, comprising:

[0005] Get the target text;

[0006] Based on a pre-established first dictionary, the target text is segmented to obtain a segmentation result of the target text, wherein the first dictionary includes a plurality of first basic language units and a plurality of second basic language units, the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words;

[0007] The pause positions of the target text are predicted according to the word segmentation result of the target text.

[0008] Optionally, the process of acquiring the plurality of first basic language units includes:

[0009] Acquire a second dictionary, wherein the second dictionary includes a plurality of grammatical words;

[0010] A first basic language unit is constructed based on the grammatical words in the second dictionary and the parts of speech of the grammatical words in the second dictionary.

[0011] Optionally, the process of obtaining the plurality of second basic language units includes:

[0012] Each grammatical word in the second dictionary that is not covered by the first basic language unit is determined as a second basic language unit to obtain a plurality of second basic language units.

[0013] Optionally, constructing the first basic language unit based on the grammatical words in the second dictionary and the parts of speech of the grammatical words in the second dictionary includes:

[0014] Based on the parts of speech of the grammatical words in the second dictionary and a predetermined part-of-speech combination, the grammatical words in the second dictionary are combined to obtain a plurality of candidate basic language units, wherein the part-of-speech combination is a combination of parts of speech of a plurality of grammatical words that are read together;

[0015] A target basic language unit is screened out from the multiple candidate basic language units, and the screened out target basic language unit is used as the constructed first basic language unit.

[0016] Optionally, the step of selecting a target basic language unit from the plurality of candidate basic language units includes:

[0017] Determine the confusions corresponding to the multiple candidate basic language units respectively;

[0018] Determining, from the plurality of candidate basic language units, a candidate basic language unit that successfully matches the text in the text library;

[0019] According to the perplexities respectively corresponding to the multiple candidate basic language units, a target basic language unit is screened out from the candidate basic language units that successfully match the text in the text library.

[0020] Optionally, the step of selecting a target basic language unit from the candidate basic language units that successfully match the text in the text library according to the perplexities respectively corresponding to the multiple candidate basic language units includes:

[0021] Determine the minimum perplexity among the perplexities corresponding to the candidate basic language units that fail to match the text in the text library as a perplexity threshold;

[0022] From the candidate basic language units that successfully match the text in the text library, select the candidate basic language units whose corresponding perplexity is less than the perplexity threshold as the target basic language units.

[0023] Optionally, predicting the pause position of the target text according to the word segmentation result of the target text includes:

[0024] Inputting the word segmentation result of the target text into a pre-established chunk model to obtain a chunk result corresponding to the target text, wherein the chunk model is trained using training text marked with text block boundaries;

[0025] The pause position of the target text is determined according to the chunk result corresponding to the target text.

[0026] Optionally, determining the pause position of the target text according to the chunk result corresponding to the target text includes:

[0027] Determine a text block boundary in a chunk result corresponding to the target text as a first candidate pause position of the target text;

[0028] Predicting a speech pause position of the speech corresponding to the target text, and mapping the speech pause position to the target text to obtain a second candidate pause position of the target text;

[0029] The pause position of the target text is determined according to the first candidate pause position of the target text and the second candidate pause position of the target text.

[0030] Optionally, the process of acquiring the training text marked with text block boundaries includes:

[0031] Obtaining a first text set;

[0032] Performing word segmentation on each text in the first text set based on the first dictionary to obtain a word segmentation result corresponding to each text in the first text set;

[0033] Acquire multiple text blocks according to the word segmentation results corresponding to each text in the first text set, wherein each text block is composed of a plurality of consecutive words in the word segmentation results corresponding to a text in the first text set;

[0034] Counting the frequency of each text block appearing in the text in the second text set, and determining the text block whose appearance frequency is greater than a preset frequency threshold as a high-frequency text block;

[0035] The word segmentation results corresponding to each text in the first text set are grouped based on the high-frequency text blocks to obtain training texts marked with text block boundaries.

[0036] A speech synthesis method, comprising:

[0037] Using any of the pause position prediction methods described above, predicting pause positions of a target text for speech synthesis;

[0038] According to the pause positions of the target text, the speech corresponding to the target text is synthesized.

[0039] A pause position prediction device, comprising: a text acquisition module, a word segmentation module and a pause position prediction module;

[0040] The text acquisition module is used to acquire the target text;

[0041] The word segmentation module is used to segment the target text based on a pre-established first dictionary to obtain a word segmentation result of the target text, wherein the first dictionary includes a plurality of first basic language units and a plurality of second basic language units, the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words;

[0042] The pause position prediction is used to predict the pause position of the target text according to the word segmentation result of the target text.

[0043] A pause position prediction device, comprising: a memory and a processor;

[0044] The memory is used to store programs;

[0045] The processor is used to execute the program to implement each step of any of the above-mentioned pause position prediction methods.

[0046] A readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, each step of any of the above-mentioned pause position prediction methods is implemented.

[0047] Through the above scheme, it can be known that the pause position prediction method provided by the present application first obtains the target text to be predicted for the pause position, then performs word segmentation on the target text based on the first dictionary with the basic language unit as the content, and finally predicts the pause position of the target text according to the word segmentation result of the target text. The pause position of the target text can be determined by the pause position prediction method provided by the present application. In addition, when determining the pause position, the present application is based on the word segmentation result of the target text, and the word segmentation result of the target text is obtained by segmenting the target text based on the first dictionary with the basic language unit as the content. Since the basic language unit is constructed on the basis of considering the continuous reading and tone change characteristics of the speech, therefore, the first dictionary with the basic language unit as the content performs word segmentation on the target text, which can avoid dividing the grammatical words that are read together, that is, the word segmentation result obtained by the first dictionary with the basic language unit as the content is more reasonable, and then, the pause position determined based on the reasonable word segmentation result is also more reasonable. According to the pause position prediction method provided by the present application, speech synthesis is performed for the more reasonable pause position predicted for the target text, and a better synthetic speech can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0049] Figure 1 A schematic diagram of a process flow of a pause position prediction method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of a process for constructing a first dictionary with basic language units as content provided in an embodiment of the present application;

[0051] Figure 3 A flowchart of a training block model provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of a pause position prediction device provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of the structure of a pause position prediction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] There are currently some pause position prediction schemes. The overall idea of ​​these pause position prediction schemes is to segment the target text based on the dictionary, and then predict the pause position of the target text based on the segmentation result of the target text. The applicant has found through research that the current dictionary is a dictionary with grammatical words as the content, and grammatical words do not take into account some important features at the speech level, such as the connected reading and tone change features at the speech level (that is, when multiple words are read together, a word will lose the single word tone, that is, the tone changes). The connected reading and tone change features have a very obvious syntactic function in speech. In other words, the current pause position prediction scheme does not take it into account from the speech level and ignores the syntactic information expressed by speech changes, which makes it difficult for the existing pause position prediction schemes to predict reasonable pause positions.

[0056] In order to be able to predict reasonable pause positions, the inventors of this case conducted further research based on the above findings. Through continuous research, they finally proposed a pause position prediction scheme with better effect. The general idea of ​​this scheme is to pre-construct a dictionary with basic language units as the content (the basic language units are proposed on the basis of considering the connected reading and tone change characteristics at the phonetic level), segment the target text based on the dictionary with basic language units as the content, and then predict the pause position of the target text based on the segmentation results of the target text.

[0057] The pause position prediction method provided in the present application can be applied to an electronic device with data processing capability, which can be a server on the network side (can be a single server, or multiple servers or a server cluster), or a terminal used on the user side, such as a PC, a notebook, a PAD, a smart phone, etc. Next, the pause position prediction method provided in the present application is introduced through the following embodiments.

[0058] First embodiment

[0059] See also Figure 1 , shows a schematic flow chart of a pause position prediction method provided in an embodiment of the present application, the method may include:

[0060] Step S101: Acquire target text.

[0061] The target text is the text whose pause position is to be determined, for example, it may be the text used for speech synthesis.

[0062] Step S101: Based on a pre-established first dictionary, segment a target text to obtain a segmentation result of the target text.

[0063] The existing dictionary is a dictionary with grammatical words as its content, while the first dictionary in this embodiment is different from the existing dictionary. It is a dictionary with basic language units as its content. The first dictionary in this embodiment includes a plurality of first basic language units and a plurality of second basic language units, wherein the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read in conjunction, and the second basic language unit is a single grammatical word that is not read in conjunction with other grammatical words.

[0064] When the target text is segmented based on the first dictionary, the segmentation is achieved by matching the target text with the words in the first dictionary, that is, the part of the target text that successfully matches a word in the dictionary is taken as a word. It should be noted that there are many methods for matching the target text with the words in the first dictionary, such as the forward matching method, the reverse matching method, etc., wherein the forward matching method refers to matching according to the positive order of the sentence (from left to right), and the reverse matching method refers to matching according to the reverse order of the sentence (from right to left). Based on the pre-established first dictionary, a word sequence will be obtained after the target text is segmented, and each word in the word sequence is a first basic language unit or a second basic language unit.

[0065] Taking into account the characteristics of connected reading and tone change at the phonetic level, the present application proposes to form a first basic language unit with multiple grammatical words that are read together, and to use a single grammatical word that is not read together with other grammatical words as a second basic language unit, and to construct a first dictionary with the first basic language unit and the second basic language unit as the content, so as to avoid dividing the multiple grammatical words that are read together during word segmentation, thereby making the word segmentation result more reasonable.

[0066] When constructing the first dictionary, a variety of construction methods can be used based on the second dictionary (an existing dictionary with grammatical words as content). For example, a number of first basic language units can be constructed based on the second dictionary and preset construction rules, and the words not covered by the first basic language units in the second dictionary are used as second basic language units. The first dictionary is then composed of the constructed first basic language units and the determined second basic language units. For another example, a basic language unit construction model can be obtained by training a training word set marked with basic language units, and the basic language unit construction model is constructed using the basic language unit, and the basic language unit is constructed based on the second dictionary.

[0067] It should be noted that grammatical words, also known as grammatical words, refer to the realization of word classes in a certain grammatical framework by lexical words (lexical words refer to the smallest combination of sound and meaning that can be used independently after abstraction, generalization, and induction based on the principle of word identity). Words that have realized the possibility of word classes are grammatical words. For example, "representative" is a lexical word, and its noun and verb parts of speech are implicit and have not yet been realized. Only when "representative" realizes the possibility of word class attribution in a specific grammatical framework and becomes a noun (he is our representative) and a verb (his words represent the voice of the majority of teachers), then the noun "representative" and the verb "representative" are grammatical words.

[0068] In the prior art, when a text is segmented, the segmentation is performed based on a dictionary with grammatical words as the content, and each word in the segmentation result is a grammatical word. However, the present application segments the target text based on a first dictionary with basic language units as the content, and each word in the segmentation result is a basic language unit. Compared with the existing segmentation method, the segmentation method provided by the present application takes into account the connected reading at the phonetic level and has a larger segmentation granularity.

[0069] Exemplarily, a text is represented as ABCDEFGH, where each letter represents a character in the text. Assuming that it is segmented based on a dictionary with grammar words as its content, the segmentation results are A / B / CD / EF / G / H. Assuming that the first dictionary contains basic language units A (a single grammar word that is not read in conjunction with other grammar words), BCD (a grammar word string that is read in conjunction), EF (a grammar word string that is read in conjunction), and GH (a grammar word string that is read in conjunction), ABCDEFGH is segmented based on the first dictionary, and the segmentation results are A / BCD / EF / GH. It can be seen that when the text is segmented based on the first dictionary, multiple grammar words that are read in conjunction will not be divided apart, but multiple grammar words that are read in conjunction will be put together. This segmentation method is more reasonable. Subsequently, based on the segmentation results obtained by this segmentation method, more reasonable pause positions can be predicted.

[0070] Step S103: Determine the pause position of the target text according to the word segmentation result of the target text.

[0071] There are many ways to determine the pause position of the target text according to the word segmentation result of the target text. This embodiment provides the following two optional implementations:

[0072] The first implementation method:

[0073] Step a1: Use the pre-established chunking model to chunk the text based on the word segmentation result of the target text to obtain the chunking result corresponding to the target text.

[0074] Among them, the chunking result corresponding to the target text includes several text blocks, each text block may be a text block composed of multiple adjacent words in the word segmentation result of the target text, or may be a word in the word segmentation result of the target text. For example, the word segmentation result of the target text is A / B / C / D / E / F (each character represents a word). Assuming that B and C are combined together and E and F are combined together during chunking, the chunking result corresponding to the target text is A / BC / D / EF. It can be seen that the chunking result A / BC / D / EF includes four text blocks, among which the first text block is a word, the second text block is a text block composed of two words B and C, the third text block is a word D, and the fourth text block is a text block composed of two words E and F.

[0075] The chunking model is trained using training texts that are labeled with text chunk boundaries.

[0076] When using a pre-established chunking model to chunk text based on the word segmentation result of the target text, the word vector of each word in the word segmentation result of the target text can be first determined to obtain a word vector sequence corresponding to the target text, and then the word vector sequence corresponding to the target text is input into the pre-established chunking model to obtain a chunking result corresponding to the target text output by the chunking model. It should be noted that each word in the word segmentation result of the target text is a basic language unit (which may be the first basic language unit or the second basic language unit), and the word vector of each word is a representation vector of the basic language unit.

[0077] Step a2: determining the text block boundary in the block result corresponding to the target text as the pause position of the target text.

[0078] In order to improve the accuracy of the pause position, this embodiment provides the following second implementation method:

[0079] Step b1-a: using the pre-established chunking model, chunking the text based on the word segmentation result of the target text to obtain the chunking result corresponding to the target text.

[0080] The specific implementation method of step b1-a is the same as the specific implementation method of the above-mentioned step a1. For details, please refer to the specific implementation method of step a1, which will not be repeated in this embodiment.

[0081] Step b2-a: determine the text block boundary in the chunk result corresponding to the target text as the first candidate pause position of the target text.

[0082] Assuming that the block result corresponding to the target text is A / BCD / EFGH / IG (each letter represents a character), the position between A and B, the position between D and E, and the position between H and I are determined as the first candidate pause positions of the target text.

[0083] Step b1-b: predict the speech pause position of the speech corresponding to the target text.

[0084] Specifically, the speech features of the speech corresponding to the target text may be obtained, and the speech features of the speech corresponding to the target text may be input into a pre-established speech pause position prediction model to obtain the speech pause positions of the speech corresponding to the target text.

[0085] Step b2-b: correspond the speech pause position of the speech corresponding to the target text to the target text to obtain a second candidate pause position of the target text.

[0086] It should be noted that step b1-a and step b2-a predict the candidate pause positions of the target text from the text dimension, that is, predict the first candidate pause positions of the target text based on the target text, while step b1-b and step b2-b predict the candidate pause positions of the target text from the voice dimension, that is, predict the second candidate pause positions of the target text based on the voice corresponding to the target text.

[0087] Step b3: determining the pause position of the target text according to the first candidate pause position of the target text and the second candidate pause position of the target text.

[0088] Specifically, the same position in the first candidate pause position of the target text and the second candidate pause position of the target text is determined as the pause position of the target text. It should be noted that if the first candidate pause position of the target text and the second candidate pause position of the target text do not have the same position, it is considered that there is no pause position in the target text.

[0089] It is understandable that if a certain position is predicted as a candidate pause position by prediction methods in two dimensions, it can be determined that the position is the pause position, and a more reasonable and accurate pause position can be determined by combining the prediction results in the two dimensions.

[0090] The pause position prediction method provided in the embodiment of the present application first obtains the target text to be predicted for the pause position, then performs word segmentation on the target text based on the first dictionary with basic language units as the content, and finally predicts the pause position of the target text according to the word segmentation result of the target text. The pause position of the target text can be determined by the pause position prediction method provided by the present application. In addition, when determining the pause position, the present application takes the word segmentation result of the target text as the basis, and the word segmentation result of the target text is obtained by performing word segmentation on the target text based on the first dictionary with basic language units as the content. Since the basic language unit is constructed on the basis of considering the connected reading and tone change characteristics of the speech, therefore, performing word segmentation on the target text based on the first dictionary with basic language units as the content can avoid dividing the grammatical words that are read together, that is, the word segmentation result obtained by performing word segmentation on the first dictionary with basic language units as the content is more reasonable, and then, the pause position determined based on the reasonable word segmentation result is also more reasonable.

[0091] Second embodiment

[0092] It can be seen from the above embodiments that the pause position of the target text is predicted based on the segmentation result of the target text, and the segmentation result of the target text is obtained by segmenting the target text based on a first dictionary with basic language units as content. This embodiment focuses on introducing an implementation method for constructing a first dictionary with basic language units as content.

[0093] See also Figure 2 , showing a schematic diagram of a process of constructing a first dictionary with basic language units as content, which may include:

[0094] Step S201: Acquire a second dictionary.

[0095] The second dictionary is an existing dictionary with grammatical words as its content, and each entry in the second dictionary is a grammatical word. It should be noted that the second dictionary in this application is only used to construct the first dictionary.

[0096] Step S202: constructing a first basic language unit based on the grammatical words in the second dictionary and the parts of speech of the grammatical words in the second dictionary.

[0097] Specifically, based on the grammatical words in the second dictionary and the parts of speech of the grammatical words in the second dictionary, the process of constructing the first basic language unit may include:

[0098] Step S2021: Based on the parts of speech of the grammatical words in the second dictionary and a predetermined part-of-speech combination, the grammatical words in the second dictionary are combined to obtain a plurality of candidate basic language units.

[0099] The part-of-speech combination in this embodiment can be but is not limited to one or more of the following part-of-speech combinations: adjective + noun, pronoun + noun, quantifier + noun, noun + noun, adverb + verb, negative word + content word (such as verb, auxiliary verb, etc.), verb + tense component (in some languages, verb and tense component are separate), function word (such as conjunction, preposition, definite article, indefinite article, auxiliary verb, modal verb, etc.) + content word (such as verb, noun, pronoun, etc.), etc.

[0100] It should be noted that the part-of-speech combination is a combination of the parts-of-speech of multiple grammatical words that are read together. In the process of realizing this case, the applicant studied the phonetic characteristics of speech in many languages, and found two important features through the research, one is the connected reading area, and the other is the position where the weak reading occurs when the single word tone is lost in the syllable language, and the weak reading position where the stress is lost in the phoneme language. The above-mentioned part-of-speech combination is summarized by the applicant after analyzing the above-mentioned two features of a large number of texts in multiple languages ​​at the phonetic level. For example, through the study of the connected reading area, the position where the weak reading occurs when the single word tone is lost in the syllable language, and the weak reading position where the stress is lost in the phoneme language, it is found that when words with the part-of-speech of adjectives and words with the part-of-speech of nouns are together, connected reading tone changes usually occur, and when words with the part-of-speech of pronouns and words with the part-of-speech of nouns are together, connected reading tone changes usually occur, and so on.

[0101] When combining the grammatical words in the second dictionary, the grammatical words in the second dictionary are combined based on the above-mentioned part-of-speech combination. For example, the grammatical words with the part of speech of adjective are combined with the grammatical words with the part of speech of noun, the grammatical words with the part of speech of pronoun are combined with the grammatical words with the part of speech of noun, the grammatical words with the part of speech of adverb are combined with the grammatical words with the part of speech of verb, and so on. In this way, several grammatical word strings are obtained, each grammatical word string is taken as a candidate basic language unit, and multiple candidate basic language units can be obtained.

[0102] Step S2021: Filter out a target basic language unit from a plurality of candidate basic language units, and use the filtered out target basic language unit as a first basic language unit to be constructed.

[0103] The grammatical word strings (i.e., candidate basic language units) obtained through step S2021 are only grammatical word strings that may be read together, and the purpose of this step is to filter out grammatical word strings that actually read together from a number of grammatical word strings that may be read together, and use the filtered grammatical word strings that actually read together as the first basic language unit.

[0104] Specifically, there are multiple ways to select a target basic language unit from multiple candidate basic language units:

[0105] In a possible implementation, a candidate basic language unit that successfully matches the text in the text library may be determined from a plurality of candidate basic language units, and the candidate basic language unit that successfully matches the text in the text library may be determined as a target basic language unit.

[0106] It should be noted that by matching the candidate basic language units with the text in the text library, unreasonable basic language units (candidate basic language units that have not been successfully matched with the text in the text library) can be filtered out. Considering that unreasonable basic language units may still exist in the candidate basic language units that have been successfully matched with the text in the text library, in order to obtain more reasonable basic language units, this embodiment provides another more preferred implementation method:

[0107] Step a1: determine the confusion corresponding to multiple candidate basic language units.

[0108] Optionally, the perplexities corresponding to multiple candidate basic language units can be determined based on the language model. Specifically, for each candidate basic language unit, the probability of a word appearing in the candidate basic language unit is first determined using a pre-trained language model, and then the perplexity corresponding to the candidate basic language unit is determined based on the probability of a word appearing in the candidate basic language unit. Specifically, the perplexity corresponding to the candidate basic language unit can be calculated according to the following formula:

[0109]

[0110] Among them, Ppl(D) represents the confusion corresponding to the candidate basic language unit D, P(W1W2W3…W N-1 W N ) is the probability of a word appearing in the candidate basic language unit D, and N is the number of words in the basic language unit D.

[0111] It should be noted that the perplexity can represent the rationality of the corresponding candidate basic language unit. The smaller the perplexity, the more reasonable the corresponding candidate basic language unit is. Conversely, the larger the perplexity, the more unreasonable the corresponding candidate basic language unit is.

[0112] Step a2: determining, from a plurality of candidate basic language units, a candidate basic language unit that successfully matches the text in the text library.

[0113] It should be noted that if a candidate basic language unit successfully matches the text in the text library, it means that the candidate basic language unit exists in the text in the text library, that is, in this embodiment, "the candidate basic language unit that successfully matches the text in the text library" refers to the candidate basic language unit that exists in the text in the text library.

[0114] Step a3: According to the perplexities corresponding to the multiple candidate basic language units, a target basic language unit is selected from the candidate basic language units that successfully match the text in the text library.

[0115] Specifically, according to the perplexities corresponding to the multiple candidate basic language units, the process of selecting the target basic language unit from the candidate basic language units that successfully match the text in the text library may include:

[0116] Step a3-1: determine the minimum perplexity among the perplexities corresponding to the candidate basic language units that fail to match the text in the text library as the perplexity threshold.

[0117] In this embodiment, “a candidate basic language unit that fails to match the text in the text library” refers to a candidate basic language unit in the text that does not exist in the text library.

[0118] Step a3-2: From the candidate basic language units that successfully match the text in the text library, select the candidate basic language units whose corresponding perplexity is less than the perplexity threshold as the target basic language units.

[0119] Exemplarily, there are 6 candidate basic language units, namely u1, u2, u3, u4, u5, and u6. The perplexity corresponding to u1 is p1, the perplexity corresponding to u2 is p2, the perplexity corresponding to u3 is p3, the perplexity corresponding to u4 is p4, the perplexity corresponding to u5 is p5, and the perplexity corresponding to u6 is p6. Assuming that the candidate basic language units that fail to match the text in the text library are u1, u3, and u5, among which p1>p3>p5, p5 is determined as the perplexity threshold. Next, based on p5, the target basic language unit is screened from u2, u4, and u6. Assuming p2 <p5、p4<p5、p6> p5, then u2 and u4 are determined as the target basic language units.

[0120] Step S203: Determine each grammatical word in the second dictionary that is not covered by the first basic unit as a second basic language unit.

[0121] Exemplarily, the second dictionary includes 10 grammatical words, namely c1, c2, c3, c4, c5, c6, c7, c8, c9, and c10. Assuming that the first basic language units c2c3, c2c4, c3c4, c7c8, and c8c9 are constructed through step S2021, c1 not covered by the first basic language unit is taken as a second basic language unit, c5 not covered by the first basic language unit is taken as a second basic language unit, c6 not covered by the first basic language unit is taken as a second basic language unit, and c10 not covered by the first basic language unit is taken as a second basic language unit.

[0122] Step S204: forming a first dictionary by using the constructed first basic language units and the determined second basic language units.

[0123] Each entry in the first dictionary is a basic language unit (a grammatical word string consisting of multiple grammatical words that are read together, or a single grammatical word that is not read together with other grammatical words). For the above example, the first dictionary finally constructed contains c2c3, c2c4, c3c4, c7c8, c8c9, c1, c5, c6, and c10.

[0124] Third embodiment

[0125] The first embodiment provides two optional implementation methods of "determining the pause positions of the target text according to the word segmentation results of the target text". Regardless of which implementation method is used, there is a process of determining the chunk results corresponding to the target text, and the chunk results corresponding to the target text are determined using a chunk model obtained by pre-training. This embodiment focuses on introducing the training process of the chunk model.

[0126] See also Figure 3, showing a schematic diagram of the process of training a chunking model, which may include:

[0127] Step S301: Acquire training text from a pre-constructed training text set.

[0128] The training text set includes a plurality of training texts marked with text block boundaries, and the construction process of the training text set includes:

[0129] Step c1, obtaining a first text set.

[0130] The first text set includes multiple texts.

[0131] Step c2: segment each text in the first text set based on the first dictionary to obtain a segmentation result corresponding to each text in the first text set.

[0132] Each text in the first text set may be a sentence or multiple sentences, and each word in the word segmentation result corresponding to each text in the first text set is a basic language unit.

[0133] Optionally, for each text in the first text set, after the text is segmented based on the first dictionary to obtain the segmentation result corresponding to the text, the segmentation result corresponding to the text can be detected using a pre-established segmentation rationality detection model to obtain a detection result, wherein the detection result is used to indicate whether there is an unreasonable division boundary in the segmentation result corresponding to the text, and when there is an unreasonable division boundary, indicate which is the unreasonable division boundary. If there is an unreasonable division boundary in the segmentation result corresponding to the text, the unreasonable division boundary can be adjusted manually or by using a boundary adjustment algorithm to make the segmentation result more reasonable, and the adjusted segmentation result is used as the final segmentation result. The segmentation legitimacy detection model is trained using segmentation result samples and segmentation results after manual correction of the segmentation result samples, wherein the segmentation result samples are segmentation results obtained after segmenting the text sample based on the first dictionary.

[0134] It should be noted that when adjusting unreasonable division boundaries, the following principles can be used as a reference:

[0135] Noun phrases are usually not divided: Noun phrases can usually serve as subjects or objects, or parts of subjects and objects, such as the combination of pronouns and nouns (for example: my mother), the combination of adjectives and nouns (for example: bluesky); predicate phrases are usually not divided, such as would like, have been, may be, etc. in English; modifiers (such as adverbs, adjectives, etc.) and nouns are usually put together and not divided; adverbial phrases are usually not divided, such as time adverbial phrases (for example: on Sunday), place adverbial phrases (for example: at school) and other adverbial phrases (for example: as possible, of course, etc.); function words (such as auxiliary words, conjunctions, directional words, auxiliary words, etc.) and content words are put together.

[0136] Step c3: Obtain multiple text blocks according to the word segmentation results corresponding to each text in the first text set.

[0137] Each text block is composed of a plurality of consecutive words in the word segmentation result corresponding to a text in the first text set. Since each word in the word segmentation result corresponding to each text in the first text set is a basic language unit, each text block is a basic language unit string including a plurality of basic language units. It should be noted that this embodiment does not limit the number of text blocks obtained, nor does it limit the number of basic language units constituting the text blocks.

[0138] Exemplarily, the word segmentation result corresponding to a text in the first text set is X1 / X2 / X3 / X4 / X5 (each X represents a basic language unit), then X1X2 can be regarded as a text block, X2X3 as a text block, X3X4 as a text block, and X4X5 as a text block. X1X2X3 can be regarded as a text block, X2X3X4 as a text block, X3X4X5 as a text block, and X1X2X3X4X5 as a text block.

[0139] Step c4: Count the frequency of occurrence of each text block in the text in the second text set, and determine the text block whose occurrence frequency is greater than a preset frequency threshold as a high-frequency text block.

[0140] The second text set may include texts in the first text set, other texts, or texts in the first text set and other texts. Preferably, the second text set is a larger text set than the first text set, that is, the second text set contains more texts than the first text set. In addition, the second text set may or may not have an intersection with the first text set.

[0141] Step c5: group the word segmentation results corresponding to each text in the first text set based on the high-frequency text blocks to obtain training texts marked with text block boundaries.

[0142] Exemplarily, the word segmentation result corresponding to a text in the first text set is X1 / X2 / X3 / X4 / X5 (each X represents a basic language unit). Assuming that the determined high-frequency blocks include X2X3, the word segmentation results X1 / X2 / X3 / X4 / X5 are grouped based on the high-frequency text blocks to obtain X1 / X2X3 / X4 / X5, among which the text block boundaries are after X1, X3, and X4.

[0143] Step c6: A training text set is formed by obtaining all the training texts marked with text block boundaries.

[0144] Step S302: Segment the training text based on the first dictionary to obtain a segmentation result of the training text.

[0145] Among them, each word in the word segmentation result of the training text is a basic language unit.

[0146] Step S303: using the chunking model, chunking the text based on the word segmentation result of the training text to obtain the chunking result corresponding to the training text.

[0147] Specifically, the word vector (i.e., the representation vector of the basic language unit) of each word (i.e., the basic language unit) in the word segmentation result of the training text is determined, and the word vector of each word in the word segmentation result of the training text is input into the chunk model to obtain the chunk result corresponding to the training text.

[0148] Step S304: determining the prediction loss of the chunk model according to the chunk results corresponding to the training text and the text chunk boundaries annotated by the training text.

[0149] Step S305: updating parameters of the block model according to the prediction loss of the block model.

[0150] The block model is iteratively trained multiple times according to the above process until the training end condition is met. The model obtained after the training is completed is the final block model.

[0151] Fourth embodiment

[0152] Based on the pause position prediction device provided in the above embodiment, this embodiment provides a speech synthesis method, which includes: using the pause position prediction method provided in the above embodiment to predict the pause positions of the target text used for speech synthesis, and synthesizing the speech corresponding to the target text according to the pause positions of the target text.

[0153] When synthesizing the speech corresponding to the target text according to the pause positions of the target text, the existing speech synthesis method can be used for speech synthesis.

[0154] The pause position prediction method provided by the above embodiment can predict more reasonable pause positions for the target text, so that speech synthesis based on reasonable pause positions can obtain better speech synthesis effect.

[0155] Fifth embodiment

[0156] An embodiment of the present application further provides a pause position prediction device. The pause position prediction device provided by the embodiment of the present application is described below. The pause position prediction device described below and the pause position prediction method described above can be referenced to each other.

[0157] See also Figure 4 , shows a schematic diagram of the structure of the pause position prediction device provided in an embodiment of the present application, which may include: a text acquisition module 401, a word segmentation module 402 and a pause position prediction module 403.

[0158] The text acquisition module 401 is used to acquire the target text.

[0159] The word segmentation module 402 is used to perform word segmentation on the target text based on a pre-established first dictionary to obtain a word segmentation result of the target text.

[0160] The first dictionary includes a plurality of first basic language units and a plurality of second basic language units, wherein the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words;

[0161] The pause position prediction module 403 is used to predict the pause position of the target text according to the word segmentation result of the target text.

[0162] Optionally, the pause position prediction device provided in the embodiment of the present application may further include: a first basic language unit construction module.

[0163] The first basic language unit building block is used to:

[0164] A second dictionary is obtained, wherein the second dictionary includes a plurality of grammatical words; and a first basic language unit is constructed based on the grammatical words in the second dictionary and the parts of speech of the grammatical words in the second dictionary.

[0165] Optionally, the pause position prediction device provided in the embodiment of the present application may further include: a second basic language unit determination module.

[0166] The second basic language unit determination module is used to determine each grammatical word in the second dictionary that is not covered by the first basic language unit as a second basic language unit to obtain a plurality of second basic language units.

[0167] Optionally, the first basic language unit construction module may include: a candidate basic language unit acquisition submodule and a candidate basic language unit screening submodule.

[0168] The candidate basic language unit acquisition submodule is used to combine the grammatical words in the second dictionary based on the parts of speech of the grammatical words in the second dictionary and a predetermined part-of-speech combination to obtain multiple candidate basic language units, wherein the part-of-speech combination is a combination of the parts of speech of multiple grammatical words that are read together.

[0169] The candidate basic language unit screening submodule is used to screen out a target basic language unit from the multiple candidate basic language units, and use the screened target basic language unit as the constructed first basic language unit.

[0170] Optionally, when the candidate basic language unit screening submodule screens out a target basic language unit from the multiple candidate basic language units, it is specifically used to:

[0171] Determine the perplexities corresponding to the multiple candidate basic language units respectively; determine the candidate basic language units that successfully match the text in the text library from the multiple candidate basic language units; and filter out the target basic language unit from the candidate basic language units that successfully match the text in the text library according to the perplexities corresponding to the multiple candidate basic language units respectively.

[0172] Optionally, when the candidate basic language unit screening submodule screens out a target basic language unit from the candidate basic language units that successfully match the text in the text library according to the perplexities respectively corresponding to the multiple candidate basic language units, it is specifically used to:

[0173] The minimum perplexity among the perplexities corresponding to the candidate basic language units that fail to match the text in the text library is determined as a perplexity threshold; and from the candidate basic language units that successfully match the text in the text library, the candidate basic language units whose corresponding perplexity is less than the perplexity threshold are screened as target basic language units.

[0174] Optionally, the pause position prediction module 403 includes: a text block submodule and a pause position determination submodule.

[0175] The text chunking submodule is used to input the word segmentation result of the target text into a pre-established chunking model to obtain a chunking result corresponding to the target text, wherein the chunking model is trained using training text marked with text chunk boundaries.

[0176] The pause position determination submodule is used to determine the pause position of the target text according to the chunk result corresponding to the target text.

[0177] Optionally, when the pause position determination submodule determines the pause position of the target text according to the chunk result corresponding to the target text, it is specifically used to:

[0178] The text block boundary in the chunk result corresponding to the target text is determined as the first candidate pause position of the target text; the speech pause position of the speech corresponding to the target text is predicted, and the speech pause position is mapped to the target text to obtain the second candidate pause position of the target text; the pause position of the target text is determined according to the first candidate pause position of the target text and the second candidate pause position of the target text.

[0179] Optionally, the pause position prediction device provided in the embodiment of the present application may further include: a training text acquisition module.

[0180] Training text acquisition module, used for:

[0181] A first text set is obtained; each text in the first text set is segmented based on the first dictionary to obtain a segmentation result corresponding to each text in the first text set; a plurality of candidate text blocks are obtained according to the segmentation result corresponding to each text in the first text set, each text block is composed of a plurality of consecutive words in the segmentation result corresponding to a text in the first text set; the frequency of occurrence of each candidate text block in the text in the second text set is counted, and the candidate text block with an occurrence frequency greater than a preset frequency threshold is determined as a high-frequency text block; the segmentation result corresponding to each text in the text set is grouped based on the high-frequency text block to obtain a training text marked with text block boundaries.

[0182] The pause position prediction device provided in the embodiment of the present application first obtains the target text to be predicted for the pause position, then performs word segmentation on the target text based on the first dictionary with the basic language unit as the content, and finally predicts the pause position of the target text according to the word segmentation result of the target text. The pause position prediction device provided in the embodiment of the present application can determine the pause position of the target text. In addition, the pause position prediction device provided in the embodiment of the present application is based on the word segmentation result of the target text when predicting the pause position, and the word segmentation result of the target text is obtained by performing word segmentation on the target text based on the first dictionary with the basic language unit as the content. Since the basic language unit is constructed on the basis of considering the connected reading and tone change characteristics of the speech, therefore, performing word segmentation on the target text based on the first dictionary with the basic language unit as the content can avoid dividing the grammatical words that are read together, that is, the word segmentation result obtained by performing word segmentation on the first dictionary with the basic language unit as the content is more reasonable, and then, the pause position determined based on the reasonable word segmentation result is also more reasonable.

[0183] Sixth embodiment

[0184] An embodiment of the present application further provides a speech synthesis device, which may include: a speech synthesis module and a pause position prediction device block provided in the above embodiment.

[0185] The pause position prediction device is used to predict the pause position of the target text.

[0186] The speech synthesis module is used to synthesize the speech corresponding to the target text according to the pause positions of the target text.

[0187] The pause position prediction device can predict more reasonable pause positions for the target text, which enables the speech synthesis module to synthesize speech with better effect according to the more reasonable pause positions.

[0188] Seventh embodiment

[0189] The present application also provides a pause position prediction device, see Figure 5 , showing a schematic diagram of the structure of the pause position prediction device, the pause position prediction device may include: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one communication bus 504;

[0190] In the embodiment of the present application, the number of the processor 501, the communication interface 502, the memory 503, and the communication bus 504 is at least one, and the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;

[0191] The processor 501 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0192] The memory 503 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0193] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0194] Get the target text;

[0195] Based on a pre-established first dictionary, the target text is segmented to obtain a segmentation result of the target text, wherein the first dictionary includes a plurality of first basic language units and a plurality of second basic language units, the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words;

[0196] The pause positions of the target text are predicted according to the word segmentation result of the target text.

[0197] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0198] Eighth embodiment

[0199] The embodiment of the present application also provides a speech synthesis device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus;

[0200] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0201] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0202] The memory 503 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0203] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0204] Using the pause position prediction method provided in the above embodiment to predict the pause positions of the target text for speech synthesis;

[0205] According to the pause positions of the target text, the speech corresponding to the target text is synthesized.

[0206] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0207] Ninth embodiment

[0208] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0209] Get the target text;

[0210] Based on a pre-established first dictionary, the target text is segmented to obtain a segmentation result of the target text, wherein the first dictionary includes a plurality of first basic language units and a plurality of second basic language units, the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words;

[0211] The pause positions of the target text are predicted according to the word segmentation result of the target text.

[0212] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0213] Tenth embodiment

[0214] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0215] Using the pause position prediction method provided in the above embodiment to predict the pause positions of the target text for speech synthesis;

[0216] According to the pause positions of the target text, the speech corresponding to the target text is synthesized.

[0217] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0218] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0219] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0220] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A pause position prediction method, characterized in that: include: Get the target text; Based on a pre-established first dictionary, the target text is segmented to obtain a segmentation result of the target text, wherein the first dictionary includes a plurality of first basic language units and a plurality of second basic language units, the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words; Inputting the word segmentation result of the target text into a pre-established chunk model to obtain a chunk result corresponding to the target text, wherein the chunk model is trained using training text marked with text block boundaries; Determine a text block boundary in a chunk result corresponding to the target text as a first candidate pause position of the target text; Predicting a speech pause position of the speech corresponding to the target text, and mapping the speech pause position to the target text to obtain a second candidate pause position of the target text; The pause position of the target text is determined according to the first candidate pause position of the target text and the second candidate pause position of the target text.

2. The pause position prediction method according to claim 1, characterized in that: The process of obtaining the plurality of first basic language units includes: Acquire a second dictionary, wherein the second dictionary includes a plurality of grammatical words; A first basic language unit is constructed based on the grammatical words in the second dictionary and the parts of speech of the grammatical words in the second dictionary.

3. The pause position prediction method according to claim 1, characterized in that: The process of acquiring the plurality of second basic language units includes: Each grammatical word in the second dictionary that is not covered by the first basic language unit is determined as a second basic language unit to obtain a plurality of second basic language units.

4. The pause position prediction method according to claim 2, characterized in that: The constructing a first basic language unit based on the grammatical words in the second dictionary and the parts of speech of the grammatical words in the second dictionary includes: Based on the parts of speech of the grammatical words in the second dictionary and a predetermined part-of-speech combination, the grammatical words in the second dictionary are combined to obtain a plurality of candidate basic language units, wherein the part-of-speech combination is a combination of parts of speech of a plurality of grammatical words that are read together; A target basic language unit is screened out from the multiple candidate basic language units, and the screened out target basic language unit is used as the constructed first basic language unit.

5. The pause position prediction method according to claim 4, characterized in that: The step of selecting a target basic language unit from the plurality of candidate basic language units comprises: Determine the confusions corresponding to the multiple candidate basic language units respectively; Determine, from the plurality of candidate basic language units, a candidate basic language unit that successfully matches the text in the text library; According to the perplexities respectively corresponding to the multiple candidate basic language units, a target basic language unit is screened out from the candidate basic language units that successfully match the text in the text library.

6. The pause position prediction method according to claim 5, characterized in that: The step of selecting a target basic language unit from the candidate basic language units that successfully match the text in the text library according to the perplexities respectively corresponding to the multiple candidate basic language units includes: Determine the minimum perplexity among the perplexities corresponding to the candidate basic language units that fail to match the text in the text library as a perplexity threshold; From the candidate basic language units that successfully match the text in the text library, select the candidate basic language units whose corresponding perplexity is less than the perplexity threshold as the target basic language units.

7. The pause position prediction method according to claim 1, characterized in that: The process of obtaining the training text marked with text block boundaries includes: Obtaining a first text set; Performing word segmentation on each text in the first text set based on the first dictionary to obtain a word segmentation result corresponding to each text in the first text set; Acquire multiple text blocks according to the word segmentation results corresponding to each text in the first text set, wherein each text block is composed of a plurality of consecutive words in the word segmentation results corresponding to a text in the first text set; Counting the frequency of each text block appearing in the text in the second text set, and determining the text block whose appearance frequency is greater than a preset frequency threshold as a high-frequency text block; The word segmentation results corresponding to each text in the first text set are grouped based on the high-frequency text blocks to obtain training texts marked with text block boundaries.

8. A speech synthesis method, characterized in that: include: Using the pause position prediction method according to any one of claims 1 to 7 to predict the pause positions of a target text for speech synthesis; According to the pause positions of the target text, the speech corresponding to the target text is synthesized.

9. A pause position prediction device, characterized in that: It includes: text acquisition module, word segmentation module and pause position prediction module; The text acquisition module is used to acquire the target text; The word segmentation module is used to segment the target text based on a pre-established first dictionary to obtain a word segmentation result of the target text, wherein the first dictionary includes a plurality of first basic language units and a plurality of second basic language units, the first basic language unit is a grammatical word string composed of a plurality of grammatical words that are read together, and the second basic language unit is a single grammatical word that is not read together with other grammatical words; The pause position prediction module includes: a text block submodule and a pause position determination submodule; The text chunking submodule is used to input the word segmentation result of the target text into a pre-established chunking model to obtain a chunking result corresponding to the target text, wherein the chunking model is trained using training text marked with text chunk boundaries; The pause position determination submodule is used to determine the text block boundary in the chunk result corresponding to the target text as the first candidate pause position of the target text; predict the speech pause position of the speech corresponding to the target text, and correspond the speech pause position to the target text to obtain the second candidate pause position of the target text; and determine the pause position of the target text according to the first candidate pause position of the target text and the second candidate pause position of the target text.

10. A pause position prediction device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the pause position prediction method according to any one of claims 1 to 7.

11. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the pause position prediction method according to any one of claims 1 to 7 is implemented.

Images