Speech synthesis method and device, electronic equipment and storage medium

Abstract

The application provides a speech synthesis method and device, electronic equipment and storage medium, and relates to the technical field of speech synthesis. The method first acquires a text to be synthesized, then determines pronunciation features of the text to be synthesized according to loaded language rules, and then performs speech synthesis on the text to be synthesized according to the pronunciation features through a speech synthesis model to obtain target synthesized speech. The pronunciation features of the text to be synthesized can be determined from the dimensions of word formation characteristics, context dependence between graphemes of the text to be synthesized and the like according to the language rules, so that the pronunciation features of the text to be synthesized are more in line with language characteristics, thereby increasing the accuracy of the obtained pronunciation features of the text to be synthesized. The speech synthesis model performs speech synthesis on the text to be synthesized according to the pronunciation features with higher accuracy, thereby improving the quality of speech synthesis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech synthesis, and in particular to a speech synthesis method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Speech synthesis, also known as text-to-speech (TTS), aims to convert input text into fluent and natural output speech, and is a key technology for realizing intelligent human-computer voice interaction.

[0003] The implementation of speech synthesis technology usually includes the following steps: 1. text analysis; 2. phoneme conversion; 3. speech synthesis. The second step of phoneme conversion can also be referred to as G2P (Grapheme-to-Phoneme), which refers to the process of converting a graphical representation (Grapheme) into a corresponding pronunciation representation (Phoneme). The quality of phoneme conversion has a decisive influence on the quality of speech synthesis.

[0004] Traditional G2P methods are based on pronunciation dictionaries, and use linguistic knowledge to construct pronunciation dictionaries. G2P conversion is performed by looking up the table. This method is suitable for languages with mature learning materials, such as Chinese and English. However, small language dictionaries are lacking, and there may be a large number of OOV words (Out-of-Vocabulary, which refers to words not appearing in a dictionary or corpus). This results in low accuracy of the pronunciation representation converted by the traditional G2P method, and further results in poor quality of speech synthesis. SUMMARY

[0005] The present application provides a speech synthesis method, device, electronic device, and storage medium to solve the problem of low accuracy of the pronunciation representation converted by the prior art, and improve the quality of speech synthesis.

[0006] The present application provides a speech synthesis method, comprising:

[0007] obtaining a text to be synthesized;

[0008] loading a language rule to determine the pronunciation features of the text to be synthesized according to the language rule;

[0009] inputting the pronunciation features into a speech synthesis model to obtain a target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model;

[0010] The speech synthesis model is trained based on sample pronunciation features and sample synthesized speech corresponding to the sample pronunciation features.

[0011] The application provides a speech synthesis method, wherein a language rule is loaded to determine pronunciation features of the text to be synthesized according to the language rule, and the method comprises the following steps:

[0012] A rule table of the language rule is loaded to determine pronunciation features of the text to be synthesized according to the rule table, wherein the rule tables of different languages have the same format.

[0013] The application provides a speech synthesis method, wherein the rule table comprises pronunciation units and corresponding phonemes.

[0014] The pronunciation features of the text to be synthesized are determined according to the rule table, and the method comprises the following steps:

[0015] The graphemes of the text to be synthesized are matched according to the rule table to obtain target pronunciation sub-strings of the graphemes and corresponding target pronunciation units.

[0016] The phonemes of the target pronunciation sub-strings are determined according to the phonemes corresponding to the target pronunciation units, so that the phoneme sequence of the graphemes is obtained according to the phonemes of each target pronunciation sub-string.

[0017] The application provides a speech synthesis method, wherein the phonemes of the target pronunciation sub-strings are determined according to the phonemes corresponding to the target pronunciation units, and the method comprises the following steps:

[0018] A candidate phoneme set of the target pronunciation sub-string and at least two phoneme selection rules corresponding to the candidate phoneme set are obtained according to the phoneme set corresponding to the target pronunciation unit.

[0019] When the target pronunciation sub-string meets the target phoneme selection rule, the candidate phoneme corresponding to the target phoneme selection rule in the candidate phoneme set is determined as the phoneme of the target pronunciation sub-string.

[0020] The application provides a speech synthesis method, wherein after the candidate phoneme set of the target pronunciation sub-string and at least two phoneme selection rules corresponding to the candidate phoneme set are obtained according to the phoneme set corresponding to the target pronunciation unit, the method further comprises the following steps:

[0021] A neighboring pronunciation sub-string of the target pronunciation sub-string is determined, and the type of the neighboring pronunciation sub-string is obtained according to the rule table.

[0022] The type of the neighboring pronunciation sub-string is matched according to the phoneme selection rule, so that whether the target pronunciation sub-string meets the phoneme selection rule is judged.

[0023] The application provides a speech synthesis method, wherein the graphemes are matched according to the rule table to obtain target pronunciation sub-strings of the graphemes and corresponding target pronunciation units, and the method comprises the following steps:

[0024] moving from the first end to the second end of the character element to obtain a candidate pronunciation substring;

[0025] According to the rule table, the candidate pronunciation substring is matched to obtain a corresponding target pronunciation unit, and it is determined that the candidate pronunciation substring is a target pronunciation substring of the character element.

[0026] According to the present application, a speech synthesis method is provided. After moving from the first end to the second end of the character element to obtain a candidate pronunciation substring, the method further comprises:

[0027] According to the rule table, the candidate pronunciation substring is matched to obtain a corresponding target pronunciation unit, and it is determined that the candidate pronunciation substring is a target pronunciation substring of the character element.

[0028] According to the present application, a speech synthesis method is provided. After moving from the first end to the second end of the character element to obtain a candidate pronunciation substring, the method further comprises:

[0029] According to the target pronunciation substring, the to-be-processed substring of the character element is determined, and the candidate pronunciation substring is updated by moving from the first end to the second end of the to-be-processed substring.

[0030] According to the present application, a speech synthesis method is provided. After obtaining the phoneme sequence of the character element, the method further comprises:

[0031] According to the language rule, the phoneme sequence of the character element is divided to obtain a syllable sequence, and the stress mark of the syllable sequence is obtained.

[0032] According to the phoneme sequence, the syllable sequence and the stress mark, the pronunciation feature of the character element is obtained.

[0033] According to the present application, a speech synthesis method is provided. According to the language rule, the phoneme sequence of the character element is divided to obtain a syllable sequence, which comprises:

[0034] According to the language rule, if it belongs to a first type of language, a syllable rule corresponding to the character element is determined to divide the phoneme sequence of the character element according to the syllable rule to obtain a syllable sequence; or,

[0035] According to the language rule, if it belongs to a second type of language, the phoneme sequence is input into a syllable division model to obtain a syllable sequence corresponding to the phoneme sequence output by the syllable division model; wherein the syllable division model is trained based on a sample phoneme sequence and a sample syllable sequence corresponding to the sample phoneme sequence.

[0036] According to the present application, a speech synthesis method is provided. The stress mark of the syllable sequence is obtained, which comprises:

[0037] determining a stress rule corresponding to the grapheme according to the language rule belonging to the first type language, so as to stress mark the syllable sequence according to the stress rule; or

[0038] inputting the syllable sequence into a stress prediction model to obtain stress marking of the syllable sequence output by the stress prediction model according to the language rule belonging to the second type language; wherein the stress prediction model is obtained by training based on sample syllable sequences and sample stress marking corresponding to sample phoneme sequences.

[0039] According to the speech synthesis method provided by the application, before loading the rule table of the language rule, the method further comprises:

[0040] determining a training strategy of the speech synthesis model;

[0041] adjusting the standard phonetic notation according to the language characteristics and the training strategy of the speech synthesis model to obtain a corresponding phoneme system;

[0042] constructing a rule table of the language rule according to the phoneme system.

[0043] The application further provides a speech synthesis device, comprising:

[0044] a text acquisition module, configured to acquire a text to be synthesized;

[0045] a pronunciation feature acquisition module, configured to load a language rule to determine pronunciation features of the text to be synthesized according to the language rule;

[0046] a speech synthesis module, configured to input the pronunciation features into a speech synthesis model to obtain target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model;

[0047] wherein the speech synthesis model is obtained by training based on sample pronunciation features and sample synthesized speech corresponding to the sample pronunciation features.

[0048] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the speech synthesis method of any of the above when executing the computer program.

[0049] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the speech synthesis method of any of the above.

[0050] The speech synthesis method, device, electronic equipment and storage medium provided by the present application first acquire a text to be synthesized, then determine the pronunciation features of the text to be synthesized according to loaded language rules, and then perform speech synthesis on the text to be synthesized according to the pronunciation features through a speech synthesis model to obtain target synthesized speech. The method can determine the pronunciation features of the text to be synthesized from the dimensions of word formation characteristics and context dependence between graphemes of the text to be synthesized according to the language rules, so that the pronunciation features of the text to be synthesized are more in line with language characteristics, thereby increasing the accuracy of the obtained pronunciation features of the text to be synthesized. The speech synthesis model performs speech synthesis on the text to be synthesized according to the pronunciation features with higher accuracy, which can improve the quality of speech synthesis. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 is a flowchart of the speech synthesis method provided by the present application;

[0053] Figure 2 is a schematic diagram of matching processing of a sub-string according to a rule table in the speech synthesis method provided by the present application;

[0054] Figure 3 is a structural schematic diagram of the speech synthesis device provided by the present application;

[0055] Figure 4 is a structural schematic diagram of the electronic equipment provided by the present application. DETAILED DESCRIPTION

[0056] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, but not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0057] In speech synthesis technology, the input of a speech synthesis model is pronunciation features such as phonemes, syllables and stress. The speech synthesis model performs reasoning based on the pronunciation features to obtain target synthesized speech, which can also be called audio. Therefore, in order to realize text-to-speech conversion, text information needs to be converted into pronunciation features first, and then the pronunciation features are modeled. As can be seen, an accurate pronunciation feature acquisition method can effectively improve the prediction effect of the speech synthesis model.

[0058] The existing method of converting text information into pronunciation features mainly includes the following types:

[0059] According to the text information, a high-accuracy dictionary is used for direct query to realize direct conversion from text information to pronunciation features; this requires the high-accuracy dictionary to accurately label phonemes, syllables, tones and stress, and is suitable for languages such as Chinese and English with mature learning materials, but it is difficult to find such a high-accuracy dictionary for small languages with relatively scarce learning materials, resulting in high cost; in addition, the dictionary is difficult to cover emerging words such as compound words and derived words or some unpopular words, and therefore cannot handle OOV words;

[0060] The text information is input into a pronunciation prediction model to obtain the pronunciation features corresponding to the text information output by the pronunciation prediction model; wherein the pronunciation prediction model is obtained by training a small amount of fine-labeled data of grapheme to phoneme mapping; this method has less dependence on high-accuracy dictionaries and can better cope with the conversion of OOV words such as derived words and compound words; however, during the training of the speech synthesis model, the internationally accepted phonetic transcription system is generally not used, but some phonetic symbols are combined or distinguished based on the training strategy, thereby forming a customized phoneme system, so even if the mapping relationship between the customized phoneme system of individual languages and the internationally accepted phonetic transcription system is relatively clear, a large amount of manpower is still needed to map and verify to obtain a small amount of fine-labeled data of grapheme to phoneme mapping; and in order to improve the accuracy of the pronunciation prediction model, it is also necessary to continuously enrich the fine-labeled data of the pronunciation prediction model during actual use to bottom out the effect of the pronunciation prediction model;

[0061] The existing high-accuracy dictionary is used for direct query and the pronunciation prediction model is used for prediction, if the grapheme in the text information is an OOV word, the pronunciation prediction model is used for prediction, otherwise the existing high-accuracy dictionary is used for direct query; however, the phoneme systems of the high-accuracy dictionary and the pronunciation prediction model are different, which may cause pronunciation jumps, prosodic breaks and other problems in synthesized speech, reducing the user experience.

[0062] Therefore, the speech synthesis method, device, electronic equipment and storage medium are provided. Figures 1-4 The speech synthesis method, device, electronic equipment and storage medium are provided.

[0063] Figure 1 The speech synthesis method provided by the present application is shown in the flowchart as shown in Figure 1 The method comprises the following steps:

[0064] S1, obtaining text to be synthesized;

[0065] S2, load a language rule to determine pronunciation features of the text to be synthesized according to the language rule;

[0066] S3, input the pronunciation features into a speech synthesis model to obtain target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model;

[0067] The speech synthesis model is trained based on sample pronunciation features and sample synthesized speech corresponding to the sample pronunciation features.

[0068] Specifically, the speech synthesis method provided in the embodiment of the present application has a speech synthesis device as its execution subject. The device can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, a tablet, etc., which is not limited here.

[0069] Firstly, step S1 is performed to obtain text to be synthesized. The text to be synthesized refers to text that needs to be used to generate target synthesized speech. The text to be synthesized is the text content of the generated target synthesized speech.

[0070] Then, step S2 is performed to determine the language of the text to be synthesized. According to the language of the text to be synthesized, a corresponding language rule is loaded to map the text to be synthesized according to the language rule to determine the pronunciation features of the text to be synthesized. The language rule refers to a set of language knowledge used to convert text in a language into corresponding pronunciation features according to the characteristics of the language.

[0071] The pronunciation features refer to a set of parameterized speech features representing the acoustic properties of phonemes in the pronunciation process. For example, the pronunciation features can include phoneme sequences, syllable sequences, tones, and stress, etc.

[0072] Next, step S3 is performed to input the pronunciation features into a speech synthesis model, which can be a large language model (LLM). The speech synthesis model can perform feature coding, context modeling, etc. on the pronunciation features such as phoneme sequences, syllable sequences, tones, and stress, etc. to generate acoustic feature parameters. The acoustic feature parameters are converted into target synthesized speech waveforms using a vocoder model, so as to obtain target synthesized speech corresponding to the text to be synthesized.

[0073] In addition, the speech synthesis model can be trained by sample pronunciation features and sample synthesized speech corresponding to the sample pronunciation features. The sample pronunciation features can be determined by mapping sample text to be synthesized according to the language rule corresponding to the sample text to be synthesized. The sample synthesized speech is target speech data corresponding to the sample pronunciation features, which can be real audio collected from a corpus, or audio feature representation extracted in advance through digital signal processing, etc.

[0074] A feasible training scheme for the initial synthesis model can include:

[0075] The sample pronunciation feature is input into the initial synthesis model to obtain a synthesis result output by the initial synthesis model, then a loss function value is calculated according to the synthesis result and the sample synthesized speech, and finally the model parameters of the initial synthesis model are updated according to the loss function value; the input process and the calculation process are iteratively executed until the loss function converges or a preset iteration number is reached, to obtain the speech synthesis model. The preset iteration number can be set as needed, and is not specifically limited here.

[0076] The speech synthesis method provided in the embodiment of the application first acquires the text to be synthesized, then determines the pronunciation feature of the text to be synthesized according to the loaded language rule, and then performs speech synthesis on the text to be synthesized according to the pronunciation feature through the speech synthesis model to obtain the target synthesized speech. According to the language rule, the pronunciation feature of the text to be synthesized can be determined from the dimensions of word formation characteristics, context dependence between graphemes of the text to be synthesized, etc., so that the pronunciation feature of the text to be synthesized is more in line with the language characteristics, thereby increasing the accuracy of the obtained pronunciation feature of the text to be synthesized. The speech synthesis model performs speech synthesis on the text to be synthesized according to the pronunciation feature with higher accuracy, which can improve the quality of speech synthesis.

[0077] Moreover, when the text to be synthesized is a small language, the pronunciation feature of the text to be synthesized is determined through the corresponding language rule, which can effectively reduce the dependence on the small language dictionary, and can also well process the conversion of OOV words such as derived words and compound words, reduce the demand for manual annotation, and thereby reduce the research and development cost of language synthesis of small languages.

[0078] The pronunciation feature of the text to be synthesized is determined through the corresponding language rule, which has little dependence on the graphics processing unit (GPU), can reduce the computational power consumption for converting the text to be synthesized into the pronunciation feature, and thereby can perform the conversion between the text to be synthesized and the pronunciation feature on a low-computational-power platform, while reducing the time overhead and improving the overall efficiency of speech synthesis.

[0079] On the basis of the above embodiment, the loaded language rule is used to determine the pronunciation feature of the text to be synthesized according to the language rule, which includes:

[0080] The rule table of the language rule is loaded to determine the pronunciation feature of the text to be synthesized according to the rule table; wherein the formats of the rule tables of different languages are the same.

[0081] Specifically, the rule table refers to a parameterized rule set describing the correspondence between the text and the pronunciation feature of the language according to the language characteristics.

[0082] For example, the language of the text to be synthesized can be determined first based on the character set. Then, a rule table with the same language rule identifier can be selected from the rule resource library. The rule table is then serialized into a binary file, which enables the rule table to be loaded correctly and quickly. The loaded binary file is then deserialized to obtain a data structure that can be directly executed or searched. Based on the rule matching method, the text to be synthesized is matched according to the rule table to determine the pronunciation features of the text to be synthesized.

[0083] The format of a rule table refers to a standardized data structure that describes the correspondence between text and pronunciation features of a language. It can standardize the recording method, field composition, field meaning, and parsing method to ensure that rule tables of different languages ​​can be parsed and executed by the same rule matching method.

[0084] In this embodiment of the invention, by separating the rule table and the rule matching method through a resource-based strategy and standardizing the format of the rule table, a multilingual processing framework can be realized. Under this processing framework, when the language of the text to be synthesized changes, only the corresponding rule table needs to be replaced, and the pronunciation features of the text to be synthesized can be determined without adjusting the processing logic of the rule matching method.

[0085] Meanwhile, under this processing framework, when adding a new language to be synthesized text that can be supported, only the corresponding rule table needs to be added, without having to repeatedly develop the corresponding rule matching method, which can improve the development efficiency of the speech synthesis system for the new language from an engineering perspective.

[0086] Based on the above embodiments, before loading the rule table for language rules, the method further includes:

[0087] Determine the training strategy for the speech synthesis model;

[0088] The standard phonetic symbols are adjusted according to the characteristics of the language and the training strategy of the speech synthesis model to obtain the corresponding phoneme system;

[0089] A rule table for the corresponding language rules is constructed based on the phoneme system.

[0090] Specifically, language characteristics refer to the set of linguistic features of a particular language when its written form is transformed into phonetic features. For example, language characteristics can be obtained by language experts summarizing the characteristics of a language; language characteristics may include phonetic units, types of phonetic units, and phonemes, etc.

[0091] Standard phonetic symbols can also be called International Phonetic Symbols, International Standard Phonetic Symbols, etc.

[0092] The phoneme system refers to a set of phoneme symbols established for a speech synthesis model for uniformly representing pronunciation features of text. It can be understood that the phoneme system is customized, and some standard alphabets can be combined or distinguished.

[0093] It should be noted that after obtaining the phoneme system of the language, the type of each pronunciation unit and its corresponding phoneme set and pronunciation unit can be determined according to the characteristics of the language, and the rules for determining the phonemes of the pronunciation unit from the phoneme set are determined, and then the rule table corresponding to the language rules is summarized, so that the basic form of the rule table is [key, prop, values, rules].

[0094] Among them, key is a pronunciation unit, which is the smallest meaningful written symbol unit converted into a phoneme of a target language. It can be understood that all pronunciation units corresponding to the language are covered in the rule table, and the pronunciation unit can be a single letter or a string of letters.

[0095] prop is the type of pronunciation unit, which is used to represent the attribute category of pronunciation unit. The types of pronunciation units in general languages include vowel pronunciation units (V) and consonant pronunciation units (C), and other types may also be present in individual small languages.

[0096] values is the phoneme set corresponding to the pronunciation unit, which is the set of all phonemes that the pronunciation unit can map to, and can include one phoneme or multiple phonemes.

[0097] rules is the rule corresponding to the pronunciation unit. It can be understood that in the case of unique mapping between pronunciation unit and phoneme, there is no need to set the corresponding rule; in the case that the pronunciation unit can map to multiple phonemes, at least one rule for each phoneme that the pronunciation unit can map to can be generated according to the position of the current pronunciation unit and the types of its adjacent pronunciation units, and the rules corresponding to the pronunciation unit are summarized.

[0098] The contents in the original format of the rule table are variable length, and when the rule table is serialized, the positions and lengths (offsets) of the key, prop, values, and rules components of each row of data of the rule table need to be accurately recorded to ensure that the computer can quickly and accurately parse and extract these variable length data items. The serialization format is shown in the following table:

[0099]

[0100] The format of the rule table is shown in the following table:

[0101]

[0102] Among them, # is the end identifier, and _ is the position identifier of the pronunciation unit.

[0103] In the first row, a is a pronunciation unit; V is the type of a is vowel; aa / ae is the phoneme that a can map includes aa and ae; #_C, C_Ce / others is that when a meets #_C and C_Ce, the pronunciation unit of a is aa, and when a meets others, the pronunciation unit of a is ae. #_C means that a is at the beginning of a sentence, and the next pronunciation unit of a is a consonant; C_Ce means that the previous pronunciation unit of a is a consonant, the next pronunciation unit of a is a consonant, and the letter after that is e; others means other cases except the above.

[0104] In the second row, e is a pronunciation unit; V is the type of e is vowel; ^ / ee is the phoneme that can be mapped includes ^ and ee; VC_# / others is that when e meets VC_#, the pronunciation unit of e is ^ ( ^ means no pronunciation), and when e meets others, the pronunciation unit of e is ee; VC_# means that the previous pronunciation unit of e is a consonant, the pronunciation unit before that is a vowel, and e is at the end of a sentence; others means other cases except the above.

[0105] In the embodiment of the present application, the corresponding phoneme system is obtained by adjusting the standard phonetic symbols according to the language characteristics and the training strategy of the speech synthesis model, which can increase the accuracy of the pronunciation features of the obtained text to be synthesized, form a phoneme system matched with the speech synthesis model, and reduce the training complexity of the speech synthesis model.

[0106] On the basis of the above embodiment, the rule table includes pronunciation units and corresponding phonemes;

[0107] According to the rule table, the pronunciation features of the text to be synthesized are determined, including:

[0108] According to the rule table, the graphemes of the text to be synthesized are matched to obtain target pronunciation substrings of the graphemes and corresponding target pronunciation units;

[0109] According to the phonemes corresponding to the target pronunciation units, the phonemes of the target pronunciation substrings are determined, so as to obtain the phoneme sequence of the graphemes according to the phonemes of each target pronunciation substring.

[0110] Specifically, first, the text to be synthesized can be obtained, and the text to be synthesized is preprocessed to determine the graphemes of the text to be synthesized. The grapheme can also be called a character, a character, etc., which is the smallest meaningful writing symbol unit.

[0111] Then, the substring of the graphemes of the text to be synthesized is matched with the pronunciation units in the rule table to obtain the target pronunciation substrings and the corresponding target pronunciation units that are successfully matched, so as to determine the phonemes of the target pronunciation substrings from the phonemes that the target pronunciation units can map.

[0112] Then the matching processing step can be repeated to obtain the phonemes of each target pronunciation sub-string of the grapheme, and arrange the corresponding phonemes according to the position of each target pronunciation sub-string in the grapheme to obtain the phoneme sequence of the grapheme.

[0113] For example, the matching processing step is repeated to obtain the phoneme aa of the target pronunciation sub-string a of the grapheme ache, the phoneme ch of the target pronunciation sub-string ch, and the phoneme not pronounced of the target pronunciation sub-string e, thereby obtaining the phoneme sequence aa ch of the grapheme.

[0114] Compared with the end-to-end mapping of the text to be synthesized and the pronunciation feature, in the embodiment of the present application, the target pronunciation sub-string of the grapheme is obtained by matching the grapheme based on the rule table, and the phoneme sequence of the grapheme and the pronunciation feature of the text to be synthesized are determined step by step, which can process OOV words, reduce the deviation in the mapping process, and improve the accuracy of the obtained pronunciation feature of the text to be synthesized.

[0115] As described above, the phonemes that the target pronunciation unit can map can be one phoneme or multiple phonemes.

[0116] On the basis of the above embodiment, the determination of the phonemes of the target pronunciation sub-string according to the phonemes corresponding to the target pronunciation unit comprises:

[0117] obtaining a candidate phoneme set of the target pronunciation sub-string and at least two phoneme selection rules corresponding to the phoneme set according to the phonemes corresponding to the target pronunciation unit;

[0118] In the case where the target pronunciation sub-string satisfies the target phoneme selection rule, the candidate phoneme corresponding to the target phoneme selection rule in the candidate phoneme set is determined as the phoneme of the target pronunciation sub-string.

[0119] Specifically, the candidate phoneme refers to a specific phoneme in the multiple phonemes that the target pronunciation unit can map. The candidate phoneme set of the target pronunciation sub-string includes at least two candidate phonemes, and each candidate phoneme corresponds to at least one target phoneme selection rule.

[0120] Firstly, the candidate phoneme set of the target pronunciation sub-string can be obtained according to the phoneme set that the target pronunciation unit can map, and the phoneme selection rule of the target pronunciation sub-string can be obtained according to the rule associated with the target pronunciation unit. Then, the candidate phoneme selection rule can be selected in turn to match the target pronunciation sub-string, and if the target pronunciation sub-string satisfies the condition limit of the phoneme selection rule, the phoneme selection rule is determined as the target phoneme selection rule, and the candidate phoneme indicated by the phoneme selection rule is determined as the phoneme of the target pronunciation sub-string.

[0121] In the embodiment of the present application, the phonemes of the target pronunciation substring of the grapheme can be determined according to the context condition of the grapheme in the text to be synthesized by the rule associated with the target pronunciation unit in the rule table, and then the phoneme sequence of the grapheme is determined according to the phonemes of each target pronunciation substring, thereby increasing the accuracy of the obtained phoneme sequence of the grapheme.

[0122] In addition, in some embodiments, the case that a target pronunciation substring satisfies multiple rules at the same time can be avoided based on the logical mutual exclusion design of the rules associated with the target pronunciation unit of the rule table, so that after it is determined that the target pronunciation substring satisfies the current phoneme selection rule, it is no longer determined whether the target pronunciation substring satisfies the subsequent phoneme selection rule, thereby reducing the runtime delay of determining the phonemes of the target pronunciation substring.

[0123] On the basis of the above-mentioned embodiment, after the candidate phoneme set of the target pronunciation substring and the corresponding at least two phoneme selection rules are obtained according to the phoneme set corresponding to the target pronunciation unit, the method further comprises:

[0124] determining the adjacent pronunciation substring of the target pronunciation substring, and obtaining the type of the adjacent pronunciation substring according to the rule table;

[0125] matching the type of the adjacent pronunciation substring according to the phoneme selection rule to determine whether the target pronunciation substring satisfies the phoneme selection rule.

[0126] Specifically, the adjacent pronunciation substring also includes the first two pronunciation substrings and the last two pronunciation substrings of the target pronunciation substring, and is not limited to the first pronunciation substring and the last pronunciation substring.

[0127] Firstly, it can be determined whether the target pronunciation substring satisfies the current phoneme selection rule according to the position of the target pronunciation substring, and if so, the adjacent pronunciation substring of the target pronunciation substring is determined according to the current phoneme selection rule. The way of determining the adjacent pronunciation substring of the target pronunciation substring is basically the same as the way of determining the target pronunciation substring of the grapheme, which will not be described here.

[0128] Then, the type of the adjacent pronunciation substring is obtained according to the type of the pronunciation unit corresponding to the adjacent pronunciation substring, it is determined whether the type of the adjacent pronunciation substring conforms to the current phoneme selection rule, if so, it is determined that the target pronunciation substring satisfies the current phoneme selection rule, and the phoneme selection rule satisfied by the target pronunciation substring is obtained; if not, it is determined that the target pronunciation substring does not satisfy the current phoneme selection rule, and the determination step of whether the target pronunciation substring satisfies the phoneme selection rule is repeated until the phoneme selection rule satisfied by the target pronunciation substring is obtained.

[0129] Compared with the method of selecting corresponding phonemes according to single characters, in the embodiment of the application, the type of adjacent pronunciation sub-strings is used to participate in the matching of the target pronunciation sub-string and the phoneme selection rule, so that the phonemes of the target pronunciation sub-string are selected based on the phonetic structure features of the grapheme, thereby improving the accuracy of the selected phonemes of the target pronunciation sub-string.

[0130] On the basis of the above-mentioned embodiment, the matching processing of the grapheme according to the rule table obtains the target pronunciation sub-string of the grapheme and the corresponding target pronunciation unit, and comprises:

[0131] The candidate pronunciation sub-string is obtained by moving from the first end to the second end of the grapheme;

[0132] In the case where the matching processing of the candidate pronunciation sub-string according to the rule table obtains the corresponding target pronunciation unit, the candidate pronunciation sub-string is determined as the target pronunciation sub-string of the grapheme.

[0133] Specifically, the first end of the grapheme can also be referred to as the grapheme termination end, which can be the last character of the grapheme, and the second end of the character can also be referred to as the grapheme starting end, which can be the first character of the grapheme. For example, the first end of the grapheme ache is e, and the second end is a. The candidate pronunciation sub-string refers to a sub-string extracted from the grapheme for matching judgment by the rule table.

[0134] In the matching processing of the candidate pronunciation sub-string according to the rule table, if the candidate pronunciation sub-string matches a pronunciation unit of the rule table successfully, the candidate pronunciation sub-string is a pronunciation sub-string; if the candidate pronunciation sub-string fails to match a pronunciation unit of the rule table, the candidate pronunciation sub-string is not a pronunciation sub-string; the candidate pronunciation sub-string can be updated, and the matching judgment step is repeated until the pronunciation sub-string is obtained.

[0135] In the embodiment of the application, the candidate pronunciation sub-string is obtained by moving from the first end to the second end of the grapheme, which can realize mapping of the pronunciation sub-string of the text to be synthesized according to the natural language reading order to obtain the corresponding phonemes, avoid additional computational overhead caused by unordered search and backtracking, and reduce the runtime delay of determining the phoneme sequence of the grapheme.

[0136] On the basis of the above-mentioned embodiment, after the candidate pronunciation sub-string is obtained by moving from the first end to the second end of the grapheme, the method further comprises:

[0137] In the case where the matching processing of the candidate pronunciation sub-string according to the rule table does not obtain the corresponding target pronunciation unit, the candidate pronunciation sub-string is updated by moving from the first end to the second end of the candidate pronunciation sub-string character by character.

[0138] Taking the initial candidate pronunciation substring ache of the grapheme ache as an example, the initial candidate pronunciation substring ache is matched according to the rule table, and in the case that no corresponding target pronunciation unit is obtained, the first end e of the initial candidate pronunciation substring ache is moved to the second end a character by character, and a candidate pronunciation substring ach is obtained by updating.

[0139] In the matching processing of the initial candidate pronunciation substring ache according to the rule table, a double-pointer mode can be used to move the first end to the second end of the candidate pronunciation substring character by character, and update the candidate pronunciation substring.

[0140] In the embodiment of the application, in the case that the candidate pronunciation substring is not matched successfully, shortening the candidate range character by character can automatically adapt to the length of the pronunciation unit of different languages, which is dynamically changed, so as to better improve the processing logic of the rule matching method, and when the language of the text to be synthesized is changed, only the corresponding rule table needs to be replaced to provide a basis.

[0141] In addition, shortening the candidate range character by character can preferentially identify and match the longest interpretable pronunciation unit, which is more in line with the real language pronunciation mechanism.

[0142] As shown in Figure 2 In some embodiments, when the candidate pronunciation substring is matched according to the rule table, the items of the rule table can be arranged in descending order according to the length of the pronunciation unit of the rule table, so that the longer candidate pronunciation substring obtained by traversing the substring from back to front can be preferentially matched with the longer pronunciation unit, reducing invalid matching judgment and reducing the runtime delay of determining the target pronunciation substring. Figure 2 In the embodiment, the left end of testwo is the starting end of the substring, and the right end of testwo is the terminal end of the substring.

[0143] On the basis of the above-mentioned embodiments, after determining that the candidate pronunciation substring is the target pronunciation substring of the grapheme, the method further comprises:

[0144] According to the target pronunciation substring, the method further comprises:

[0145] Specifically, the to-be-processed substring refers to the remaining substring after the matched and confirmed target pronunciation substring is removed from the grapheme corresponding to the text to be synthesized. Taking the grapheme ache as an example, after the target pronunciation substrings a and ch are confirmed, the candidate pronunciation substring e can be obtained by updating.

[0146] In the embodiment of the present application, after obtaining the corresponding target pronunciation substring through each matching process, the target pronunciation substring is removed from the input sequence to obtain a to-be-processed substring. Subsequently, the to-be-processed substring can be matched according to the rule table to obtain the corresponding target pronunciation unit and the rule, and then the phoneme selection rule of the to-be-processed substring is determined, and the phoneme of the target pronunciation substring is selected using the phonetic structure feature of the to-be-processed substring, which can further improve the accuracy of the selected phoneme of each target pronunciation substring.

[0147] On the basis of the above-mentioned embodiment, after obtaining the phoneme sequence of the grapheme, the method further comprises:

[0148] According to the language rule, the phoneme sequence of the grapheme is divided to obtain a syllable sequence, and the stress mark of the syllable sequence is obtained.

[0149] According to the phoneme sequence, the syllable sequence and the stress mark, the pronunciation feature of the grapheme is obtained.

[0150] In the embodiment of the present application, according to the language rule, different ways can be selected to divide the phoneme sequence of the grapheme to obtain a syllable sequence, and different ways can be used to determine the stress mark of the syllable sequence, which can improve the flexibility of the acquisition method of the syllable sequence and the stress mark, so as to select a suitable acquisition method and reduce the consumption of computing power.

[0151] On the basis of the above-mentioned embodiment, the division of the phoneme sequence of the grapheme to obtain a syllable sequence comprises:

[0152] According to the language rule belonging to a first type of language, the syllable rule corresponding to the grapheme is determined, so that the phoneme sequence of the grapheme is divided to obtain a syllable sequence according to the syllable rule; or,

[0153] According to the language rule belonging to a second type of language, the phoneme sequence is input into a syllable division model to obtain a syllable sequence corresponding to the phoneme sequence output by the syllable division model; wherein the syllable division model is trained based on a sample phoneme sequence and a sample syllable sequence corresponding to the sample phoneme sequence.

[0154] The first type of language is a language in which syllables have obvious rules. According to the language, the corresponding syllable rule can be determined. For example, the default consonant-vowel structure can be used for syllable division, or a special syllable division rule can be used for syllable division. The specific setting method of the special syllable division rule can be designed according to the characteristics of the specific language, which is not limited in the embodiment of the present application.

[0155] The second type of language is a language in which syllables do not have obvious rules.

[0156] A feasible training scheme for the initial segmentation model can include:

[0157] The sample phoneme sequence is input into the initial segmentation model to obtain a segmentation result output by the initial segmentation model, then a loss function value is calculated according to the segmentation result and the sample syllable sequence, and finally the model parameters of the initial segmentation model are updated according to the loss function value; the input process and the calculation process are iteratively executed until the loss function converges or a preset iteration number is reached, and a speech segmentation model is obtained. The preset iteration number can be set as needed, and is not specifically limited here.

[0158] On the basis of the above embodiment, the stress annotation of the syllable sequence is obtained, including:

[0159] According to the language rule belonging to the first type language, the stress rule corresponding to the grapheme is determined to annotate the stress of the syllable sequence according to the stress rule; or,

[0160] According to the language rule belonging to the second type language, the syllable sequence is input into a stress prediction model to obtain the stress annotation of the syllable sequence output by the stress prediction model; wherein the stress prediction model is trained based on sample syllable sequences and corresponding sample stress annotations of sample phoneme sequences.

[0161] The first type language is a language with obvious rules for stress. The stress of the syllable at the target position can be annotated according to the corresponding stress rule determined according to the language. The specific correspondence between the language and the target position can be designed according to the characteristics of the specific language, which is not further limited in the embodiments of the present application.

[0162] The second type language is a language without obvious rules for stress.

[0163] A feasible training scheme for the initial prediction model can include:

[0164] The sample syllable is input into the initial prediction model to obtain a prediction result output by the initial prediction model, then a loss function value is calculated according to the prediction result and the sample stress annotation, and finally the model parameters of the initial prediction model are updated according to the loss function value; the input process and the calculation process are iteratively executed until the loss function converges or a preset iteration number is reached, and a speech prediction model is obtained. The preset iteration number can be set as needed, and is not specifically limited here.

[0165] In some embodiments, when the phoneme sequence of the grapheme carries the stress annotation, the stress annotation carried in the phoneme sequence is directly used. The priority of the stress annotation carried in the phoneme sequence is higher than the stress annotation determined by the stress rule and the stress prediction model.

[0166] The pronunciation feature acquisition device provided by the present application is described below. The pronunciation feature acquisition device described below can be referred to the pronunciation feature acquisition method described above.

[0167] Figure 3 FIG. 1 is a structural schematic diagram of a speech synthesis device provided by the present application. As shown in FIG. 1, the device comprises the following modules: Figure 3

[0168] A text acquisition module 31 is configured to acquire a text to be synthesized.

[0169] A pronunciation feature acquisition module 32 is configured to load a language rule to determine a pronunciation feature of the text to be synthesized according to the language rule.

[0170] A speech synthesis module 33 is configured to input the pronunciation feature into a speech synthesis model to obtain a target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model.

[0171] The speech synthesis model is trained based on a sample pronunciation feature and a sample synthesized speech corresponding to the sample pronunciation feature.

[0172] In the above embodiment, the pronunciation feature acquisition module 32 is specifically configured to load a rule table of the language rule to determine the pronunciation feature of the text to be synthesized according to the rule table. The rule tables of different languages have the same format.

[0173] In the above embodiment, the rule table comprises a pronunciation unit and a corresponding phoneme.

[0174] The pronunciation feature acquisition module 32 is specifically configured to:

[0175] match the grapheme of the text to be synthesized according to the rule table to obtain a target pronunciation substring of the grapheme and a corresponding target pronunciation unit;

[0176] determine a phoneme of the target pronunciation substring according to the phoneme corresponding to the target pronunciation unit, and obtain a phoneme sequence of the grapheme according to the phoneme of each target pronunciation substring.

[0177] In the above embodiment, the pronunciation feature acquisition module 32 is specifically configured to:

[0178] obtain a candidate phoneme set of the target pronunciation substring and at least two phoneme selection rules corresponding to the target pronunciation unit according to the phoneme set corresponding to the target pronunciation unit;

[0179] In the case that the target pronunciation substring satisfies the target phoneme selection rule, determine a candidate phoneme corresponding to the target phoneme selection rule in the candidate phoneme set as the phoneme of the target pronunciation substring.​

[0180] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to:

[0181] determine adjacent pronunciation sub-strings of the target pronunciation sub-string, and obtain the types of the adjacent pronunciation sub-strings according to the rule table;

[0182] match the types of the adjacent pronunciation sub-strings according to the phoneme selection rule to determine whether the target pronunciation sub-string satisfies the phoneme selection rule.

[0183] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to:

[0184] obtain a candidate pronunciation sub-string by moving from the first end to the second end of the grapheme;

[0185] when the candidate pronunciation sub-string is matched according to the rule table to obtain a corresponding target pronunciation unit, determine that the candidate pronunciation sub-string is the target pronunciation sub-string of the grapheme.

[0186] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to: when the candidate pronunciation sub-string is not matched according to the rule table to obtain a corresponding target pronunciation unit, move character by character from the first end to the second end of the candidate pronunciation sub-string, and update the candidate pronunciation sub-string.

[0187] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to: determine a to-be-processed sub-string of the grapheme according to the target pronunciation sub-string, and move from the first end to the second end of the to-be-processed sub-string to update the candidate pronunciation sub-string.

[0188] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to:

[0189] divide the phoneme sequence of the grapheme according to the language rule to obtain a syllable sequence, and annotate the stress of the syllable sequence;

[0190] obtain the pronunciation feature of the grapheme according to the phoneme sequence, the syllable sequence, and the stress annotation.

[0191] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to:

[0192] when the language rule belongs to a first type of language, determine a syllable rule corresponding to the grapheme to divide the phoneme sequence of the grapheme according to the syllable rule to obtain a syllable sequence; or

[0193] According to the language rule belonging to the second type language, the phoneme sequence is input into a syllable division model to obtain a syllable sequence corresponding to the phoneme sequence output by the syllable division model; wherein the syllable division model is obtained based on training of a sample phoneme sequence and a sample syllable sequence corresponding to the sample phoneme sequence.

[0194] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to:

[0195] According to the language rule belonging to the first type language, a stress rule corresponding to the grapheme is determined to mark the stress of the syllable sequence according to the stress rule; or,

[0196] According to the language rule belonging to the second type language, the syllable sequence is input into a stress prediction model to obtain the stress marking of the syllable sequence output by the stress prediction model; wherein the stress prediction model is obtained based on training of a sample syllable sequence and a sample stress marking corresponding to the sample phoneme sequence.

[0197] On the basis of the above-mentioned embodiments, the pronunciation feature acquisition module 32 is specifically configured to:

[0198] Determine the training strategy of the speech synthesis model;

[0199] According to the language characteristics and the training strategy of the speech synthesis model, the standard phonetic notation is adjusted to obtain a corresponding phoneme system;

[0200] According to the phoneme system, a rule table of the corresponding language rule is constructed.

[0201] Figure 4 An example of an entity structure schematic diagram of an electronic device is shown in Figure 4 The electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can invoke the logical instructions in the memory 430 to execute a speech synthesis method, which includes: obtaining a text to be synthesized; loading a language rule to determine the pronunciation features of the text to be synthesized according to the language rule; inputting the pronunciation features into a speech synthesis model to obtain a target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model; wherein the speech synthesis model is obtained based on training of a sample pronunciation feature and a sample synthesized speech corresponding to the sample pronunciation feature.

[0202] Further, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as standalone products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0203] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the speech synthesis method provided by the above-mentioned methods, and the method comprises: obtaining a text to be synthesized; loading a language rule to determine pronunciation features of the text to be synthesized according to the language rule; inputting the pronunciation features into a speech synthesis model to obtain target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model; wherein the speech synthesis model is obtained based on sample pronunciation features and sample synthesized speech corresponding to the sample pronunciation features.

[0204] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the speech synthesis method provided by the above-mentioned methods, and the method comprises: obtaining a text to be synthesized; loading a language rule to determine pronunciation features of the text to be synthesized according to the language rule; inputting the pronunciation features into a speech synthesis model to obtain target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model; wherein the speech synthesis model is obtained based on sample pronunciation features and sample synthesized speech corresponding to the sample pronunciation features.

[0205] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0206] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0207] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A speech synthesis method, characterized in that, include: Obtain the text to be synthesized; Load language rules to determine the pronunciation features of the text to be synthesized based on the language rules; The pronunciation features are input into the speech synthesis model to obtain the target synthesized speech corresponding to the text to be synthesized, which is output by the speech synthesis model. The speech synthesis model is trained based on sample pronunciation features and the synthesized speech of the corresponding sample pronunciation features. The loading of language rules, to determine the pronunciation features of the text to be synthesized according to the language rules, includes: Load the rule table of the language rules to determine the pronunciation features of the text to be synthesized based on the rule table; wherein the rule table of different languages ​​has the same format; The rule table includes pronunciation units and corresponding phonemes; The pronunciation features of the text to be synthesized are determined according to the rule table, including: The character elements of the text to be synthesized are matched according to the rule table to obtain the target pronunciation substring and the corresponding target pronunciation unit of the character element; The phonemes of the target pronunciation substring are determined based on the phonemes corresponding to the target pronunciation unit, so as to obtain the phoneme sequence of the character based on the phonemes of each target pronunciation substring; The step of determining the phonemes of the target pronunciation substring based on the phonemes corresponding to the target pronunciation unit includes: Based on the phoneme set corresponding to the target pronunciation unit, obtain the candidate phoneme set of the target pronunciation substring and the corresponding selection rules for at least two phonemes; After obtaining the candidate phoneme set and at least two corresponding phoneme selection rules for the target pronunciation substring based on the phoneme set corresponding to the target pronunciation unit, the method further includes: Determine the adjacent pronunciation substrings of the target pronunciation substring, and obtain the type of the adjacent pronunciation substrings according to the rule table; The type of the adjacent pronunciation substring is matched according to the phoneme selection rule to determine whether the target pronunciation substring satisfies the phoneme selection rule; If the target pronunciation substring satisfies the target phoneme selection rule, the candidate phonemes in the candidate phoneme set corresponding to the target phoneme selection rule are determined as the phonemes of the target pronunciation substring.

2. The speech synthesis method according to claim 1, characterized in that, The step of matching the character elements according to the rule table to obtain the target pronunciation substring and the corresponding target pronunciation unit of the character elements includes: The candidate pronunciation substring is obtained by moving from the first end to the second end of the character element; If the candidate pronunciation substring is matched according to the rule table to obtain the corresponding target pronunciation unit, then the candidate pronunciation substring is determined to be the target pronunciation substring of the character.

3. The speech synthesis method according to claim 2, characterized in that, After obtaining the candidate pronunciation substring by moving from the first end to the second end of the character element, the method further includes: The candidate pronunciation substring is matched according to the rule table. If no corresponding target pronunciation unit is found, the candidate pronunciation substring is updated by moving character by character from the first end to the second end.

4. The speech synthesis method according to claim 2, characterized in that, After determining that the candidate pronunciation substring is the target pronunciation substring of the character, the method further includes: The candidate pronunciation substring is updated by determining the substring to be processed based on the target pronunciation substring and moving from the first end to the second end of the substring to be processed.

5. The speech synthesis method according to claim 1, characterized in that, After obtaining the phoneme sequence of the character, the method further includes: The phoneme sequence of the character is divided into syllable sequences according to the language rules, and the stress marks of the syllable sequences are also obtained. The pronunciation features of the character are obtained based on the phoneme sequence, the syllable sequence, and the stress annotation.

6. The speech synthesis method according to claim 5, characterized in that, The step of dividing the phoneme sequence of the character according to the language rules to obtain a syllable sequence includes: Based on the language rules, if the character belongs to the first type of language, determine the syllable rules corresponding to the character, and then divide the phoneme sequence of the character according to the syllable rules to obtain a syllable sequence; or, According to the language rule, it belongs to the second type of language. The phoneme sequence is input into the syllable segmentation model to obtain the syllable sequence corresponding to the phoneme sequence output by the syllable segmentation model. The syllable segmentation model is trained based on the sample phoneme sequence and the sample syllable sequence corresponding to the sample phoneme sequence.

7. The speech synthesis method according to claim 5, characterized in that, Obtaining the stress annotation of the syllable sequence includes: Based on the language rules, if the language belongs to the first type of language, determine the stress rules corresponding to the character elements, and then mark the stress of the syllable sequence according to the stress rules; or... According to the language rule, it belongs to the second type of language. The syllable sequence is input into the stress prediction model to obtain the stress label of the syllable sequence output by the stress prediction model. The stress prediction model is trained based on the sample syllable sequence and the sample stress label corresponding to the sample phoneme sequence.

8. The speech synthesis method according to claim 1, characterized in that, Before loading the rule table for language rules, the method further includes: Determine the training strategy for the speech synthesis model; The standard phonetic symbols are adjusted according to the characteristics of the language and the training strategy of the speech synthesis model to obtain the corresponding phoneme system; A rule table for the corresponding language rules is constructed based on the phoneme system.

9. A speech synthesis device, characterized in that, include: The text acquisition module is used to acquire the text to be synthesized. The pronunciation feature acquisition module is used to load language rules to determine the pronunciation features of the text to be synthesized based on the language rules. The speech synthesis module is used to input the pronunciation features into the speech synthesis model to obtain the target synthesized speech corresponding to the text to be synthesized output by the speech synthesis model; The speech synthesis model is trained based on sample pronunciation features and the synthesized speech of the corresponding sample pronunciation features. The pronunciation feature acquisition module is specifically used to load a rule table of language rules, so as to determine the pronunciation features of the text to be synthesized according to the rule table; wherein, the rule table of different languages ​​has the same format; The rule table includes pronunciation units and corresponding phonemes; The pronunciation feature acquisition module is specifically used for: The character elements of the text to be synthesized are matched according to the rule table to obtain the target pronunciation substring and the corresponding target pronunciation unit of the character element; The phonemes of the target pronunciation substring are determined based on the phonemes corresponding to the target pronunciation unit, so as to obtain the phoneme sequence of the character based on the phonemes of each target pronunciation substring; The pronunciation feature acquisition module is specifically used for: Based on the phoneme set corresponding to the target pronunciation unit, obtain the candidate phoneme set of the target pronunciation substring and the corresponding selection rules for at least two phonemes; Determine the adjacent pronunciation substrings of the target pronunciation substring, and obtain the type of the adjacent pronunciation substrings according to the rule table; The type of the adjacent pronunciation substring is matched according to the phoneme selection rule to determine whether the target pronunciation substring satisfies the phoneme selection rule; If the target pronunciation substring satisfies the target phoneme selection rule, the candidate phonemes in the candidate phoneme set corresponding to the target phoneme selection rule are determined as the phonemes of the target pronunciation substring.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the speech synthesis method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the speech synthesis method as described in any one of claims 1 to 8.

Images