A pronunciation prediction method and related device

Abstract

The present application discloses a pronunciation prediction method and related devices. First, the text to be synthesized is segmented to obtain a segmentation sequence. For the first category of words in the segmentation sequence, the pronunciation information of the first category of words is determined based on a preset corpus resource library. For the second category of words other than the first category of words in the segmentation sequence, their pronunciation categories are determined based on the part-of-speech information of each word in the segmentation sequence, and their pronunciation information is determined based on the pronunciation information determination method corresponding to their pronunciation category. In the present application, combined with the corpus resource library and the preset pronunciation information determination method corresponding to each pronunciation category, it is possible to cover the pronunciation information determination in various situations. Therefore, it is possible to accurately determine the pronunciation information of each word in the text to be synthesized, thereby improving the effect of speech synthesis.

Description

Technical Field

[0001] The present application relates to the field of speech synthesis technology, and in particular to a pronunciation prediction method and related devices. Background Art

[0002] Current speech synthesis systems first segment the text to be synthesized into individual words, then predict the pronunciation of each word to determine its pronunciation information. Finally, synthesis is performed based on this pronunciation information. The accuracy of each word's pronunciation information plays a crucial role in the final synthesis.

[0003] Existing pronunciation prediction schemes generally adopt the mode of searching corpus resource library, i.e. pre-building corpus resource library, which includes the pronunciation information of multiple words. When the words after word segmentation are predicted for pronunciation, the pronunciation information of the words can be directly determined from the corpus resource library. However, for some words after word segmentation, the pronunciation information included in the corpus resource library may be multiple, in which case, the pronunciation information of these words cannot be accurately determined. In addition, there are also some words that may not be included in the corpus. In this case, existing pronunciation prediction schemes can adopt other means (such as skipping the silent section of the word or filling the silent section, adopting any means in the machine learning methods such as reading by letter, using C45 decision tree to predict pronunciation) to predict the pronunciation information of these words, but the forms of these words are diverse, and the means adopted may not be able to accurately determine the pronunciation information of some forms of words.

[0004] Therefore, how to provide a pronunciation prediction method to accurately determine the pronunciation information of each word in the text to be synthesized, thereby improving the effect of speech synthesis, has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, this application provides a pronunciation prediction method and related devices to accurately determine the pronunciation information of each word in the text to be synthesized, thereby improving the effect of speech synthesis. The specific solution is as follows:

[0006] The first aspect of the present application provides a pronunciation prediction method, comprising:

[0007] Performing word segmentation processing on the text to be synthesized to obtain a word segmentation sequence, wherein the word segmentation sequence includes multiple words;

[0008] Determining pronunciation information of the first category of words based on a preset corpus resource library;

[0009] Determining the pronunciation category of each second-category word based on the part-of-speech information of each word in the segmented word sequence, where the second-category words are other words in the segmented word sequence except the first-category words;

[0010] For each second-category word, the pronunciation information of the second-category word is determined based on a preset pronunciation information determination method corresponding to the pronunciation category of the second-category word.

[0011] In a possible implementation, determining the pronunciation information of the first category of words based on a preset corpus resource library includes:

[0012] Looping through each word in the word segmentation sequence, searching the corpus resource library for pronunciation information corresponding to the word, and obtaining a pronunciation information search result for the word, wherein the pronunciation information search result for the word is used to indicate the pronunciation information corresponding to the word in the corpus resource library;

[0013] The first category of words and the pronunciation information of the first category of words are determined based on the pronunciation information search results of each word; the first category of words are words in the word segmentation sequence for which unique pronunciation information can be determined based on the corpus resource library, and the pronunciation information determined from the corpus resource library is the pronunciation information of the first category of words.

[0014] In a possible implementation, determining the pronunciation category of each second-category word based on the part-of-speech information of each word in the word segmentation sequence includes:

[0015] Looping through each word in the word segmentation sequence, searching the corpus resource library for part-of-speech information corresponding to the word, and obtaining a part-of-speech information search result for the word, wherein the part-of-speech information search result is used to indicate the part-of-speech information corresponding to the word in the corpus resource library;

[0016] Based on the pronunciation information search results of each word, generating a pronunciation information sequence and a pronunciation information attention mask sequence;

[0017] Based on the part-of-speech information search results of each word, generating a part-of-speech information sequence and a part-of-speech information attention mask sequence;

[0018] The pronunciation category of each of the second-category words is determined based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence.

[0019] In a possible implementation, determining the pronunciation category of each second-category word based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence includes:

[0020] Inputting the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model;

[0021] Among them, the part-of-speech and pronunciation category prediction model uses a training word segmentation sequence, a training pronunciation information sequence, a training pronunciation information attention mask sequence, a training part-of-speech information sequence, and a training part-of-speech information attention mask sequence as training samples, and uses a pronunciation category label sequence corresponding to the training pronunciation information sequence and a part-of-speech information label sequence corresponding to the training part-of-speech information sequence as sample labels, and is trained with the part-of-speech prediction result output by the part-of-speech and pronunciation category prediction model being close to the part-of-speech information label sequence and the output pronunciation category prediction result being close to the pronunciation category label sequence as training objectives.

[0022] In one possible implementation, inputting the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model includes:

[0023] Performing feature extraction processing on the word segmentation sequence to obtain a word segmentation feature vector sequence;

[0024] Performing feature extraction processing on the pronunciation information sequence to obtain a pronunciation information feature vector sequence;

[0025] Performing feature extraction processing on the part-of-speech information sequence to obtain a part-of-speech information feature vector sequence;

[0026] Performing feature fusion processing on the word segmentation feature vector sequence, the pronunciation information feature vector sequence, the part-of-speech information feature vector sequence, the pronunciation information attention mask sequence, and the part-of-speech information attention mask sequence to obtain a fused feature vector sequence;

[0027] Inputting the fused feature vector sequence into the part-of-speech and pronunciation category prediction model to obtain the part-of-speech prediction results of each word in the word segmentation sequence and the pronunciation category prediction results of each word in the word segmentation sequence output by the part-of-speech and pronunciation category prediction model;

[0028] The pronunciation category of each word in the second category is determined from the pronunciation category prediction results of each word in the word segmentation sequence.

[0029] In a possible implementation, determining the pronunciation information of the second category words based on a preset pronunciation information determination method corresponding to the pronunciation category of the second category words includes:

[0030] Obtaining the pronunciation prediction modules corresponding to the preset pronunciation categories;

[0031] Determining a target pronunciation prediction module from each pronunciation prediction module, wherein the target pronunciation prediction module is a pronunciation prediction module corresponding to the pronunciation category of the second category words among the pronunciation prediction modules;

[0032] The text or the second-category words are provided to the target pronunciation prediction module to obtain pronunciation information of the second-category words output by the target pronunciation prediction module.

[0033] A second aspect of the present application provides a pronunciation prediction device, comprising:

[0034] A word segmentation processing unit, configured to perform word segmentation processing on the text to be synthesized to obtain a word segmentation sequence, wherein the word segmentation sequence includes a plurality of words;

[0035] A first pronunciation information determining unit, configured to determine pronunciation information of a first category of words based on a preset corpus resource library;

[0036] a pronunciation category determination unit, configured to determine the pronunciation category of each second category word based on the part-of-speech information of each word in the segmented word sequence, where the second category words are words other than the first category words in the segmented word sequence;

[0037] The second pronunciation information determining unit is configured to determine the pronunciation information of each second-category word based on a preset pronunciation information determining method corresponding to the pronunciation category of the second-category word.

[0038] In a possible implementation, the first pronunciation information determining unit is specifically configured to:

[0039] Looping through each word in the word segmentation sequence, searching the corpus resource library for pronunciation information corresponding to the word, and obtaining a pronunciation information search result for the word, wherein the pronunciation information search result for the word is used to indicate the pronunciation information corresponding to the word in the corpus resource library;

[0040] The first category of words and the pronunciation information of the first category of words are determined based on the pronunciation information search results of each word; the first category of words are words in the word segmentation sequence for which unique pronunciation information can be determined based on the corpus resource library, and the pronunciation information determined from the corpus resource library is the pronunciation information of the first category of words.

[0041] In a possible implementation, the pronunciation category determination unit includes:

[0042] a part-of-speech information search result determining unit, configured to loop through each word in the word segmentation sequence, search the corpus resource library for part-of-speech information corresponding to the word, and obtain a part-of-speech information search result for the word, wherein the part-of-speech information search result for the word is used to indicate the part-of-speech information corresponding to the word in the corpus resource library;

[0043] A pronunciation information sequence and pronunciation information attention mask sequence generating unit, configured to generate a pronunciation information sequence and a pronunciation information attention mask sequence based on the pronunciation information search result of each word;

[0044] A part-of-speech information sequence and part-of-speech information attention mask sequence generating unit, configured to generate a part-of-speech information sequence and a part-of-speech information attention mask sequence based on the part-of-speech information search result of each word;

[0045] The pronunciation category determination subunit is used to determine the pronunciation category of each second-category word based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence.

[0046] In a possible implementation, the pronunciation category determination subunit is specifically used to:

[0047] Inputting the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model;

[0048] Among them, the part-of-speech and pronunciation category prediction model uses a training word segmentation sequence, a training pronunciation information sequence, a training pronunciation information attention mask sequence, a training part-of-speech information sequence, and a training part-of-speech information attention mask sequence as training samples, and uses a pronunciation category label sequence corresponding to the training pronunciation information sequence and a part-of-speech information label sequence corresponding to the training part-of-speech information sequence as sample labels, and is trained with the part-of-speech prediction result output by the part-of-speech and pronunciation category prediction model being close to the part-of-speech information label sequence and the output pronunciation category prediction result being close to the pronunciation category label sequence as training objectives.

[0049] In a possible implementation, the pronunciation category determination subunit includes:

[0050] A first feature extraction processing unit is used to perform feature extraction processing on the word segmentation sequence to obtain a word segmentation feature vector sequence;

[0051] A second feature extraction processing unit is used to perform feature extraction processing on the pronunciation information sequence to obtain a pronunciation information feature vector sequence;

[0052] a third feature extraction processing unit, configured to perform feature extraction processing on the part-of-speech information sequence to obtain a part-of-speech information feature vector sequence;

[0053] a feature fusion processing unit, configured to perform feature fusion processing on the word segmentation feature vector sequence, the pronunciation information feature vector sequence, the part-of-speech information feature vector sequence, the pronunciation information attention mask sequence, and the part-of-speech information attention mask sequence to obtain a fused feature vector sequence;

[0054] a part-of-speech prediction result and pronunciation category prediction result determination unit, configured to input the fused feature vector sequence into the part-of-speech and pronunciation category prediction model, and obtain the part-of-speech prediction result of each word in the word segmentation sequence and the pronunciation category prediction result of each word in the word segmentation sequence output by the part-of-speech and pronunciation category prediction model;

[0055] The determination subunit is configured to determine the pronunciation category of each word in the second category from the pronunciation category prediction results of each word in the word segmentation sequence.

[0056] In a possible implementation, the second pronunciation information determining unit is specifically configured to:

[0057] Obtaining the pronunciation prediction modules corresponding to the preset pronunciation categories;

[0058] Determining a target pronunciation prediction module from each pronunciation prediction module, wherein the target pronunciation prediction module is a pronunciation prediction module corresponding to the pronunciation category of the second category words among the pronunciation prediction modules;

[0059] The text or the second-category words are provided to the target pronunciation prediction module to obtain pronunciation information of the second-category words output by the target pronunciation prediction module.

[0060] A third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the pronunciation prediction method of the first aspect or any implementation of the first aspect.

[0061] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0062] The memory is used to store computer programs;

[0063] The processor is configured to execute the computer program so that the electronic device can implement the pronunciation prediction method of the first aspect or any implementation manner of the first aspect.

[0064] In a fifth aspect, the present application provides a computer storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the pronunciation prediction method of the first aspect or any implementation of the first aspect.

[0065] By means of the above technical solution, the present application provides a pronunciation prediction method and related devices, which first perform word segmentation processing on the text to be synthesized to obtain a word segmentation sequence. For the first category of words in the word segmentation sequence, the pronunciation information of the first category of words is determined based on a preset corpus resource library. For the second category of words other than the first category of words in the word segmentation sequence, their pronunciation categories are determined based on the part-of-speech information of each word in the word segmentation sequence, and their pronunciation information is determined based on the pronunciation information determination method corresponding to their pronunciation category. In the present application, combined with the corpus resource library and the preset pronunciation information determination method corresponding to each pronunciation category, it is possible to cover the pronunciation information determination in various situations. Therefore, it is possible to accurately determine the pronunciation information of each word in the text to be synthesized, thereby improving the effect of speech synthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0067] Figure 1 A flowchart of a pronunciation prediction method provided in this application;

[0068] Figure 2 This is a schematic diagram of an example of a Japanese forward word segmentation algorithm provided in this application;

[0069] Figure 3 A schematic diagram of a word segmentation sequence obtained by a Japanese forward word segmentation algorithm provided in this application;

[0070] Figure 4 A schematic diagram of the principle of a part-of-speech and pronunciation category prediction model provided in this application;

[0071] Figure 5 A schematic diagram of the structure of a pronunciation prediction device provided in this application;

[0072] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0073] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0074] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0075] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0076] Current speech synthesis systems first segment the text to be synthesized into individual words, then predict the pronunciation of each word to determine its pronunciation information. Finally, synthesis is performed based on this pronunciation information. The accuracy of each word's pronunciation information plays a crucial role in the final synthesis.

[0077] Existing pronunciation prediction solutions generally rely on searching a corpus resource library. This involves pre-building a corpus resource library that contains pronunciation information for multiple words. When predicting the pronunciation of a segmented word, the pronunciation information can be directly determined from the corpus resource library. However, for a given word, the corpus resource library may contain multiple pronunciations, making it impossible to accurately determine the pronunciation of the word.

[0078] In addition, with the development of the Internet and the trend of globalization, various cultural phenomena and hot events have emerged one after another, bringing with them the emergence of various new words, hot words, and buzzwords. For example, the term "ChatGPT" brought about by the popularity of large models frequently appears on various social networks. These new words, hot words, and buzzwords are often not included in the corpus resource library in a timely manner and become unregistered (Out Of Vocabulary, OOV) words. However, these new words, hot words, and buzzwords are widely used in the field of speech synthesis technology. For example, in some news and self-media applications, speech synthesis is often used to publish content containing various new words, hot words, and buzzwords. In this case, if the word after word segmentation is an OOV word, it is impossible to directly determine the pronunciation information of the word from the corpus resource library.

[0079] Currently, any of the following schemes can be used to determine the pronunciation information of the unregistered word: skip the word and keep it silent or fill it with a silent segment; read it by letter; use machine learning methods such as C45 decision tree to predict pronunciation. However, the scheme of skipping the word and keeping it silent or filling it with a silent segment will cause the speech synthesis results to have problems such as missing words, abrupt word meanings, and unnatural and coherent speech; the scheme of reading by letter will cause the pronunciation of some words (such as iflytek, etc.) in the speech synthesis results to be unauthentic; the scheme of using machine learning methods such as C45 decision tree to predict pronunciation only considers the mapping relationship between the word's letter combination and the pronunciation phoneme. When words that are pronounced by letter appear (such as NBA, ChatGPT, etc.), strange pronunciations such as spelling will appear. It can be seen that the above schemes cannot adapt to the diverse forms of unregistered words, and will result in the pronunciation information of some forms of unregistered words being unable to be accurately determined.

[0080] To address the above issues, embodiments of the present application provide a pronunciation prediction method. The pronunciation prediction method of the present application can be performed by an electronic device such as a mobile phone, computer, server, or server cluster, or a specially designed intelligent device, or by a pronunciation prediction device provided in the electronic device or intelligent device. The pronunciation prediction device can be implemented using software, hardware, or a combination of both.

[0081] The pronunciation prediction method of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0082] Reference Figure 1 , Figure 1 A flowchart of a pronunciation prediction method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, a pronunciation prediction method provided in an embodiment of the present application may include the following steps:

[0083] S101 , performing word segmentation processing on the text to be synthesized to obtain a word segmentation sequence, wherein the word segmentation sequence includes multiple words.

[0084] In the present application, as an implementable method, the text to be synthesized can be an initial text. Specifically, the initial text can be a text containing information such as text, numbers, symbols, abbreviations, etc. in any language. The initial text can be obtained by any means such as direct input, image and text recognition, or database import. Considering that the initial text may be non-standard, in the present application, the initial text can be regularized to obtain the regularized text as the text to be synthesized.

[0085] In this application, regularization can be used to normalize the initial text. Regularization mainly includes two aspects: 1. Regularizing non-standard characters according to the protocol standard of the speech synthesis system, including but not limited to converting full-width characters to their corresponding half-width characters; merging and escaping mathematical symbols, punctuation, and vowel symbols in Arabic and other scripts. 2. Based on the pronunciation language, the text transcription system completes the conversion of special symbols into local language characters. For example, in the English language synthesis scenario, the initial text "The price of the shirt is $5." will be transcribed as "The price of the shirt is five dollars."

[0086] In this application, as an implementation method, any word segmentation method such as rule-based resource library matching, dynamic programming forward and backward word segmentation algorithm, CRF (Conditional Random Field) word segmentation cost prediction can be used to segment the text to obtain a word segmentation sequence. This application does not impose any restrictions on this.

[0087] For ease of understanding, the following word segmentation processing examples are given in this application:

[0088] Example 1: For example, in Indo-European languages ​​such as English and Spanish, after word segmentation of "The price of the shirt is five dollars," the resulting word sequence can be: ["The", "price", "of", "the", "shirt", "is", "five", "dollars"];

[0089] Example 2: For non-space segmentation languages ​​represented by Sino-Tibetan languages ​​such as Chinese and Japanese, after the segmentation of "Nanjing Yangtze River Bridge", the resulting segmentation sequence can be ["Nanjing", "City", "Yangtze River", "Bridge"].

[0090] Example 3: Reference Figure 2 , Figure 2This is an example of a Japanese forward word segmentation algorithm provided in the embodiment of the present application. Based on this algorithm, the word segmentation sequence obtained is as follows Figure 3 shown.

[0091] S102, determining pronunciation information of the first category of words based on a preset corpus resource library;

[0092] In the present application, the first category of words are words in the word segmentation sequence for which unique pronunciation information can be determined based on the corpus resource library, and the pronunciation information determined from the corpus resource library is the pronunciation information of the first category of words.

[0093] In this application, the preset corpus resource library includes multiple words and specific information of each word. The specific information of a word includes the pronunciation information of the word and the part-of-speech information of the word, wherein the pronunciation information can be the pronunciation phoneme stress and the part-of-speech information can be the part-of-speech abbreviation.

[0094] In this application, the abbreviations for parts of speech are as follows:

[0095]

[0096]

[0097] S103 . Determine the pronunciation category of each second-category word based on the part-of-speech information of each word in the word segmentation sequence, where the second-category words are words other than the first-category words in the word segmentation sequence.

[0098] In this application, pronunciation categories include but are not limited to polyphones, derivatives, compound words, abbreviations, words + abbreviations, and camel case conjunctions.

[0099] S104: For each word in the second category, determine the pronunciation information of the word in the second category based on a preset pronunciation information determination method corresponding to the pronunciation category of the word in the second category.

[0100] In the present application, a method for determining pronunciation information corresponding to each pronunciation category may be preset, and the method for determining pronunciation information corresponding to each pronunciation category may be implemented using rules, algorithms, or models.

[0101] It should be noted that the above steps can determine the pronunciation information of each word in the word segmentation sequence, extract the acoustic features through the acoustic model, and then pass the acoustic features into the vocoder to output the speech synthesis audio.

[0102] The present embodiment discloses a pronunciation prediction method, which first performs word segmentation processing on the text to be synthesized to obtain a word segmentation sequence. For the first category of words in the word segmentation sequence that can determine unique pronunciation information based on a preset corpus resource library, the pronunciation information determined from the corpus resource library is the pronunciation information of the first category of words. For the second category of words other than the first category of words in the word segmentation sequence, the pronunciation category is determined based on the part-of-speech information of each word in the word segmentation sequence, and the pronunciation information is determined based on the pronunciation information determination method corresponding to its pronunciation category. In the present application, combined with the corpus resource library and the preset pronunciation information determination method corresponding to each pronunciation category, it is possible to cover the pronunciation information determination in various situations. Therefore, it is possible to accurately determine the pronunciation information of each word in the text to be synthesized, thereby improving the effect of speech synthesis.

[0103] In another embodiment of the present application, a specific implementation method for determining pronunciation information of first-category words based on a preset corpus resource library is described. The method may include the following steps:

[0104] S201: Loop through each word in the word segmentation sequence, search the corpus resource library to see whether there is pronunciation information corresponding to the word, and obtain a pronunciation information search result for the word. The pronunciation information search result for the word is used to indicate the pronunciation information corresponding to the word in the corpus resource library.

[0105] It should be noted that for each word in the word segmentation sequence, the pronunciation information search result may contain only one pronunciation information, multiple pronunciation information, or even zero pronunciation information. If the pronunciation information search result contains zero pronunciation information, it means that the word is not included in the corpus resource library. If the pronunciation information search result contains multiple pronunciation information, it means that although the word is included in the corpus resource library, the recorded pronunciation information of the word is ambiguous and needs further confirmation.

[0106] S202: Determine the first category of words and the pronunciation information of the first category of words based on the pronunciation information search results of each word; the first category of words are words in the word segmentation sequence for which unique pronunciation information can be determined based on the corpus resource library, and the pronunciation information determined from the corpus resource library is the pronunciation information of the first category of words.

[0107] In another embodiment of the present application, a specific implementation method for determining the pronunciation category of each second-category word based on the part-of-speech information of each word in the word segmentation sequence is described. The method may include the following steps:

[0108] S301: Loop through each word in the word segmentation sequence, search the corpus resource library for part-of-speech information corresponding to the word, and obtain a part-of-speech information search result for the word. The part-of-speech information search result for the word is used to indicate the part-of-speech information corresponding to the word in the corpus resource library.

[0109] It should be noted that for each word in the word segmentation sequence, the part-of-speech information search result may contain only one part-of-speech information, multiple parts-of-speech information, or even zero part-of-speech information. If the part-of-speech information search result contains zero part-of-speech information, it means that the corpus resource library does not include the word. If the part-of-speech information search result contains multiple parts-of-speech information, it means that although the corpus resource library includes the word, the recorded part-of-speech information for the word is ambiguous and needs further confirmation.

[0110] S302: Based on the pronunciation information search results of each word, generate a pronunciation information sequence and a pronunciation information attention mask sequence.

[0111] In the present application, the pronunciation information sequence includes the pronunciation information annotation results of each word in the word segmentation sequence; wherein, for a certain word, if its pronunciation information search result contains only one pronunciation information, or contains multiple pronunciation information, then the pronunciation information annotation result of the word can be its pronunciation information search result, and different pronunciation information can be separated by a preset separator (such as " / "). If its pronunciation information search result contains 0 pronunciation information, then the pronunciation information annotation result of the word can be a preset identifier (for example, OOV). The pronunciation information attention mask sequence is used to indicate the pronunciation category prediction weight of each word in the word segmentation sequence, wherein the pronunciation category prediction weight of the first category of words in the word segmentation sequence can be 0, and the pronunciation category prediction weight of the second category of words can be 1.

[0112] S303: Based on the part-of-speech information search results of each word, generate a part-of-speech information sequence and a part-of-speech information attention mask sequence; the part-of-speech information sequence includes the part-of-speech information of each word in the word segmentation sequence.

[0113] In the present application, the part-of-speech information sequence includes the part-of-speech information tagging results of each word in the word segmentation sequence; wherein, for a certain word, if its part-of-speech information search result contains only one part-of-speech information, or contains multiple part-of-speech information, then the part-of-speech information tagging result of the word can be its part-of-speech information search result, and different part-of-speech information can be separated by a preset separator (such as " / "); if its part-of-speech information search result contains 0 part-of-speech information, then the part-of-speech information tagging result of the word can be a preset identifier (for example, OOV). The part-of-speech information attention mask sequence is used to indicate the part-of-speech information prediction weight of each word in the word segmentation sequence, wherein, for a word that contains only one part-of-speech information in the part-of-speech information search result, its part-of-speech information prediction weight can be 0, and for a word that contains multiple part-of-speech information in the part-of-speech information search result or a word that contains 0 part-of-speech information, its part-of-speech information prediction weight can be 1.

[0114] For ease of understanding, taking the English text "I have read abook about chatGPT" to be synthesized as an example, word segmentation processing is performed on it, and the obtained word segmentation sequence is ["I", "have", "read", "a", "book", "about", "chatGPT"], the pronunciation information sequence can be ["((ai)1)", "((hh ac vv)1)", "((rr ii dd)1) / ((rrae dd)1)", "((ax)1) / ((ei)1)", "((bb oc kg)1)", "((ax)0)((bb ao td)1)", "OOV"], and the pronunciation information attention mask sequence can be [0, 0, 1, 1, 0, 1, 1]; the part-of-speech information sequence can be ["noun", "aux / verb", "verb", "noun / art", "noun", "prep", "OOV"], and the part-of-speech information attention mask sequence can be [0, 1, 0, 1, 0, 0, 1].

[0115] In this example, the word ["have"] has two possible part-of-speech information: 1) an auxiliary verb in the perfect tense and 2) the verb "to have." The word ["a"] has two possible pronunciation information: 1) the pronunciation of an article ((ax)1) and 2) the pronunciation of a noun ((ei)1). Since no pronunciation or part-of-speech information corresponding to the word ["chatGPT"] is found in the corpus, ["chatGPT"] is annotated "OOV" in the pronunciation information sequence and "OOV" in the part-of-speech information sequence.

[0116] S304: Determine the pronunciation category of each of the second-category words based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence.

[0117] In the present application, the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence can be input into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model; wherein the part-of-speech and pronunciation category prediction model is trained using the training word segmentation sequence, the training pronunciation information sequence, the training pronunciation information attention mask sequence, the training part-of-speech information sequence, and the training part-of-speech information attention mask sequence as training samples, the pronunciation category label sequence corresponding to the training pronunciation information sequence and the part-of-speech information label sequence corresponding to the training part-of-speech information sequence as sample labels, and the part-of-speech prediction result output by the part-of-speech and pronunciation category prediction model is close to the part-of-speech information label sequence and the pronunciation category prediction result output is close to the pronunciation category label sequence as training objectives.

[0118] In this application, the part-of-speech and pronunciation category prediction model may include but is not limited to deep learning models such as fully connected layers, convolutional neural networks, recurrent neural networks, long short-term memory neural networks, residual neural networks, and attention mechanisms.

[0119] Specifically, the step of inputting the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model may include the following steps:

[0120] S401: Perform feature extraction processing on the word segmentation sequence to obtain a word segmentation feature vector sequence.

[0121] In the present application, a word embedding process may be used to perform feature extraction on the word segmentation sequence to obtain a word segmentation feature vector sequence.

[0122] S402: Perform feature extraction processing on the pronunciation information sequence to obtain a pronunciation information feature vector sequence.

[0123] In the present application, a word embedding process may be used to perform feature extraction on the pronunciation information sequence to obtain a pronunciation information feature vector sequence.

[0124] S403: Perform feature extraction processing on the part-of-speech information sequence to obtain a part-of-speech information feature vector sequence.

[0125] In the present application, a word embedding process may be used to perform feature extraction on the part-of-speech information sequence to obtain a part-of-speech information feature vector sequence.

[0126] S404: performing feature fusion processing on the word segmentation feature vector sequence, the pronunciation information feature vector sequence, the part-of-speech information feature vector sequence, the pronunciation information attention mask sequence, and the part-of-speech information attention mask sequence to obtain a fused feature vector sequence.

[0127] For ease of understanding, the text to be synthesized is still taken as the English text "I have read a book about chatGPT" as an example. After word segmentation processing, the obtained word segmentation sequence is ["I", "have", "read", "a", "book", "about", "chatGPT"], the pronunciation information sequence can be ["((ai)1)", "((hh ac vv)1)", "((rr ii dd)1) / ((rr ae dd)1)", "((ax)1) / ((ei)1)", "((bb oc kg)1)", "((ax)0)((bb ao td)1)", "OOV"], and the pronunciation information attention mask sequence can be [0, 0, 1, 1, 0, 0, 1]; the part-of-speech information sequence can be ["noun", "aux / verb", "verb", "noun / art", "noun", "prep", "OOV"], and the part-of-speech information attention mask sequence can be [0, 1, 0, 1, 0, 0, 1]. Then the fused feature vector sequence can be as follows:

[0128] Among them, the first column can be a word segmentation feature vector sequence, the second column can be a pronunciation information feature vector sequence, and the part followed by ellipsis can contain multiple columns, corresponding to the part-of-speech information feature information vector sequence, the pronunciation information attention mask sequence, and the part-of-speech information attention mask sequence respectively.

[0129] S405: Inputting the fused feature vector sequence into the part-of-speech and pronunciation category prediction model to obtain the part-of-speech prediction results of each word in the word segmentation sequence and the pronunciation category prediction results of each word in the word segmentation sequence output by the part-of-speech and pronunciation category prediction model.

[0130] In the present application, after obtaining the fusion feature vector, the part-of-speech and pronunciation category prediction model can predict the part-of-speech prediction results of each word in the word segmentation sequence and the pronunciation category prediction results of each word in the word segmentation sequence. For example, it is predicted that the part-of-speech of the word ["have"] is the auxiliary verb aux, the part-of-speech of the word ["a"] is the article art, the part-of-speech of the word ["chatGPT"] is the noun noun, the word ["read"] is a polyphone pronunciation category, and the word ["chatGPT"] is a word + abbreviation pronunciation category. For ease of understanding, refer to Figure 4 , Figure 4 A schematic diagram of the principles of a part-of-speech and pronunciation category prediction model provided in an embodiment of the present application.

[0131] S406: Determine the pronunciation category of each word in the second category from the pronunciation category prediction results of each word in the word segmentation sequence.

[0132] In another embodiment of the present application, a specific implementation method of determining the pronunciation information of each second-category word in step S104 based on a preset pronunciation information determination method corresponding to the pronunciation category of the second-category word is described. The method may include the following steps:

[0133] S501: Obtaining a pronunciation prediction module corresponding to each preset pronunciation category.

[0134] As previously mentioned, in this application, pronunciation categories include, but are not limited to, polyphones, derivatives, compound words, abbreviations, words + abbreviations, and camel-case conjunctions. Therefore, in this application, the pronunciation prediction modules corresponding to each preset pronunciation category include, but are not limited to, a polyphone pronunciation prediction module, a derivative pronunciation prediction module, a compound word pronunciation prediction module, abbreviation pronunciation prediction module, word + abbreviation pronunciation prediction module, and camel-case conjunction pronunciation prediction module.

[0135] As mentioned above, the pronunciation information corresponding to each pronunciation category can be determined using rules, algorithms, or models. Therefore, each pronunciation prediction module can be implemented using rules, algorithms, or models. For example, the polyphonetic word pronunciation prediction module can be implemented using a model, and the word + abbreviation pronunciation prediction module can use a dynamic programming algorithm to gradually traverse the dictionary tree, forward match the current longest word, determine the segmentation position, pronounce the letter portion according to the abbreviation, and search the corpus resource library for the word portion to determine the pronunciation information.

[0136] The camel case conjunction pronunciation prediction module uses a forward maximum matching algorithm with rule adjustment and stack call depth limitation. Finally, the word is divided into multiple items, and the pronunciation information is determined by searching the corpus resource library item by item.

[0137] The derivative word pronunciation prediction module determines the pronunciation of derivative words based on a rule-based approach. Specifically, derivative words are divided into prefixes and suffixes based on their affixes: ① prefixes, such as in / compitable; ② suffixes, such as look / ing. Accordingly, prefix and suffix matching involves root variations, such as: ① doubling t / d / p, e.g., sit / ting, pad / ding; ② dropping the ending consonant e / y / and changing y to i, e.g., googl(e) / ing, happ / iness, ly / ing; ③...

[0138] For English-Chinese mixed pinyin words, you can use a dynamic programming algorithm to recursively segment words and determine pronunciation information.

[0139] It should be noted that the pronunciation prediction modules corresponding to each pronunciation category are not limited to language.

[0140] For example, the Arabic vowel recovery module: Arabic writing does not contain vowels, only consonants. Based on Transformer pronunciation prediction and combined with rule library resources, the vowels in Arabic text can be restored to obtain correct pronunciation information. In the figure, the characters above the Arabic characters are the restored vowel information.

[0141] Another example is the Korean jamo pronunciation prediction module: the pronunciation of a word in Korean is predicted by the front jamo, the back jamo and the middle Korean text. Scoreable word vectors Jamo prediction is performed on each word, which can be divided into three types: ① only the front jamo ② only the back jamo ③ both the front and back jamo exist. Based on the Transformer model prediction and dictionary rule matching, the correct pronunciation of Korean words can be completed.

[0142] S502: Determine a target pronunciation prediction module from each pronunciation prediction module, where the target pronunciation prediction module is a pronunciation prediction module corresponding to the pronunciation category of the second category words among the pronunciation prediction modules;

[0143] S503: providing the text or the second-category words to the target pronunciation prediction module to obtain pronunciation information of the second-category words output by the target pronunciation prediction module.

[0144] For example, for the word "read" in the above example, the above fused feature vector sequence can be input into the polyphonetic word pronunciation prediction model, which can predict two pronunciation prediction value vectors for "read". After softMax normalization, the result is [0.19652, 0.80348], and the corresponding pronunciation information of the word "read" is ((rr ae dd)1).

[0145] For example, the word chatGPT in the above example can be input into the word + abbreviation pronunciation prediction module. The dynamic programming algorithm gradually traverses the dictionary tree and forward matches the current longest word. The segmentation position is chat / GPT, and the last three digits are the pronunciation of the abbreviation. The pronunciation is filled in in sequence to obtain the pronunciation information of the word as ((ch ac td)1)((jhii)1)((pb ii)1)((td ii)1).

[0146] For example, the camel case conjunction "NoPainsNoGains" and the camel case nomenclature word can be input into the camel case conjunction pronunciation prediction module. The camel case conjunction pronunciation prediction module uses the forward maximum matching algorithm plus rule adjustment and limits the stack call depth. Finally, the word is divided into No / Pains / No / Gains, and the pronunciation information is filled in by searching the corpus resources item by item.

[0147] The above describes a pronunciation prediction method provided by an embodiment of the present application. The following describes a device for executing the above pronunciation prediction method.

[0148] See also Figure 5 , Figure 5 This is a structural diagram of a pronunciation prediction device provided in an embodiment of the present application. Figure 5 As shown, the pronunciation prediction device includes:

[0149] A word segmentation processing unit 11 is used to perform word segmentation processing on the text to be synthesized to obtain a word segmentation sequence, wherein the word segmentation sequence includes multiple words;

[0150] A first pronunciation information determining unit 12 is configured to determine pronunciation information of a first category of words based on a preset corpus resource library;

[0151] a pronunciation category determination unit 13, configured to determine the pronunciation category of each second category word based on the part-of-speech information of each word in the segmented word sequence, where the second category words are words other than the first category words in the segmented word sequence;

[0152] The second pronunciation information determining unit 14 is configured to determine the pronunciation information of each second-category word based on a preset pronunciation information determining method corresponding to the pronunciation category of the second-category word.

[0153] In a possible implementation, the first pronunciation information determining unit is specifically configured to:

[0154] Looping through each word in the word segmentation sequence, searching the corpus resource library for pronunciation information corresponding to the word, and obtaining a pronunciation information search result for the word, wherein the pronunciation information search result for the word is used to indicate the pronunciation information corresponding to the word in the corpus resource library;

[0155] The first category of words and the pronunciation information of the first category of words are determined based on the pronunciation information search results of each word. The first category of words are words in the word segmentation sequence for which unique pronunciation information can be determined based on the corpus resource library, and the pronunciation information determined from the corpus resource library is the pronunciation information of the first category of words.

[0156] In a possible implementation, the pronunciation category determination unit includes:

[0157] a part-of-speech information search result determining unit, configured to loop through each word in the word segmentation sequence, search the corpus resource library for part-of-speech information corresponding to the word, and obtain a part-of-speech information search result for the word, wherein the part-of-speech information search result for the word is used to indicate the part-of-speech information corresponding to the word in the corpus resource library;

[0158] A pronunciation information sequence and pronunciation information attention mask sequence generating unit, configured to generate a pronunciation information sequence and a pronunciation information attention mask sequence based on the pronunciation information search result of each word;

[0159] A part-of-speech information sequence and part-of-speech information attention mask sequence generating unit, configured to generate a part-of-speech information sequence and a part-of-speech information attention mask sequence based on the part-of-speech information search result of each word;

[0160] The pronunciation category determination subunit is used to determine the pronunciation category of each second-category word based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence.

[0161] In a possible implementation, the pronunciation category determination subunit is specifically used to:

[0162] Inputting the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model;

[0163] Among them, the part-of-speech and pronunciation category prediction model uses a training word segmentation sequence, a training pronunciation information sequence, a training pronunciation information attention mask sequence, a training part-of-speech information sequence, and a training part-of-speech information attention mask sequence as training samples, and uses a pronunciation category label sequence corresponding to the training pronunciation information sequence and a part-of-speech information label sequence corresponding to the training part-of-speech information sequence as sample labels, and is trained with the part-of-speech prediction result output by the part-of-speech and pronunciation category prediction model being close to the part-of-speech information label sequence and the output pronunciation category prediction result being close to the pronunciation category label sequence as training objectives.

[0164] In a possible implementation, the pronunciation category determination subunit includes:

[0165] A first feature extraction processing unit is used to perform feature extraction processing on the word segmentation sequence to obtain a word segmentation feature vector sequence;

[0166] A second feature extraction processing unit is used to perform feature extraction processing on the pronunciation information sequence to obtain a pronunciation information feature vector sequence;

[0167] a third feature extraction processing unit, configured to perform feature extraction processing on the part-of-speech information sequence to obtain a part-of-speech information feature vector sequence;

[0168] a feature fusion processing unit, configured to perform feature fusion processing on the word segmentation feature vector sequence, the pronunciation information feature vector sequence, the part-of-speech information feature vector sequence, the pronunciation information attention mask sequence, and the part-of-speech information attention mask sequence to obtain a fused feature vector sequence;

[0169] a part-of-speech prediction result and pronunciation category prediction result determination unit, configured to input the fused feature vector sequence into the part-of-speech and pronunciation category prediction model, and obtain the part-of-speech prediction result of each word in the word segmentation sequence and the pronunciation category prediction result of each word in the word segmentation sequence output by the part-of-speech and pronunciation category prediction model;

[0170] The determination subunit is configured to determine the pronunciation category of each word in the second category from the pronunciation category prediction results of each word in the word segmentation sequence.

[0171] In a possible implementation, the second pronunciation information determining unit is specifically configured to:

[0172] Obtaining the pronunciation prediction modules corresponding to the preset pronunciation categories;

[0173] Determining a target pronunciation prediction module from each pronunciation prediction module, wherein the target pronunciation prediction module is a pronunciation prediction module corresponding to the pronunciation category of the second category words among the pronunciation prediction modules;

[0174] The text or the second-category words are provided to the target pronunciation prediction module to obtain pronunciation information of the second-category words output by the target pronunciation prediction module.

[0175] An electronic device is also provided in an embodiment of the present application. Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0176] like Figure 6 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0177] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0178] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the pronunciation prediction methods provided in the embodiments of the present application.

[0179] An embodiment of the present application also provides a computer-readable storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any pronunciation prediction method provided in the embodiment of the present application.

[0180] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0181] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0182] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0183] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A pronunciation prediction method, characterized in that: include: Performing word segmentation processing on the text to be synthesized to obtain a word segmentation sequence, wherein the word segmentation sequence includes multiple words; Determine the pronunciation information of the first category of words based on the preset corpus resource library, Looping through each word in the word segmentation sequence, searching the corpus resource library for part-of-speech information corresponding to the word, and obtaining a part-of-speech information search result for the word, wherein the part-of-speech information search result is used to indicate the part-of-speech information corresponding to the word in the corpus resource library; Based on the pronunciation information search results of each word, a pronunciation information sequence and a pronunciation information attention mask sequence are generated, wherein the pronunciation information attention mask sequence is used to indicate the pronunciation category prediction weight of each word in the word segmentation sequence, including: the pronunciation category prediction weight of the first category of words and the pronunciation category prediction weight of the second category of words; Based on the part-of-speech information search results for each word, a part-of-speech information sequence and a part-of-speech information attention mask sequence are generated, wherein the part-of-speech information attention mask sequence is used to indicate the part-of-speech information prediction weight of each word in the word segmentation sequence, including: the part-of-speech information prediction weight corresponding to a word containing only one part-of-speech information in the part-of-speech information search results, the part-of-speech information prediction weight corresponding to a word containing multiple part-of-speech information and a word containing zero part-of-speech information in the part-of-speech information search results; Determining, based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence, a pronunciation category of each second-category word, where the second-category words are words other than the first-category words in the word segmentation sequence, and the pronunciation categories include at least: polyphones, derivatives, compounds, abbreviations, words + abbreviations, and camel case conjunctions; Obtaining the pronunciation prediction modules corresponding to the preset pronunciation categories; Determining a target pronunciation prediction module from each pronunciation prediction module, wherein the target pronunciation prediction module is a pronunciation prediction module corresponding to the pronunciation category of the second category words among the pronunciation prediction modules; The text or the second-category words are provided to the target pronunciation prediction module to obtain pronunciation information of the second-category words output by the target pronunciation prediction module.

2. The method according to claim 1, characterized in that The step of determining the pronunciation information of the first category of words based on a preset corpus resource library includes: Looping through each word in the word segmentation sequence, searching the corpus resource library for pronunciation information corresponding to the word, and obtaining a pronunciation information search result for the word, wherein the pronunciation information search result for the word is used to indicate the pronunciation information corresponding to the word in the corpus resource library; The first category of words and the pronunciation information of the first category of words are determined based on the pronunciation information search results of each word; the first category of words are words in the word segmentation sequence for which unique pronunciation information can be determined based on the corpus resource library, and the pronunciation information determined from the corpus resource library is the pronunciation information of the first category of words.

3. The method according to claim 1, characterized in that The determining of the pronunciation category of each second-category word based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence includes: Inputting the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model; Among them, the part-of-speech and pronunciation category prediction model uses a training word segmentation sequence, a training pronunciation information sequence, a training pronunciation information attention mask sequence, a training part-of-speech information sequence, and a training part-of-speech information attention mask sequence as training samples, and uses a pronunciation category label sequence corresponding to the training pronunciation information sequence and a part-of-speech information label sequence corresponding to the training part-of-speech information sequence as sample labels, and is trained with the part-of-speech prediction result output by the part-of-speech and pronunciation category prediction model being close to the part-of-speech information label sequence and the output pronunciation category prediction result being close to the pronunciation category label sequence as training objectives.

4. The method according to claim 3, characterized in that The step of inputting the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence into a part-of-speech and pronunciation category prediction model to obtain the pronunciation category of each second-category word output by the part-of-speech and pronunciation category prediction model includes: Performing feature extraction processing on the word segmentation sequence to obtain a word segmentation feature vector sequence; Performing feature extraction processing on the pronunciation information sequence to obtain a pronunciation information feature vector sequence; Performing feature extraction processing on the part-of-speech information sequence to obtain a part-of-speech information feature vector sequence; Performing feature fusion processing on the word segmentation feature vector sequence, the pronunciation information feature vector sequence, the part-of-speech information feature vector sequence, the pronunciation information attention mask sequence, and the part-of-speech information attention mask sequence to obtain a fused feature vector sequence; Inputting the fused feature vector sequence into the part-of-speech and pronunciation category prediction model to obtain the part-of-speech prediction results of each word in the word segmentation sequence and the pronunciation category prediction results of each word in the word segmentation sequence output by the part-of-speech and pronunciation category prediction model; The pronunciation category of each word in the second category is determined from the pronunciation category prediction results of each word in the word segmentation sequence.

5. A pronunciation prediction device, characterized in that include: A word segmentation processing unit, configured to perform word segmentation processing on the text to be synthesized to obtain a word segmentation sequence, wherein the word segmentation sequence includes a plurality of words; A first pronunciation information determining unit, configured to determine pronunciation information of a first category of words based on a preset corpus resource library; a pronunciation category determination unit, looping through each word in the word segmentation sequence, searching the corpus resource library for part-of-speech information corresponding to the word, and obtaining a part-of-speech information search result for the word, wherein the part-of-speech information search result is used to indicate the part-of-speech information corresponding to the word in the corpus resource library; Based on the pronunciation information search results of each word, a pronunciation information sequence and a pronunciation information attention mask sequence are generated, and the pronunciation information attention mask sequence is used to indicate the pronunciation category prediction weight of each word in the word segmentation sequence, including: the pronunciation category prediction weight of the first category of words and the pronunciation category prediction weight of the second category of words; Based on the part-of-speech information search results of each word, a part-of-speech information sequence and a part-of-speech information attention mask sequence are generated, and the part-of-speech information attention mask sequence is used to indicate the part-of-speech information prediction weight of each word in the word segmentation sequence, including: the part-of-speech information search results only contain one part-of-speech information. The method comprises the following steps: determining the part-of-speech information prediction weight corresponding to the word containing the part-of-speech information, the part-of-speech information prediction weight corresponding to the word containing multiple part-of-speech information and the word containing 0 part-of-speech information in the part-of-speech information search result; determining the pronunciation category of each second-category word based on the word segmentation sequence, the pronunciation information sequence, the pronunciation information attention mask sequence, the part-of-speech information sequence, and the part-of-speech information attention mask sequence, wherein the second-category words are other words in the word segmentation sequence except the first-category words, and the pronunciation categories include at least: polyphonic words, derivative words, compound words, abbreviations, words + abbreviations, and camel case conjunctions; The second pronunciation information determination unit is used to obtain the pronunciation prediction modules corresponding to each preset pronunciation category; determine the target pronunciation prediction module from each pronunciation prediction module, and the target pronunciation prediction module is the pronunciation prediction module corresponding to the pronunciation category of the second category words in each pronunciation prediction module; provide the text or the second category words to the target pronunciation prediction module to obtain the pronunciation information of the second category words output by the target pronunciation prediction module.

6. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the pronunciation prediction method according to any one of claims 1 to 4.

7. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program so that the electronic device can implement the pronunciation prediction method according to any one of claims 1 to 4.

8. A computer storage medium, characterized in that The computer storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the pronunciation prediction method according to any one of claims 1 to 4.

Images