Polyphone phonetic notation method and related device

Abstract

The embodiment of the invention discloses a polyphone phonetic notation method and a related device. The method comprises the following steps: acquiring a phonetic notation reference text; the phonetic notation reference text comprises a target polyphone to be subjected to phonetic notation and a context text of the target polyphone; determining semantic features corresponding to the phonetic notation reference text through a semantic analysis model; performing word segmentation processing on the phonetic notation reference text to obtain each segmented word included in the phonetic notation reference text, and determining part-of-speech corresponding to each segmented word included in the phonetic notation reference text; determining word segmentation features corresponding to the phonetic notation reference text according to each word segmentation included in the phonetic notation reference text and the part of speech corresponding to the word segmentation; and through a pronunciation classification model, according to the semantic features and the word segmentation features, determining a phonetic notation result corresponding to the target polyphone in the phonetic notation reference text. According to the method, correct phonetic notation can be carried out on the polyphone in the text, so that the situation that the polyphone is misread is avoided.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a method for pronouncing polyphonic characters and related devices. Background Art

[0002] Text-to-speech (TTS) is a technology that converts text information into speech, which is now widely used in fields such as intelligent assistants, accessibility technologies, e-book reading, dubbing, etc. TTS technology includes two main parts: text processing and speech synthesis. Text processing is responsible for converting specific text (such as user input text, e-book text, etc.) into corresponding speech indication information, and speech synthesis is responsible for converting the speech indication information obtained through text processing into audible speech.

[0003] In practical applications, when using TTS technology to convert text including polyphonic characters into corresponding speech, it is very easy to make mistakes in pronouncing polyphonic characters, that is, mispronouncing the polyphonic characters in the text as their other pronunciations, such as pronouncing xue1 in "exploitation" as xiao1, which will seriously affect the user experience of related functions. Therefore, how to correctly pronounce polyphonic characters in text to avoid mispronouncing polyphonic characters has become an urgent problem to be solved at present. Summary of the Invention

[0004] The embodiments of this application provide a method for pronouncing polyphonic characters and related devices, which can correctly pronounce polyphonic characters in text to avoid mispronouncing polyphonic characters.

[0005] The first aspect of this application provides a method for pronouncing polyphonic characters, and the method includes:

[0006] Obtain a phonetic reference text; the phonetic reference text includes a target polyphonic character to be pronounced and the context text of the target polyphonic character;

[0007] Determine the semantic features corresponding to the phonetic reference text through a semantic analysis model;

[0008] Perform word segmentation on the phonetic reference text to obtain each word segment included in the phonetic reference text, and determine the词性 corresponding to each word segment included in the phonetic reference text; according to each word segment included in the phonetic reference text and its corresponding词性, determine the word segmentation features corresponding to the phonetic reference text;

[0009] Through a pronunciation classification model, determine the pronunciation result corresponding to the target polyphonic character in the phonetic reference text according to the semantic features and the word segmentation features.

[0010] The second aspect of this application provides a device for pronouncing polyphonic characters, and the device includes: It should be noted that the "词性" in the original text may be a specific term in Chinese related to grammar, and it is directly translated here. If there is a more accurate English equivalent in the context of language technology, it can be adjusted accordingly. Also, the " " etc. tags are preserved as they are according to the requirements.

[0011] The acquisition module is used to acquire the phonetic reference text; the phonetic reference text includes the target polyphonic character to be phonetized and the context text of the target polyphonic character;

[0012] The first feature determination module is used to determine the semantic features corresponding to the phonetic reference text through a semantic analysis model;

[0013] The second feature determination module is used to perform word segmentation processing on the phonetic reference text to obtain each word included in the phonetic reference text, and determine the part of speech corresponding to each word included in the phonetic reference text; based on each word included in the phonetic reference text and its corresponding part of speech, determine the word segmentation features corresponding to the phonetic reference text.

[0014] The phonetic annotation module is used to determine the phonetic annotation result corresponding to the target polyphonic character in the phonetic annotation reference text based on the semantic features and the word segmentation features using a pronunciation classification model.

[0015] Optionally, the second feature determination module includes:

[0016] The first determining unit is configured to, for each character in the phonetic reference text, determine the word segment to which the character belongs in the phonetic reference text; determine the positional feature corresponding to the character based on the position of the character in the word segment; determine the phonetic feature corresponding to the character based on the polyphonic character detection result corresponding to the word segment; and determine the part-of-speech feature corresponding to the character based on the part-of-speech corresponding to the word segment.

[0017] The second determining unit is used to determine the word segmentation features corresponding to the phonetic reference text based on the position features, phonetic features, and part-of-speech features of each character in the phonetic reference text.

[0018] Optionally, the first determining unit includes:

[0019] The first position feature determination unit is used to determine a position identifier that indicates the position of the character in the word segment if the character and other characters together constitute the word segment, and use it as the position feature corresponding to the character;

[0020] The second position feature determination unit is used to determine a position identifier for indicating an independent character as the position feature corresponding to the character if the character independently constitutes the word segment.

[0021] Optionally, the first determining unit includes:

[0022] The first phonetic feature determination unit is used to determine the pronunciation of the character recorded in the dictionary when the polyphonic character detection result corresponding to the word segment indicates that the word segment includes polyphonic characters and the word segment can be retrieved in the dictionary. If the character is the polyphonic character in the word segment, the unit determines the pronunciation of the character recorded in the dictionary when it belongs to the word segment and uses it as the phonetic feature corresponding to the character. If the character is not the polyphonic character in the word segment, the unit determines the default phonetic identifier as the phonetic feature corresponding to the character.

[0023] The second phonetic feature determination unit is used to determine the default phonetic identifier as the phonetic feature corresponding to the character when the polyphonic character detection result corresponding to the word segment indicates that the word segment does not contain polyphonic characters, or when the polyphonic character detection result corresponding to the word segment indicates that the word segment contains polyphonic characters, but the word segment cannot be retrieved in the dictionary.

[0024] Optionally, the first determining unit includes:

[0025] The part-of-speech feature determination unit is used to determine the part-of-speech identifier used to indicate the part of speech corresponding to the word segmentation, as the part-of-speech feature corresponding to the character.

[0026] Optionally, the phonetic module includes:

[0027] An input feature vector determination unit is used to determine an input feature vector based on the semantic features and the word segmentation features;

[0028] The output feature vector determination unit is used to process the input feature vector through the gated loop unit in the pronunciation classification model to obtain the output feature vector;

[0029] A fusion unit is used to fuse the input feature vector and the output feature vector to obtain a fused feature vector;

[0030] The first processing unit is used to process the fused feature vector through the linear layer in the pronunciation classification model to obtain the classification feature vector;

[0031] The classification unit is used to classify the target polyphonic character based on the classification feature vector through the classification layer in the pronunciation classification model, and to determine the phonetic result corresponding to the target polyphonic character.

[0032] Optionally, the first processing unit includes:

[0033] The extraction unit is used to extract the sub-feature vector corresponding to the target polyphonic character from the fused feature vector according to the index of the target polyphonic character in the phonetic reference text;

[0034] The classification feature vector determination unit is used to process the sub-feature vector through the linear layer to obtain the classification feature vector.

[0035] Optionally, the classification unit includes:

[0036] An initial weight determination unit is used to determine the initial weight of each candidate pronunciation supported by the pronunciation classification model as 1 if the candidate pronunciation is a possible pronunciation of the target polyphonic character, and 0 if the candidate pronunciation is not a possible pronunciation of the target polyphonic character.

[0037] The target weight determination unit is used to classify based on the classification feature vector through the classification layer and determine the target weight corresponding to each candidate pronunciation supported by the pronunciation classification model; the target weight is determined based on the initial weight, and if the initial weight corresponding to the candidate pronunciation is 0, then the target weight corresponding to the candidate pronunciation is also 0;

[0038] The phonetic result determination unit is used to determine the candidate pronunciation with the largest corresponding target weight, which is used as the phonetic result corresponding to the target polyphonic character.

[0039] Optionally, the acquisition module includes:

[0040] The acquisition unit is used to detect whether each sentence in the original text contains a polyphonic character; if the sentence contains the polyphonic character and the polyphonic character independently constitutes a word in the sentence, then the polyphonic character is taken as the target polyphonic character, and the phonetic reference text is extracted from the sentence based on the position of the target polyphonic character in the sentence and a preset text extraction window.

[0041] Optionally, the device further includes:

[0042] The detection unit is used to detect whether the length of each sentence in the original text is within a preset length range; if the length of the sentence is less than the lower threshold of the preset length range, the sentence is merged with other adjacent sentences into one sentence; if the length of the sentence is greater than the upper threshold of the preset length range, the sentence is split into multiple sentences.

[0043] The replacement unit is used to, for each clause in the original text, delete the special character if the clause contains a preset special character, or replace the special character with a preset regular character.

[0044] Optionally, the device further includes:

[0045] The generation unit is used to generate a speech synthesis markup language corresponding to the target text to which the target polyphonic character belongs, based on the phonetic annotation result corresponding to the target polyphonic character; the speech synthesis markup language is used to indicate the pronunciation mode of the speech when converting the target text into the corresponding speech.

[0046] A third aspect of this application provides a computer device, the device comprising a processor and a memory:

[0047] The memory is used to store computer programs;

[0048] The processor is configured to execute, according to the computer program, the steps of the polyphonic character phonetic notation method described in the first aspect above.

[0049] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program for performing the steps of the polyphonic character phonetic notation method described in the first aspect above.

[0050] The fifth aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the polyphonic character phonetic notation method described in the first aspect.

[0051] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0052] This application provides a method for annotating the pronunciation of polyphonic characters. The method includes: obtaining a phonetic reference text, which includes the target polyphonic character to be annotated and its context text; since in many cases, it is difficult to accurately determine the pronunciation of a polyphonic character based solely on the character itself, this application, when annotating the pronunciation of a target polyphonic character, not only obtains the target polyphonic character but also its context text. Then, based on the phonetic reference text composed of the target polyphonic character and its context text, the phonetic result of the target polyphonic character is determined, thereby achieving phonetic annotation of the target polyphonic character by referencing its context. Then, through a semantic analysis model, the semantic features corresponding to the phonetic reference text are determined, that is, the semantic features that can characterize the semantic information of the phonetic reference text are identified. Furthermore, the phonetic reference text is segmented to obtain the individual words included in the phonetic reference text, and the part-of-speech tag of each word is determined. Then, based on the individual words and their corresponding parts of speech, the segmentation features of the phonetic reference text are determined, i.e., features related to the segmentation in the phonetic reference text and valuable for annotating the target polyphonic character. Finally, using a pronunciation classification model, based on the aforementioned semantic and segmentation features, the pronunciation result corresponding to the target polyphonic character in the phonetic reference text is determined. That is, the pronunciation classification model, referring to the semantic and segmentation features of the phonetic reference text, performs a pronunciation classification task for the target polyphonic character, thereby determining the correct pronunciation of the target polyphonic character in the phonetic reference text. Thus, by referring to the semantic information of the context in which the target polyphonic character belongs, as well as the relevant information of other segmentation words in the context, accurate pronunciation of the target polyphonic character is achieved, thereby avoiding mispronunciation of the target polyphonic character when converting text to speech. Attached Figure Description

[0053] Figure 1 This is a schematic diagram illustrating an application scenario of the polyphonic character phonetic notation method provided in the embodiments of this application;

[0054] Figure 2 A flowchart illustrating the method for phonetic notation of polyphonic characters provided in this application embodiment;

[0055] Figure 3 This is a schematic diagram of the structure of the pronunciation classification model provided in the embodiments of this application;

[0056] Figure 4 A schematic diagram illustrating the implementation of the polyphonic character phonetic notation method provided in this application embodiment;

[0057] Figure 5 A schematic diagram of an e-book reading interface provided in an embodiment of this application;

[0058] Figure 6A schematic diagram of the player interface provided in an embodiment of this application;

[0059] Figure 7 A schematic diagram of the interface for entering e-book listening mode provided in an embodiment of this application;

[0060] Figure 8 This is a schematic diagram of the structure of the polyphonic character phonetic notation device provided in the embodiments of this application;

[0061] Figure 9 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0062] Figure 10 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0063] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0064] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0065] The phonetic notation method for polyphonic characters provided in this application can be executed by a computer device, which can be a terminal device or a server. The terminal device includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server.

[0066] It should be noted that the information, data and signals involved in the embodiments of this application are all authorized by the relevant parties or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0067] To facilitate understanding of the polyphonic character annotation method provided in this application embodiment, the following example uses a server as the execution subject of the polyphonic character annotation method to illustrate the application scenarios of the polyphonic character annotation method.

[0068] See Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of the polyphonic character phonetic notation method provided in the embodiments of this application. For example... Figure 1 As shown, this application scenario includes a database 110 and a server 120. The server 120 can access the database 110 via the network, or the database 110 can be integrated into the server 120.

[0069] Database 110 stores the text content of the e-book and the corresponding Speech Synthesis Markup Language (SSML) file. The SSML file is used to convert the text in the e-book into its corresponding speech; it indicates the output method of the speech, such as pronunciation, pitch, speed, and volume. In this embodiment, for text in the e-book containing polyphonic characters, the phonetic transcription of those characters can be written into the corresponding SSML file so that the polyphonic characters can be correctly read out when the text is converted into speech based on the SSML file.

[0070] In practical applications, when generating corresponding SSML files for an e-book, server 120 can retrieve the text content of the e-book from database 110, and then generate a corresponding SSML sub-file for each sentence in the text content. This SSML sub-file is used to indicate the speech output method of its corresponding sentence. When server 120 detects that a sentence in the e-book contains a polyphonic character, it can use that polyphonic character as the target polyphonic character and then execute the polyphonic character phonetic annotation method provided in this application embodiment to annotate the target polyphonic character.

[0071] Specifically, server 120 can first extract the phonetic reference text from the sentence containing the target polyphonic character. This phonetic reference text includes the target polyphonic character to be annotated and its context text. Since in many cases, it is often difficult to accurately determine the pronunciation of a polyphonic character based solely on the character itself, when annotating the target polyphonic character, server 120 not only obtains the target polyphonic character but also its context text. Then, based on the phonetic reference text composed of the target polyphonic character and its context text, the phonetic result of the target polyphonic character is determined, thereby achieving phonetic annotation of the target polyphonic character by referencing its context.

[0072] Server 120 can determine the semantic features corresponding to the phonetic reference text through a semantic analysis model, that is, determine the semantic features that can represent the semantic information of the phonetic reference text.

[0073] Furthermore, server 120 will perform word segmentation on the phonetic reference text to obtain each word segment included in the phonetic reference text, and determine the part of speech corresponding to each word segment. Then, server 120 can determine the word segmentation features corresponding to the phonetic reference text based on the aforementioned word segments and their corresponding parts of speech; that is, it can determine the features related to the word segments in the phonetic reference text that are valuable for annotating the target polyphonic characters.

[0074] Server 120 can determine the phonetic result corresponding to the target polyphonic character in the phonetic reference text by using the pronunciation classification model and based on the aforementioned semantic features and word segmentation features; that is, the pronunciation classification model refers to the semantic features and word segmentation features corresponding to the phonetic reference text to perform a pronunciation classification task for the target polyphonic character, thereby determining the pronunciation that the target polyphonic character should correspond to in the phonetic reference text.

[0075] Finally, after obtaining the phonetic result corresponding to the target polyphonic character, the server 120 can generate the SSML sub-file corresponding to the sentence to which the target polyphonic character belongs based on the phonetic result, and generate the SSML file corresponding to the e-book based on the SSML sub-files corresponding to each sentence in the currently processed e-book, and then provide the SSML file corresponding to the e-book to the database 110 so that the database 110 can store the SSML file corresponding to the e-book.

[0076] Furthermore, the embodiments of this application can also be applied to scenarios where text provided by a user is dubbed. That is, the user can upload the text to be dubbed to the server through a terminal device. After receiving the text, if the server detects and determines that it includes a polyphonic character, it can take the polyphonic character as the target polyphonic character and extract the phonetic reference text including the target polyphonic character and its context text from the text to be dubbed. Then, the method provided in the embodiments of this application is executed to annotate the target polyphonic character based on the phonetic reference text. Furthermore, based on the determined phonetic result, the SSML file corresponding to the text to be dubbed can be determined, and the SSML file can be fed back to the terminal device, so that the terminal device can output the voice corresponding to the text to be dubbed based on the SSML file.

[0077] It should be understood that the application scenarios described above for annotating polyphonic characters in e-book text and text to be dubbed are merely examples. In practical applications, the polyphonic character annotation method provided in this application embodiment can also be applied to other scenarios that require annotating polyphonic characters. No limitations are made here on the application scenarios of the polyphonic character annotation method provided in this application embodiment.

[0078] The following describes in detail the method for annotating polyphonic characters provided in this application through method embodiments.

[0079] See Figure 2 , Figure 2 This is a flowchart illustrating the method for annotating polyphonic characters according to an embodiment of this application. For ease of description, the following description uses a server as the executing entity of this method. Figure 2 As shown, this method for annotating polyphonic characters includes the following steps:

[0080] S201: Obtain the phonetic reference text; the phonetic reference text includes the target polyphonic character to be phonetized and the context text of the target polyphonic character.

[0081] Phonetic reference text refers to text that includes the target polyphonic character to be transcribed and its surrounding text. The phonetic reference text can be extracted from the text containing the target polyphonic character according to a preset extraction strategy. The text containing the target polyphonic character can be the text that needs to be converted to speech; for example, it could be text from an e-book, text provided by the user to be transcribed, or text used by a smart assistant for voice output.

[0082] The target polyphonic character refers to the polyphonic character to be phonetically annotated. In the embodiments of the present application, the target polyphonic character may specifically be a polyphonic character that does not form a word with other characters, that is, a polyphonic character that exists alone in a sentence. For example, in a text content "the tune of Prelude to Water Melody", according to the polyphonic characters defined in the dictionary, the "tune" in the above text content is a polyphonic character, which can be pronounced as "diao4" (diao represents the pinyin and 4 represents the tone), and can also be pronounced as "tiao2" (tiao represents the pinyin and 2 represents the tone); for the first "tune" in the above text content, it forms a word "Prelude to Water Melody" with other characters, and according to the records in the dictionary, the pronunciation of "tune" in the word "Prelude to Water Melody" is "diao4"; while for the second "tune" in the above text content, it does not form a word with other characters but exists alone in the text content, so the second "tune" in the above text content can be used as the target polyphonic character to be phonetically annotated.

[0083] Other similar examples may also include: "It has been noisy since early morning, making it impossible to sleep well", according to the polyphonic characters defined in the dictionary, the "jue" in the above text content is a polyphonic character, which can be pronounced as "jue2" (jue represents the pinyin and 2 represents the tone), and can also be pronounced as "jiao4" (jiao represents the pinyin and 4 represents the tone). Since the "jue" in the above text does not form a word with other characters but exists alone in the text content, the "jue" can be used as the target polyphonic character to be phonetically annotated; or, it may also include: "He has been very strange recently, as if possessed", according to the polyphonic characters defined in the dictionary, the "zhuo" in the above text content is a polyphonic character, which can be pronounced as "zhe" (zhe represents the pinyin, and no number indicates a light tone), "zhao2" (zhao represents the pinyin and 2 represents the tone), "zhuo2" (zhuo represents the pinyin and 2 represents the tone), and "zhao1" (zhao represents the pinyin and 1 represents the tone). Since the "zhuo" in the above text does not form a word with other characters but exists alone in the text content, the "zhuo" can be used as the target polyphonic character to be phonetically annotated.

[0084] The context text of the target polyphonic character is obtained by intercepting from the text including the target polyphonic character according to a preset intercepting length. Since the preset intercepting length is fixed and the position of the target polyphonic character in the text is random, the context text of the target polyphonic character may only include the text above the target polyphonic character, may also only include the text below the target polyphonic character, or may also include both the context text above and below the target polyphonic character.

[0085] For example, when the target polyphonic character is the last character in its text, it is truncated according to a preset truncation length, and the context text of the target polyphonic character only includes the text preceding it; when the target polyphonic character is the first character in its text, it is truncated according to a preset truncation length, and the context text of the target polyphonic character only includes the text following it; when the target polyphonic character is located in the middle of its text, it is truncated according to a preset truncation length, and the context text of the target polyphonic character includes both the text preceding and following it. In this regard, this application does not specifically limit the context text of the target polyphonic character.

[0086] As an example, the method provided in this application can be used to annotate the pronunciation of polyphonic characters in e-book text. In this scenario, the server can obtain the text content of the e-book from a database used to store e-book resources. Then, it can detect whether a target polyphonic character exists in each sentence of the e-book text. If the detection determines that a target polyphonic character exists in a certain sentence of the e-book text, the target polyphonic character and its context text can be extracted from that sentence to form a phonetic reference text. The phonetic annotation is then performed on the target polyphonic character based on the phonetic reference text. The obtained phonetic annotation result can be used to generate an SSML file corresponding to the e-book, so that when it is necessary to convert the e-book text into speech, the corresponding conversion can be performed based on the SSML file.

[0087] As another example, the method provided in this application can be used to annotate the pronunciation of polyphonic characters in a user-provided text to be dubbed. In this scenario, the server can obtain the text to be dubbed uploaded by the user through a terminal device, and then detect whether there is a target polyphonic character in the text. If the detection determines that there is a target polyphonic character in the text, the target polyphonic character and its context text can be extracted from the text to form a phonetic reference text, and the pronunciation of the target polyphonic character can be annotated based on the phonetic reference text. The obtained phonetic result can be used to generate an SSML file corresponding to the text to be dubbed, thereby converting the text to be dubbed into the corresponding speech. When this application is applied to the scenario of annotating the pronunciation of polyphonic characters in e-book text, each sentence corresponding to the original text of the e-book can be detected and preprocessed first, and then the phonetic reference text can be extracted from each preprocessed sentence included in the e-book.

[0088] In one possible implementation, the method provided in this application embodiment may further include:

[0089] For each clause in the original text, check whether the length of the clause is within a preset length range; if the length of the clause is less than the lower threshold of the preset length range, merge the clause with other adjacent clauses into one clause; if the length of the clause is greater than the upper threshold of the preset length range, split the clause into multiple clauses.

[0090] For each clause in the original text, if the clause contains a preset special character, then the special character is deleted, or the special character is replaced with a preset regular character.

[0091] The original text refers to the text resource obtained by the server. For example, the original text could be the text content of an ebook. The server can send a data retrieval request to the database used to store ebook resources to obtain the original text. After obtaining the original text, it can be segmented into sentences using regular expressions. Specifically, regular expressions segment sentences by searching for punctuation marks (such as periods, semicolons, exclamation marks, and question marks) in the original text.

[0092] It should be noted that the original text can be plain text (text, txt) or Extensible Markup Language (XML) format. When the original text is in XML format, the XML tags in the original text need to be replaced to avoid the XML tags affecting the clauses.

[0093] The preset length range is used to indicate the pre-set length requirements for sentences in the original text. The preset length range includes an upper limit threshold and a lower limit threshold for sentence length. The upper limit threshold is used to indicate the maximum length of a sentence, and the lower limit threshold is used to indicate the minimum length of a sentence.

[0094] As an example, for each clause in the original text, it is possible to detect whether the length of the clause is within a preset length range. That is, the length of each clause in the original text can be compared with the preset length range to determine whether the length of each clause in the original text is within the preset length range.

[0095] If the length of a clause is less than the lower threshold of a preset length range, it means that the length of the clause does not meet the preset length requirement. In this case, the clause can be merged with other adjacent clauses to ensure that the length of the merged clause is within the preset length range. The clause can be merged with the preceding adjacent clause or the following adjacent clause. This application does not specifically limit the position of other adjacent clauses.

[0096] If the length of a clause exceeds the upper limit of the preset length range, it means that the length of the clause does not meet the preset length requirement. In this case, punctuation marks can be added to the clause to split it into multiple clauses, so as to ensure that the length of the split clauses is within the preset length range.

[0097] It should be noted that the reason for setting the aforementioned preset length range and requiring the length of each clause in the original text to be within this range is that the extraction of the phonetic reference text described above is performed within clauses, meaning that the phonetic reference text needs to be extracted from a single clause. Clauses that are too long or too short will affect the extraction result. For example, clauses that are too short will result in extracted phonetic reference text that is too short to accurately reflect the context of the target polyphonic character. Conversely, clauses that are too long will result in extracted phonetic reference text that reflects overly complex information, including a lot of information unrelated to the target polyphonic character. Therefore, this embodiment requires the length of each clause in the original text to be within the preset length range to prevent the aforementioned problems from occurring in the extracted phonetic reference text due to clauses being too long or too short.

[0098] As an example, for each clause in the original text, if the clause contains a preset special character, then the special character is deleted from the clause, or replaced with a preset regular character. The preset special character refers to a pre-defined character other than regular characters, numbers, and punctuation marks. For example, preset special characters could be emojis or zero-width characters (non-printable characters with a byte width of 0). Preset regular characters refer to pre-defined characters used to replace special characters; for example, preset regular characters could be "*". This application does not specifically limit the definition of special characters and preset regular characters.

[0099] It should be noted that the reason for deleting or replacing special characters in the clauses is that special characters are difficult to play a substantial role in the relevant semantic analysis process and are prone to ambiguity and misunderstanding. Therefore, it is necessary to process special characters in advance.

[0100] Therefore, the above method can be used to preprocess the sentences in the original text, and the phonetic reference text can be determined based on the preprocessed sentences. This helps to improve the quality of the determined phonetic reference text, so that the determined phonetic reference text can accurately represent the context of the target polyphonic character, which is conducive to accurately annotating the target polyphonic character based on the phonetic reference text.

[0101] In one possible implementation, the above-mentioned S201 "obtaining the phonetic reference text" may include:

[0102] For each clause in the original text, detect whether the clause contains polyphonic characters. If the clause contains polyphonic characters and the polyphonic characters form independent words in the clause, then the polyphonic characters are taken as target polyphonic characters. Based on the position of the target polyphonic characters in the clause and the preset text extraction window, extract the phonetic reference text in the clause.

[0103] As an example, each clause in the original text can be a clause after the above preprocessing. For each clause in the original text, a pre-defined polyphonic character dictionary can be used to detect whether the clause contains polyphonic characters. If the clause contains polyphonic characters and the polyphonic characters independently form a word in the clause, then the polyphonic characters are taken as target polyphonic characters. That is, if the clause contains polyphonic characters and the polyphonic characters do not form a word with other characters, then the polyphonic characters can be taken as target polyphonic characters.

[0104] Furthermore, it can extract phonetic reference text from a clause based on the position of the target polyphonic character within the clause and a preset text extraction window. The preset text extraction window refers to a fixed window size used when extracting the phonetic reference text from a clause. For example, the preset text extraction window is `window_size`. Based on the position of the target polyphonic character within the clause and the preset text extraction window, phonetic reference text conforming to the preset text extraction window size can be extracted from the clause containing the target polyphonic character.

[0105] For example, if the target polyphonic character is located in the middle of a clause, then a maximum of window_size / / 2 characters are retained before and after the target polyphonic character to extract the phonetic reference text from the clause, including the target polyphonic character and its surrounding text. If the target polyphonic character is located at the beginning of a clause, i.e., the characters preceding the target polyphonic character are less than window_size / / 2 characters, then window_size characters are extracted from the beginning of the target polyphonic character to extract the phonetic reference text from the clause, including the target polyphonic character and its following text. If the target polyphonic character is located at the end of a clause, i.e., the characters following the target polyphonic character are less than window_size / / 2 characters, then window_size characters are extracted from the beginning of the target polyphonic character to extract the phonetic reference text from the clause, including the target polyphonic character and its preceding text. If both the characters before and after the target polyphonic character are less than window_size / / 2 characters, then the entire clause can be used as the phonetic reference text.

[0106] Therefore, the above method can be used to identify clauses containing polyphonic characters from the original text. If a clause contains a polyphonic character, the target polyphonic character and its phonetic reference text can be further identified from that clause. Furthermore, the phonetic reference text is extracted from the clause based on the position of the target polyphonic character within the clause and a preset text extraction window. This allows for accurate phonetic annotation of the target polyphonic character during subsequent extraction, based on the context of the preset extraction length, avoiding interference from other information in the clause containing the target polyphonic character.

[0107] S202: Determine the semantic features corresponding to the phonetic reference text through a semantic analysis model.

[0108] Semantic analysis models, in the field of natural language processing, are models used to understand the deeper meaning and intent within text content. They encode input text to obtain feature representations that characterize the semantic information of that input text. In this embodiment, by processing the phonetic reference text using a semantic analysis model, semantic features that characterize the semantic information of the phonetic reference text can be obtained. For example, the semantic analysis model can be a pre-trained deep learning model such as Bert-Base-Chinese, ERNIE-3.0-base-zh, or Chinese-Macbert-Base.

[0109] Bert-Base-Chinese is a pre-trained deep learning model specifically designed for processing Chinese text data. Based on the Bidirectional Encoder Representations from Transformers (BERT) architecture, it uses a bidirectional encoder (Transformer) to understand contextual information within the text. Pre-trained on a large amount of Chinese text data, the Bert-Base-Chinese model learns rich language representations and can be used to handle various Chinese natural language processing tasks, such as text classification, named entity recognition, and sentiment analysis.

[0110] As an example, before processing through a semantic analysis model, the phonetic reference text needs to be preprocessed and encoded. Specifically, the preprocessing process may include adding the character CLS to the beginning of the phonetic reference text to identify the start position and adding the character SEP to the end of the phonetic reference text to identify the end position, thus obtaining the preprocessed phonetic reference text.

[0111] Then, encoding processing can be performed on the preprocessed phonetic reference text. Specifically, the encoding processing may include: First, each character in the preprocessed phonetic reference text can be converted into a corresponding token by a tokenizer. The tokens corresponding to the characters in the above preprocessed phonetic reference text are the indices in a predefined dictionary. Based on the tokens corresponding to each character in the preprocessed phonetic reference text, an embedding lookup can be performed to determine the embedding vector representation corresponding to each token, that is, the encoding feature corresponding to each character. After that, the encoding features corresponding to each character can be concatenated to obtain the encoding feature corresponding to the phonetic reference text.

[0112] Finally, the encoding feature corresponding to the phonetic reference text is input into a semantic analysis model. By processing this encoding feature, the semantic analysis model can output the semantic feature corresponding to the phonetic reference text. The semantic feature corresponding to the phonetic reference text refers to the output result of the semantic analysis model and is used to represent the semantic information of the phonetic reference text.

[0113] S203: Perform word segmentation on the phonetic reference text to obtain each word segment included in the phonetic reference text, and determine the respective词性 corresponding to each word segment included in the phonetic reference text; According to each word segment included in the phonetic reference text and its respective词性, determine the word segment feature corresponding to the phonetic reference text.

[0114] Word segmentation processing refers to splitting continuous Chinese text into independent words and characters. As an example, a word segmentation tool can be used to perform word segmentation on the phonetic reference text to obtain each word segment included in the phonetic reference text. At the same time, the respective词性 corresponding to each word segment included in the phonetic reference text can also be determined by the word segmentation tool. Each word segment included in the phonetic reference text refers to the word segmentation result obtained through word segmentation processing. For example, if the phonetic reference text is "the tune of Prelude to Water Melody", each word segment included in the phonetic reference text is "Prelude to Water Melody", "of", and "tune".

[0115] The respective词性 corresponding to each word segment included in the phonetic reference text is used to indicate the function of each word segment in the sentence. For example, the词性 corresponding to a word segment may include but is not limited to nouns, verbs, auxiliary words, and adjectives, etc. During the process of performing word segmentation on the phonetic reference text by a word segmentation tool, the word segmentation result and its corresponding词性 result can be determined synchronously, that is, the respective word segments included in the phonetic reference text and their respective词性 can be obtained.

[0116] As an example, the word segmentation tools can be the word segmentation model qqseg, the Chinese word segmentation tool jieba, and the natural language processing toolkit hanlp, etc. For example, when the word segmentation tool is the word segmentation model qqseg, inputting the phonetic annotation reference text into the word segmentation model for word segmentation processing can obtain the word segmentation result and its corresponding part-of-speech result. For example, when the phonetic annotation reference text is "the tune of Prelude to Water Melody", the word segmentation result and its corresponding part-of-speech result output by the word segmentation model are: {"seg": ["the tune of Prelude to Water Melody", "of", "tune"], "pos": ["n", "u", "v"]}. Here, "seg" represents the word segmentation result, "pos" represents the part-of-speech result, "n" represents a noun, "v" represents a verb, and "u" represents a particle.

[0117] The word segmentation feature refers to the feature that has reference value for phonetically annotating the target polyphonic character determined according to each word segment included in the phonetic annotation reference text and its corresponding part-of-speech. The word segmentation feature can, for example, include at least one of the position feature, the phonetic annotation feature, and the part-of-speech feature. Among them, the position feature is used to characterize the position information of the characters in each word segment, the phonetic annotation feature is used to characterize the phonetic annotation information of the characters in each word segment, and the part-of-speech feature is used to characterize the part-of-speech information of the characters in each word segment.

[0118] Exemplarily, a feature processing strategy can be preset in advance, and this feature processing strategy is used to provide corresponding processing rules when determining the word segmentation feature corresponding to the phonetic annotation reference text. Correspondingly, when determining the word segmentation feature corresponding to the phonetic annotation reference text, the word segmentation feature corresponding to the phonetic annotation reference text can be determined through the above feature processing strategy according to each word segment included in the phonetic annotation reference text and its corresponding part-of-speech.

[0119] In a possible implementation manner, the above S203: "Determine the word segmentation feature corresponding to the phonetic annotation reference text according to each word segment included in the phonetic annotation reference text and its corresponding part-of-speech" can include S2031 - S2032 (not shown in the figure):

[0120] S2031: For each character in the phonetic annotation reference text, determine the word segment to which the character belongs in the phonetic annotation reference text; determine the position feature corresponding to the character according to the position of the character in the word segment; determine the phonetic annotation feature corresponding to the character according to the polyphonic character detection result corresponding to the word segment; determine the part-of-speech feature corresponding to the character according to the part-of-speech corresponding to the word segment.

[0121] When it is determined the respective word segments included in the phonetic reference text and their corresponding word natures, for each character in the phonetic reference text, it is possible to determine the word segment to which each character belongs in the phonetic reference text, that is, according to the word segmentation result of the phonetic reference text, it can be determined which word or which single character each character belongs to. For example, the phonetic reference text is "the tune of Prelude to Water Melody", the respective word segments included in the phonetic reference text are "Prelude to Water Melody", "of", and "tune". According to the above word segmentation result of the phonetic reference text, it can be seen that the first "tune" in the phonetic reference text belongs to the word segment "Prelude to Water Melody", and the second "tune" in the phonetic reference text belongs to the word segment "tune".

[0122] When determining the position feature, a position feature processing strategy can be set in advance, and this position feature processing strategy is used to provide corresponding processing rules when determining the position feature corresponding to the character. Correspondingly, when determining the position feature corresponding to the character, through the above position feature processing strategy, according to the position of the character in the word segment, the position feature corresponding to the character can be determined. The position feature corresponding to the character is used to represent the position information of the character in the phonetic reference text.

[0123] As an example, the above "according to the position of the character in the word segment, determine the position feature corresponding to the character" may include:

[0124] If the character and other characters together form a word segment, then determine the position identifier used to indicate the position of the character in the word segment as the position feature corresponding to the character;

[0125] If the character independently forms a word segment, then determine the position identifier used to indicate the position of the independent character as the position feature corresponding to the character.

[0126] When it is determined the word segment to which the character in the phonetic reference text belongs, it can be determined whether the character and other characters together form a word segment (that is, the word segment is a word, and the character and other characters together form the word segment), or whether the character independently forms a word segment (that is, the word segment is the character).

[0127] If the character and other characters together form a word segment, then the position identifier used to indicate the position of the character in the word segment can be directly used as the position feature corresponding to the character. The position identifier is used to indicate the position of the character in the word segment. For example, the position identifier may include B indicating the starting character in the word, M indicating the middle character in the word, and E indicating the ending character in the word. For example, for the word segment "Prelude to Water Melody", the character "water" and other characters together form the word segment, and this character is at the starting position of the word, and the position feature corresponding to this character is B. Correspondingly, the position feature corresponding to the word segment "Prelude to Water Melody" is "BMME".

[0128] If a character independently forms a segmented word, the position identifier used to indicate the position of the independent character can be directly used as the position feature corresponding to the character. The position identifier at this time is used to indicate that the segmented word only includes one independent character. For example, the position identifier can be S representing the independent character. For example, for the two segmented words "的" and "调", both of these segmented words are independently formed by one character, and the position features corresponding to these two characters are both S.

[0129] Thus, through the above method, when the segmented word to which the character belongs is determined, according to the position of the character in the segmented word, the position feature corresponding to the character can be determined, and then the position feature corresponding to the phonetic reference text can be determined, so that in the process of determining the phonetic result of the target polyphonic character, the influence of the position where the target polyphonic character is located on the phonetic result can be considered, and thus the accuracy of phonetic annotation for the target polyphonic character can be improved from the perspective of position information.

[0130] When determining the phonetic feature, a phonetic feature processing strategy can be preset in advance. This phonetic feature processing strategy is used to provide corresponding processing rules when determining the phonetic feature corresponding to the character. Correspondingly, when determining the phonetic feature corresponding to the character, the phonetic feature corresponding to the character can be determined through the above phonetic feature processing strategy according to the polyphonic character detection result corresponding to the segmented word. The phonetic feature corresponding to the character is used to represent the phonetic information of the character in the phonetic reference text. The polyphonic character detection result corresponding to the segmented word is used to indicate whether the segmented word includes a polyphonic character. For example, the segmented word can be detected for polyphonic characters based on a preset polyphonic character dictionary to obtain the polyphonic character detection result corresponding to the segmented word. In this regard, the present application does not specifically limit the detection method for determining the polyphonic character detection result corresponding to the segmented word.

[0131] As an example, the above "determining the phonetic feature corresponding to the character according to the polyphonic character detection result corresponding to the segmented word" can include:

[0132] When the polyphonic character detection result corresponding to the segmented word indicates that the segmented word includes a polyphonic character and the segmented word can be retrieved in the dictionary, if the character is a polyphonic character in the segmented word, determine the pronunciation of the character recorded in the dictionary when it belongs to the segmented word as the phonetic feature corresponding to the character; if the character is not a polyphonic character in the segmented word, determine the default phonetic identifier as the phonetic feature corresponding to the character;

[0133] When the polyphonic character detection result corresponding to the segmented word indicates that the segmented word does not include a polyphonic character, or when the polyphonic character detection result corresponding to the segmented word indicates that the segmented word includes a polyphonic character but the segmented word cannot be retrieved in the dictionary, determine the default phonetic identifier as the phonetic feature corresponding to the character.

[0134] When the polyphone detection result corresponding to a word segment indicates that the word segment contains polyphones and the word segment can be retrieved in the dictionary, if a character is a polyphone in the word segment, the pronunciation recorded in the dictionary when the character belongs to the word segment can be directly used as the pronunciation feature corresponding to the character. If the character is not a polyphone in the word segment, the default pronunciation identifier can be directly used as the pronunciation feature corresponding to the character. The default pronunciation identifier is the default identifier used to represent the pronunciation of non-polyphone characters. For example, the default pronunciation identifier can be UNK.

[0135] For example, for the word segment "Water Melody in the Mid-Autumn Festival", the polyphone detection result of this word segment indicates that the word segment contains polyphones and the word segment can be retrieved in the dictionary to obtain the corresponding pronunciation. The character "tiao" is a polyphone in the word segment, and the pronunciation recorded in the dictionary when "tiao" belongs to "Water Melody in the Mid-Autumn Festival" can be directly used as the pronunciation feature of this character, that is, diao4. Moreover, the three characters "shui", "ge", and "tou" in this word segment are not polyphones in the word segment, so the default pronunciation identifier can be directly used as the pronunciation features corresponding to the above three characters respectively. That is, the pronunciation feature corresponding to "shui" is UNK, the pronunciation feature corresponding to "ge" is UNK, and the pronunciation feature corresponding to "tou" is UNK.

[0136] When the polyphone detection result corresponding to a word segment indicates that the word segment does not contain polyphones, or when the polyphone detection result corresponding to a word segment indicates that the word segment contains polyphones but the word segment cannot be retrieved in the dictionary, that is, when the word segment is composed of independent characters, the default pronunciation identifier can be directly used as the pronunciation feature corresponding to the character.

[0137] For example, for the word segment "de", the polyphone detection result of this word segment indicates that the word segment does not contain polyphones. Therefore, the default pronunciation identifier can be directly used as the pronunciation feature corresponding to the character. That is, the pronunciation feature corresponding to "de" is UNK. For the word segment "tiao", the polyphone detection result of this word segment indicates that the word segment contains polyphones, but the word segment is composed of the independent character "tiao". That is to say, although the character "tiao" is a polyphone, the pronunciation of this character cannot be determined based on a single character alone, that is, the word segment cannot be retrieved in the dictionary. Therefore, the default pronunciation identifier can be directly used as the pronunciation feature corresponding to the character. That is, the pronunciation feature corresponding to "tiao" is UNK.

[0138] Thus, through the above method, when the word segment to which a character belongs is determined, according to the polyphonic character detection result corresponding to the word segment, the phonetic feature corresponding to the character can be determined, and further the phonetic feature corresponding to the phonetic reference text can be determined, so that in the process of determining the phonetic result of the target polyphonic character, the influence of polyphonic characters other than the target polyphonic character in the phonetic reference text on the phonetic result can be considered, thereby improving the accuracy of phonetic annotation for the target polyphonic character from the perspective of phonetic information.

[0139] When determining the词性特征, a词性特征processing strategy can be set in advance, and this词性特征processing strategy is used to provide corresponding processing rules when determining the词性特征corresponding to a character. Accordingly, when determining the词性特征corresponding to a character, the词性特征corresponding to the character can be determined through the above词性特征processing strategy according to the词性of the word segment. The词性特征corresponding to a character is used to characterize the词性information of the character in the phonetic reference text.

[0140] As an example, the above "determining the词性特征corresponding to a character according to the词性of the word segment" may include:

[0141] Determining a词性标识indicating the词性of the word segment as the词性特征corresponding to the character.

[0142] The词性标识is used to indicate the词性of the word segment. For example, the词性标识may include n representing a noun, v representing a verb, and u representing a particle, etc. When the词性of each word segment included in the phonetic reference text is determined, the词性标识corresponding to the word segment can be directly used as the词性特征of the character belonging to the word segment.

[0143] For example, for the word segment "Shui Diao Ge Tou", the词性of this word segment is a noun, then the词性标识n representing a noun can be directly used as the词性特征of each character in this word segment, that is, the词性特征of each character in this word segment is n; for the word segment "de", the词性of this word segment is a particle, then the词性标识u representing a particle can be directly used as the词性特征of the independent character in this word segment, that is, the词性特征of the independent character "de" in this word segment is u; for the word segment "tiao", the词性of this word segment is a verb, then the词性标识v representing a verb can be directly used as the词性特征of the independent character in this word segment, that is, the词性特征of the independent character "tiao" in this word segment is v.

[0144] It should be noted that there seems to be some incorrect or unclear expressions like "词性特征" in the original text which should be replaced with appropriate and accurate terms in a real context. The above translation is based on the original text as it is.Thus, through the above method, when the word segment to which a character belongs is determined, the词性 feature corresponding to the character can be determined according to the词性 of the corresponding word segment, and then the词性 feature corresponding to the phonetic reference text can be determined. In the process of determining the phonetic result of the target polyphonic character, the influence of词性 on the phonetic result can be considered, so that the accuracy of phonetic annotation of the target polyphonic character can be improved from the perspective of词性 information.

[0145] S2032: Determine the word segment feature corresponding to the phonetic reference text according to the position feature, phonetic feature and词性 feature corresponding to each character in the phonetic reference text.

[0146] When the position feature corresponding to each character in the phonetic reference text is determined, the position features of each character in the phonetic reference text can be concatenated to obtain the position feature corresponding to the phonetic reference text. For example, the position feature corresponding to the phonetic reference text "The tune of Prelude to Water Melody" is [B, M, M, E, S, S]; when the phonetic feature corresponding to each character in the phonetic reference text is determined, the phonetic features of each character in the phonetic reference text can be concatenated to obtain the phonetic feature corresponding to the phonetic reference text. For example, the phonetic feature corresponding to the phonetic reference text "The tune of Prelude to Water Melody" is ["UNK", "diao4", "UNK", "UNK", "UNK", "UNK"]; when the词性 feature corresponding to each character in the phonetic reference text is determined, the词性 features of each character in the phonetic reference text can be concatenated to obtain the词性 feature corresponding to the phonetic reference text. For example, the词性 feature corresponding to the phonetic reference text "The tune of Prelude to Water Melody" is [n, n, n, n, u, v].

[0147] Finally, the determined position feature, phonetic feature and词性 feature corresponding to the phonetic reference text can be used as the word segment feature corresponding to the phonetic reference text.

[0148] Thus, through the above method, the position feature, phonetic feature and词性 feature corresponding to each character in the phonetic reference text can be determined. Furthermore, based on the position feature, phonetic feature and词性 feature corresponding to each character, the word segment feature corresponding to the phonetic reference text can be determined. It can be seen that through the above method, a more comprehensive and detailed feature analysis of the phonetic reference text is carried out, strengthening the influence of the context on the phonetic annotation of the target polyphonic character, thereby improving the accuracy of phonetic annotation of the target polyphonic character.

[0149] It should be understood that in practical applications, S202 and S203 can be executed simultaneously, or S202 can be executed first and then 203, or S203 can be executed first and then S202. The embodiments of this application do not limit the execution order of S202 and S203 in any way.

[0150] S204: Determine the phonetic transcription result corresponding to the target polyphonic character in the phonetic transcription reference text according to the semantic features and word segmentation features through the pronunciation classification model.

[0151] The pronunciation classification model refers to a classification model used to determine the pronunciation corresponding to a polyphonic character. After determining the semantic features and word segmentation features of the phonetic transcription reference text, the word segmentation features of the phonetic transcription reference text can be encoded to obtain a word segmentation feature vector. Then, the semantic features of the phonetic transcription reference text (the semantic features output by the semantic analysis model are themselves feature vectors) and the word segmentation feature vector can be input into the pronunciation classification model for processing. The pronunciation classification model processes based on the input semantic features and word segmentation feature vectors, and can obtain the phonetic transcription result corresponding to the target polyphonic character in the phonetic transcription reference text. The phonetic transcription result corresponding to the target polyphonic character refers to the pronunciation corresponding to the target polyphonic character. For example, when the phonetic transcription reference text is "the tune of Prelude to Water Melody", and the target polyphonic character is "tune", the phonetic transcription result corresponding to the target polyphonic character is "diao4" (diao represents the pinyin, and 4 represents the tone).

[0152] In one possible implementation manner, the above S204 "Determine the phonetic transcription result corresponding to the target polyphonic character in the phonetic transcription reference text according to the semantic features and word segmentation features through the pronunciation classification model" may include S2041 - S2045 (not shown in the figure):

[0153] S2041: Determine the input feature vector according to the semantic features and word segmentation features.

[0154] The input feature vector refers to the feature vector input into the pronunciation classification model, and the input feature vector is composed of semantic features (the semantic features are themselves semantic feature vectors) and word segmentation feature vectors.

[0155] When the word segmentation features include the position features, phonetic features, and词性 features of the phonetic transcription reference text, the position features can be encoded through position encoding to obtain a position feature vector corresponding to the position features; the phonetic features can be encoded through phonetic encoding to obtain a phonetic feature vector corresponding to the phonetic features; the词性 features can be encoded through词性 encoding to obtain a词性 feature vector corresponding to the词性 features. Among them, position encoding is used to encode the input position features to obtain a feature vector representation that can represent the position information of the position features; phonetic encoding is used to encode the input phonetic features to obtain a feature vector representation that can represent the phonetic information of the phonetic features;词性 encoding is used to encode the input词性 features to obtain a feature vector representation that can represent the词性 of the词性 features.

[0156] As an example, refer to Figure 3 , Figure 3The diagram below illustrates the structure of the pronunciation classification model provided in this application. Before encoding the positional features, phonetic features, and part-of-speech features, preprocessing is required. Specifically, the preprocessing process can involve adding a character CLS to the beginning of the positional feature to identify the starting position and a character SEP to the end of the positional feature to identify the ending position, thus obtaining the preprocessed positional features; similarly, adding a character CLS to the beginning of the phonetic feature to identify the starting position and a character SEP to the end of the phonetic feature to identify the ending position, thus obtaining the preprocessed phonetic features; and similarly, adding a character CLS to the beginning of the part-of-speech feature to identify the starting position and a character SEP to the end of the part-of-speech feature to identify the ending position, thus obtaining the preprocessed part-of-speech features.

[0157] Then, encoding processing can be performed separately for the preprocessed positional features, preprocessed phonetic features, and preprocessed part-of-speech features. The encoding process for positional features can include generating positional feature vectors corresponding to the preprocessed positional features using sine and cosine functions. The encoding process for phonetic features can include: first, using a tokenizer to convert each phonetic feature in the preprocessed phonetic features into a corresponding token; the tokens corresponding to these phonetic features are indices in a predefined dictionary. Based on the token corresponding to each phonetic feature, an embedding lookup can be performed to determine the embedding vector representation corresponding to each token, i.e., the feature vector corresponding to each phonetic feature; then, the feature vectors corresponding to each phonetic feature can be concatenated to obtain the phonetic feature vector corresponding to the phonetic feature. The encoding process for part-of-speech features can include the following steps: First, each part-of-speech feature in the preprocessed part-of-speech features can be converted into a corresponding token using a tokenizer. The tokens corresponding to the part-of-speech features are indices in a predefined dictionary. Based on the token corresponding to each part-of-speech feature, an embedding lookup can be performed to determine the embedding vector representation corresponding to each token, which is the feature vector corresponding to each part-of-speech feature. Then, the feature vectors corresponding to each part-of-speech feature can be concatenated to obtain the part-of-speech feature vector corresponding to the part-of-speech feature.

[0158] Therefore, after determining the position feature vector, phonetic feature vector, and part-of-speech feature vector, the semantic features, position feature vector, phonetic feature vector, and part-of-speech feature vector can be concatenated to obtain the input feature vector.

[0159] S2042: The gated recurrent unit in the pronunciation classification model processes the input feature vector to obtain the output feature vector.

[0160] A Gated Recurrent Unit (GRU) is a neural network architecture used to process sequential data. It is well-suited for handling time-dependent data such as time series and language data, addressing the vanishing and exploding gradient problems common to traditional Recurrent Neural Networks (RNNs) when processing long sequences. GRU employs two mechanisms: an update gate and a reset gate. The update gate helps the model decide when to discard old memories, while the reset gate determines when to combine new input information with old memories. This allows GRU to better capture long-term dependencies in long sequences.

[0161] The pronunciation classification model includes a gated recurrent unit (GRU), which extracts features from the input feature vector to obtain an output feature vector. The output feature vector refers to the output of the GRU.

[0162] Therefore, the input feature vector is fed into the gated recurrent unit in the pronunciation classification model. The gated recurrent unit can extract features from the input feature vector based on the update gate and the reset gate to obtain the output feature vector.

[0163] S2043: The input feature vector and the output feature vector are fused to obtain the fused feature vector.

[0164] After determining the output feature vector, the input and output feature vectors can be fused. This is done by adding the input and output feature vectors digit by digit to obtain the fused feature vector. The fused feature vector represents the feature vector representation after fusing the input and output feature vectors.

[0165] S2044: The classification feature vector is obtained by processing the linear layer in the pronunciation classification model based on the fused feature vector.

[0166] The linear layer is used to perform a linear transformation on the fused feature vector, mapping it to a new feature space to change its dimension. The categorical feature vector refers to the output of the linear layer; the categorical feature vector logits is used to characterize the unnormalized scores corresponding to the possible phonetic transcriptions of the target polyphonic character.

[0167] The pronunciation classification model includes a linear layer, which can be referenced. Figure 3 The linear layer nn.Linear in the algorithm takes the fused feature vector as input and performs a linear transformation to obtain the classification feature vector.

[0168] In one possible implementation, the above-mentioned S2044, "processing the linear layer in the pronunciation classification model based on the fused feature vector to obtain a classification feature vector," may include:

[0169] Based on the index of the target polyphonic character in the phonetic reference text, extract the sub-feature vector corresponding to the target polyphonic character from the fused feature vector;

[0170] The classification feature vector is obtained by processing the sub-feature vectors through a linear layer.

[0171] The index of the target polyphonic character in the phonetic reference text indicates its position within the text. The sub-feature vector corresponding to the target polyphonic character indicates the feature vector representation of relevant information about the target polyphonic character in the fused feature vector.

[0172] In the linear layer processing based on the fused feature vector, firstly, the sub-feature vector corresponding to the target polyphonic character can be extracted from the corresponding position in the fused feature vector based on the index of the target polyphonic character in the phonetic reference text. For example, if the fused feature vector has a dimension of N*D, where N represents the number of characters in the phonetic reference text and D represents the feature dimension of each character, and the target polyphonic character is located at the i-th position in the phonetic reference text, then the i-th D-dimensional feature vector can be extracted from the fused feature vector as the sub-feature vector corresponding to the target polyphonic character.

[0173] After determining the sub-feature vector corresponding to the target polyphonic character, the sub-feature vector can be input into a linear layer. By performing a linear transformation on the sub-feature vector corresponding to the target polyphonic character through the linear layer, the classification feature vector corresponding to the target polyphonic character can be obtained.

[0174] Therefore, by using the above method, after determining the fusion feature vector, the sub-feature vector corresponding to the target polyphonic character can be extracted from the fusion feature vector. Then, the sub-feature vector corresponding to the target polyphonic character is processed through a linear layer to avoid the influence of the sub-feature vectors corresponding to other characters in the phonetic reference text on the phonetic results, thereby improving the accuracy of the phonetic results of the target polyphonic character.

[0175] S2045: By classifying the target polyphonic character based on the classification feature vector through the classification layer in the pronunciation classification model, the phonetic annotation result corresponding to the target polyphonic character is determined.

[0176] The classification layer is used to classify the classification feature vectors to output the probability scores corresponding to the possible pronunciations of the target polyphonic character.

[0177] The pronunciation classification model includes a classification layer, which can be referenced. Figure 3The Weighted Softmax in the classification layer inputs the classification feature vector into the classification layer for classification, and can output the probability scores corresponding to the possible pronunciations of the target polyphonic character, that is Figure 3 The probability score of "diao4" is 0.89, the probability score of "tiao2" is 0.11, the probability score of "de2" is 0.0, and the probability score of "cha1" is 0.0. After determining the probability scores corresponding to the possible pronunciations of the target polyphonic character, the pronunciation with the largest probability score can be selected as the phonetic annotation result corresponding to the target polyphonic character.

[0178] In a possible implementation manner, the above S2045 "Through the classification layer in the pronunciation classification model, classify based on the classification feature vector to determine the phonetic annotation result corresponding to the target polyphonic character" may include:

[0179] For each candidate pronunciation supported by the pronunciation classification model, if the candidate pronunciation is a possible pronunciation of the target polyphonic character, determine that the initial weight corresponding to the candidate pronunciation is 1; if the candidate pronunciation is not a possible pronunciation of the target polyphonic character, determine that the initial weight corresponding to the candidate pronunciation is 0;

[0180] Through the classification layer, classify based on the classification feature vector to determine the target weight corresponding to each candidate pronunciation supported by the pronunciation classification model; the target weight is determined based on the initial weight. If the initial weight corresponding to the candidate pronunciation is 0, the target weight corresponding to the candidate pronunciation is also 0;

[0181] Determine the candidate pronunciation with the largest corresponding target weight as the phonetic annotation result corresponding to the target polyphonic character.

[0182] A candidate pronunciation refers to a candidate item that may be used as the pronunciation of the target polyphonic character provided by the pronunciation classification model, such as Figure 3 "diao4", "tiao2", "de2", and "cha1" in. The possible pronunciations of the target polyphonic character refer to all the pronunciations that can be retrieved in the dictionary for the target polyphonic character. Taking "调" as an example, the pronunciations that can be retrieved for "调" in the dictionary are "diao4" and "tiao2". The initial weight corresponding to the candidate pronunciation is used to indicate the initial possibility of the candidate pronunciation as the phonetic annotation result of the target polyphonic character. The target weight corresponding to the candidate pronunciation is used to indicate the final possibility of the candidate pronunciation as the phonetic annotation result predicted by the pronunciation classification model.

[0183] For each candidate pronunciation supported by the pronunciation classification model, if the candidate pronunciation is a possible pronunciation of the target polyphonic character, the initial weight corresponding to the candidate pronunciation can be set to 1. That is, if the candidate pronunciation is the same as the possible pronunciation of the target polyphonic character, the initial probability of the candidate pronunciation as the phonetic result of the target polyphonic character is set to 1, indicating that the candidate pronunciation may be used as the phonetic result of the target polyphonic character. If the candidate pronunciation is not a possible pronunciation of the target polyphonic character, the initial weight corresponding to the candidate pronunciation can be set to 0. That is, if the candidate pronunciation is not the same as the possible pronunciation of the target polyphonic character, the initial probability of the candidate pronunciation as the phonetic result of the target polyphonic character is set to 0, indicating that the candidate pronunciation cannot be used as the phonetic result of the target polyphonic character.

[0184] Then, a classification layer can be used to classify based on the classification feature vectors, determining the target weights corresponding to each candidate pronunciation supported by the pronunciation classification model, that is, the final probability of each candidate pronunciation as the target polyphonic character. For example, the process of classification based on the classification feature vectors through the classification layer can be referred to Formulas 1 and 2.

[0185] weighted_exp_logits = exp(logits - max_logits) * weights formula 1

[0186] probs = weighted_exp_logits / sum(weighted_exp_logits) Formula 2

[0187] In Formula 1, logits refers to the output of the linear layer, i.e., the classification feature vector; max_logits represents the maximum value in the classification feature vector; exp represents the indicator function; weights represents the initial weights corresponding to the candidate pronunciations determined by the above method; and weighted_exp_logits represents the probability of each candidate pronunciation after initial weighting being the phonetic result of the target polyphonic character.

[0188] In Formula 2, probs represents the target weight corresponding to the candidate pronunciation, i.e. Figure 3 The target weights for diao4, tiao2, de2, and cha1 are 0.89, 0.11, 0.0, and 0.0, respectively. `sum(weighted_exp_logits)` represents the sum of probabilities that each candidate pronunciation in the phonetic reference text will be the target polyphonic character. Furthermore, Formula 2 is used to normalize the results calculated by Formula 1, ensuring that the sum of the target weights for each candidate pronunciation is ultimately 1.

[0189] According to Formulas 1 and 2, during the classification process in the classification layer, the target weight of the candidate pronunciation is determined based on its initial weight. If the initial weight of the candidate pronunciation is 0, then the target weight of the candidate pronunciation must also be 0.

[0190] Once the target weights corresponding to each candidate pronunciation are determined, the candidate pronunciation corresponding to the maximum target weight can be used as the phonetic result for the target polyphonic character.

[0191] Therefore, using the above method, the initial weights corresponding to each candidate pronunciation supported by the pronunciation classification model can be determined in advance. Then, through the classification layer, the target weights corresponding to each candidate pronunciation can be determined. Finally, the candidate pronunciation with the largest target weight is used as the phonetic result for the target polyphonic character. It is evident that the classification layer, based on the initial weights of the candidate pronunciations, achieves the transfer of classification feature vectors, ensuring that the pronunciation classification model will not output results other than the possible pronunciations of the target polyphonic character, thereby improving the accuracy of the target polyphonic character's pronunciation.

[0192] Therefore, using the pronunciation classification model provided by the above method, the pronunciation results corresponding to the target polyphonic characters in the pronunciation reference can be determined based on semantic features and word segmentation features. It is evident that by constructing various feature vectors with reference value for the pronunciation of target polyphonic characters, and by leveraging GRU and residual concepts to enhance the pronunciation classification model's ability to understand contextual information, the accuracy of determining the correct pronunciation of target polyphonic characters based on their contextual information is improved.

[0193] In the polyphonic character phonetic annotation method provided in this application embodiment, the method includes: obtaining a phonetic reference text, which includes the target polyphonic character to be annotated and its context text; since in many cases, it is often difficult to accurately determine the corresponding pronunciation based solely on the polyphonic character itself, this application, when annotating the target polyphonic character, in addition to obtaining the target polyphonic character, also obtains its context text, and then determines the phonetic annotation result of the target polyphonic character based on the phonetic reference text composed of the target polyphonic character and its context text, thereby realizing the phonetic annotation of the target polyphonic character by referencing its context. Then, through a semantic analysis model, the semantic features corresponding to the phonetic reference text are determined, that is, the semantic features that can characterize the semantic information of the phonetic reference text are determined. Furthermore, the phonetic reference text is segmented to obtain the individual words included in the phonetic reference text, and the part-of-speech tag of each word is determined. Then, based on the individual words and their corresponding parts of speech, the segmentation features of the phonetic reference text are determined, i.e., features related to the segmentation in the phonetic reference text and valuable for annotating the target polyphonic character. Finally, using a pronunciation classification model, based on the aforementioned semantic and segmentation features, the pronunciation result corresponding to the target polyphonic character in the phonetic reference text is determined. That is, the pronunciation classification model, referring to the semantic and segmentation features of the phonetic reference text, performs a pronunciation classification task for the target polyphonic character, thereby determining the correct pronunciation of the target polyphonic character in the phonetic reference text. Thus, by referring to the semantic information of the context in which the target polyphonic character belongs, as well as the relevant information of other segmentation words in the context, accurate pronunciation of the target polyphonic character is achieved, thereby avoiding mispronunciation of the target polyphonic character when converting text to speech.

[0194] When this application embodiment is applied to annotating the pronunciation of polyphonic characters in e-book text, after determining the pronunciation result of the target polyphonic character in the pronunciation reference text, an SSML sub-file corresponding to the sentence to which the target polyphonic character belongs can be generated based on the pronunciation result of the target polyphonic character, and an SSML file corresponding to the e-book can be generated based on the SSML sub-files corresponding to each sentence in the currently processed e-book.

[0195] In one possible implementation, the method provided in this application embodiment may further include:

[0196] Based on the phonetic transcription results corresponding to the target polyphonic character, a speech synthesis markup language corresponding to the target text to which the target polyphonic character belongs is generated; the speech synthesis markup language is used to indicate the pronunciation mode of the speech when converting the target text into the corresponding speech.

[0197] The target text refers to the text that includes the target polyphonic character. For example, the target text can be the clause where the target polyphonic character belongs, or the target text can also be the paragraph where the target polyphonic character belongs, etc. In this regard, this application does not specifically limit the target text.

[0198] The Speech Synthesis Markup Language is a markup language based on the xml format, used to control the way of text-to-speech (TTS) synthesis. Through SSML, the output of speech synthesis can be adjusted more meticulously, including but not limited to the pitch, speed, volume, and pronunciation of the speech. Among them, TTS is a technology that converts text information into audible speech. TTS includes two main parts: text processing and speech synthesis. Text processing is used to convert the input text into phonemes or other speech-related symbols, and the speech synthesis part is used to convert these symbols into audible sounds. SSML is a commonly used tool for the speech synthesis part, used to control the process of speech synthesis.

[0199] For example, through SSML, it can be specified that a certain word is output in a specific pitch, or the speech output can also be adjusted by inserting a pause in a certain part of the sentence, making the speech synthesis result more natural and in line with the language habits of the public.

[0200] For example, the sentence example is: "In front of him, there are almost no difficult things", and the SSML corresponding to this sentence is: " <speak>In front of him, <p phoneme＝\"ji1\”> Several Hu <p phoneme＝\"mei2\”> without have <p phoneme＝\"nan2\”> Disaster thing.< / speak> ".

[0201] In this application, the Speech Synthesis Markup Language is used to indicate the pronunciation method of the speech when converting the target text into the corresponding speech, that is, it is used to indicate the output method of the speech corresponding to the target text. For example, it indicates the pronunciation, pitch, speed, and volume of the speech corresponding to the target text.

[0202] In the case where the phonetic notation result of the target polyphonic character is determined, according to the SSML format, based on the phonetic notation result corresponding to the target polyphonic character, an SSML sub-file corresponding to the target text where the target polyphonic character belongs can be generated. For example, taking the target polyphonic character as "diao4" and the target text where the target polyphonic character belongs as "the tune in Prelude to Water Melody", the speech synthesis markup language corresponding to the target text is: " <speak>Water Melody <p phoneme＝\"diao4\”> Adjust .< / speak> ".

[0203] Moreover, in the process of generating the SSML sub-file corresponding to the target text based on the phonetic notation result of the target polyphonic character, the pitch, speed, volume, and pronunciation method in the SSML sub-file can also be adjusted.

[0204] In practical applications, the above method can generate SSML sub-files corresponding to different sentences or paragraphs. These SSML sub-files are then concatenated according to their reading order to obtain the SSML file corresponding to the ebook. Additionally, the server can provide this SSML file to a database for storage.

[0205] Therefore, by using the above method, after determining the phonetic transcription results corresponding to the target polyphonic character, an SSML sub-file corresponding to the target text to which the target polyphonic character belongs can be generated based on the phonetic transcription results. Then, based on each SSML sub-file, the SSML file corresponding to the e-book can be determined. Since the SSML sub-file corresponding to the target text is generated based on the relatively accurate phonetic transcription results of the target polyphonic character, the phonetic transcription results in the SSML file corresponding to the e-book are also relatively accurate, thereby improving the accuracy of the speech output based on the SSML file corresponding to the e-book.

[0206] Finally, you can refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the implementation of the polyphonic character phonetic notation method provided in this application embodiment. First, the chapters of an e-book can be segmented into sentences to obtain the individual sentences included in the e-book. Then, each sentence can be preprocessed. Specifically, it can detect whether the length of the sentence is within a preset length range and whether each sentence contains preset special characters. If special characters are included, they are either deleted or replaced with preset regular characters. Afterward, phonetic reference text can be extracted from the preprocessed sentences.

[0207] After identifying the phonetic reference text, semantic features can be determined using a semantic analysis model. Subsequently, word segmentation can be performed on the preprocessed sentences to obtain the individual words in the phonetic reference text and their corresponding parts of speech.

[0208] Based on the individual words in the phonetic reference text and their corresponding parts of speech, the word segmentation features of the phonetic reference text can be determined. Finally, using a pronunciation classification model, based on the semantic and word segmentation features determined above, the phonetic annotation results corresponding to the target polyphonic characters in the phonetic reference text can be output. Furthermore, based on the phonetic annotation results corresponding to the target polyphonic characters, an SSML file of the target text to which the target polyphonic characters belong can be generated.

[0209] Furthermore, in practical applications, the database stores a large amount of ebook text content and corresponding SSML files. Based on the SSML files, users' needs for listening to ebooks can be met. For example, refer to... Figure 5 , Figure 5This is a schematic diagram of the e-book reading interface provided in an embodiment of this application. Users can enter the player interface by clicking the "Listen" control in the e-book reading interface.

[0210] For reference Figure 6 , Figure 6 This is a schematic diagram of the player interface provided in an embodiment of this application. Users can enter the e-book listening mode by clicking the playback control on this interface; alternatively, they can refer to... Figure 7 , Figure 7 This is a schematic diagram of the interface for entering the listening mode provided in the embodiments of this application. For example, a user can select the text content to be listened to by long-pressing the current sentence. In response to the long-press operation, the terminal device can display a function bar above the selected text content to be listened to. When the user clicks the "Listen to Current" control in the function bar, the user can enter the e-book listening mode.

[0211] The backend implementation method for entering e-book listening mode can include: when the backend server receives a playback operation triggered by the user based on the playback control, or a listening operation triggered by the "Listen to Current" control, it can retrieve the SSML sub-file of the current chapter of the e-book from the database. Then, it can determine the original text index of the current position based on the current listening progress bar, and then find the SSML sub-file of the corresponding sentence from the SSML sub-file based on the determined original text index, so that speech output can be performed based on the SSML sub-file of the current position.

[0212] Based on the polyphonic character phonetic notation method provided in the preceding embodiments, this application also provides a polyphonic character phonetic notation device. The following, in conjunction with... Figure 8 To explain, Figure 8 This is a schematic diagram of the structure of the polyphonic character phonetic notation device 800 provided in an embodiment of this application. The device includes:

[0213] The acquisition module 801 is used to acquire the phonetic reference text; the phonetic reference text includes the target polyphonic character to be annotated and the context text of the target polyphonic character;

[0214] The first feature determination module 802 is used to determine the semantic features corresponding to the phonetic reference text through a semantic analysis model;

[0215] The second feature determination module 803 is used to perform word segmentation processing on the phonetic reference text to obtain each word included in the phonetic reference text, and determine the part of speech corresponding to each word included in the phonetic reference text; and determine the word segmentation features corresponding to the phonetic reference text based on each word included in the phonetic reference text and its corresponding part of speech.

[0216] The phonetic annotation module 804 is used to determine the phonetic annotation result corresponding to the target polyphonic character in the phonetic annotation reference text based on the semantic features and the word segmentation features through a pronunciation classification model.

[0217] Optionally, the second feature determination module 803 includes:

[0218] The first determining unit is configured to, for each character in the phonetic reference text, determine the word segment to which the character belongs in the phonetic reference text; determine the positional feature corresponding to the character based on the position of the character in the word segment; determine the phonetic feature corresponding to the character based on the polyphonic character detection result corresponding to the word segment; and determine the part-of-speech feature corresponding to the character based on the part-of-speech corresponding to the word segment.

[0219] The second determining unit is used to determine the word segmentation features corresponding to the phonetic reference text based on the position features, phonetic features, and part-of-speech features of each character in the phonetic reference text.

[0220] Optionally, the first determining unit includes:

[0221] The first position feature determination unit is used to determine a position identifier that indicates the position of the character in the word segment if the character and other characters together constitute the word segment, and use it as the position feature corresponding to the character;

[0222] The second position feature determination unit is used to determine a position identifier for indicating an independent character as the position feature corresponding to the character if the character independently constitutes the word segment.

[0223] Optionally, the first determining unit includes:

[0224] The first phonetic feature determination unit is used to determine the pronunciation of the character recorded in the dictionary when the polyphonic character detection result corresponding to the word segment indicates that the word segment includes polyphonic characters and the word segment can be retrieved in the dictionary. If the character is the polyphonic character in the word segment, the unit determines the pronunciation of the character recorded in the dictionary when it belongs to the word segment and uses it as the phonetic feature corresponding to the character. If the character is not the polyphonic character in the word segment, the unit determines the default phonetic identifier as the phonetic feature corresponding to the character.

[0225] The second phonetic feature determination unit is used to determine the default phonetic identifier as the phonetic feature corresponding to the character when the polyphonic character detection result corresponding to the word segment indicates that the word segment does not contain polyphonic characters, or when the polyphonic character detection result corresponding to the word segment indicates that the word segment contains polyphonic characters, but the word segment cannot be retrieved in the dictionary.

[0226] Optionally, the first determining unit includes:

[0227] The part-of-speech feature determination unit is used to determine the part-of-speech identifier used to indicate the part of speech corresponding to the word segmentation, as the part-of-speech feature corresponding to the character.

[0228] Optionally, the phonetic module 804 includes:

[0229] An input feature vector determination unit is used to determine an input feature vector based on the semantic features and the word segmentation features;

[0230] The output feature vector determination unit is used to process the input feature vector through the gated loop unit in the pronunciation classification model to obtain the output feature vector;

[0231] A fusion unit is used to fuse the input feature vector and the output feature vector to obtain a fused feature vector;

[0232] The first processing unit is used to process the fused feature vector through the linear layer in the pronunciation classification model to obtain the classification feature vector;

[0233] The classification unit is used to classify the target polyphonic character based on the classification feature vector through the classification layer in the pronunciation classification model, and to determine the phonetic result corresponding to the target polyphonic character.

[0234] Optionally, the first processing unit includes:

[0235] The extraction unit is used to extract the sub-feature vector corresponding to the target polyphonic character from the fused feature vector according to the index of the target polyphonic character in the phonetic reference text;

[0236] The classification feature vector determination unit is used to process the sub-feature vector through the linear layer to obtain the classification feature vector.

[0237] Optionally, the classification unit includes:

[0238] An initial weight determination unit is used to determine the initial weight of each candidate pronunciation supported by the pronunciation classification model as 1 if the candidate pronunciation is a possible pronunciation of the target polyphonic character, and 0 if the candidate pronunciation is not a possible pronunciation of the target polyphonic character.

[0239] The target weight determination unit is used to classify based on the classification feature vector through the classification layer and determine the target weight corresponding to each candidate pronunciation supported by the pronunciation classification model; the target weight is determined based on the initial weight, and if the initial weight corresponding to the candidate pronunciation is 0, then the target weight corresponding to the candidate pronunciation is also 0;

[0240] The phonetic result determination unit is used to determine the candidate pronunciation with the largest corresponding target weight, which is used as the phonetic result corresponding to the target polyphonic character.

[0241] Optionally, the acquisition module 801 includes:

[0242] The acquisition unit is used to detect whether each sentence in the original text contains a polyphonic character; if the sentence contains the polyphonic character and the polyphonic character independently constitutes a word in the sentence, then the polyphonic character is taken as the target polyphonic character, and the phonetic reference text is extracted from the sentence based on the position of the target polyphonic character in the sentence and a preset text extraction window.

[0243] Optionally, the device further includes:

[0244] The detection unit is used to detect whether the length of each sentence in the original text is within a preset length range; if the length of the sentence is less than the lower threshold of the preset length range, the sentence is merged with other adjacent sentences into one sentence; if the length of the sentence is greater than the upper threshold of the preset length range, the sentence is split into multiple sentences.

[0245] The replacement unit is used to, for each clause in the original text, delete the special character if the clause contains a preset special character, or replace the special character with a preset regular character.

[0246] Optionally, the device further includes:

[0247] The generation unit is used to generate a speech synthesis markup language corresponding to the target text to which the target polyphonic character belongs, based on the phonetic annotation result corresponding to the target polyphonic character; the speech synthesis markup language is used to indicate the pronunciation mode of the speech when converting the target text into the corresponding speech.

[0248] This application also provides a computer device, which may specifically be a terminal device or a server. The terminal device and server provided in this application will be described below from the perspective of hardware implementation.

[0249] See Figure 9 , Figure 9 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 9As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a computer as an example:

[0250] Figure 9 This is a block diagram illustrating a portion of the structure of a computer associated with the terminal provided in an embodiment of this application. (Reference) Figure 9 The computer includes: a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230 (including a touch panel 1231 and other input devices 1232), a display unit 1240 (including a display panel 1241), a sensor 1250, an audio circuit 1260 (connected to a speaker 1261 and a microphone 1262), a wireless fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290, etc. Those skilled in the art will understand that... Figure 9 The computer architecture shown does not constitute a limitation on the computer and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0251] The memory 1220 can be used to store software programs and modules. The processor 1280 executes various computer functions and data processing by running the software programs and modules stored in the memory 1220. The memory 1220 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer (such as audio data, telephone directory, etc.). In addition, the memory 1220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0252] The processor 1280 is the control center of the computer, connecting various parts of the computer through various interfaces and lines. It performs various computer functions and processes data by running or executing software programs and / or modules stored in the memory 1220, and by calling data stored in the memory 1220. Optionally, the processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 1280.

[0253] In this embodiment of the application, the processor 1280 included in the terminal is used to execute the steps in the polyphonic character phonetic notation method described in the foregoing embodiments.

[0254] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a server 1300 provided in an embodiment of this application. The server 1300 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1322 (e.g., one or more processors) and memory 1332, and one or more storage media 1330 (e.g., one or more mass storage devices) for storing application programs 1342 or data 1344. The memory 1332 and storage media 1330 can be temporary or persistent storage. The program stored in the storage media 1330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1322 may be configured to communicate with the storage media 1330 and execute the series of instruction operations stored in the storage media 1330 on the server 1300.

[0255] Server 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input / output interfaces 1358, and / or one or more operating systems, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0256] The steps performed by the server in the above embodiments can be based on this Figure 10 The server structure shown.

[0257] The CPU 1322 is used to execute the steps in the polyphonic character phonetic notation method described in the foregoing embodiments.

[0258] This application also provides a computer-readable storage medium for storing a computer program that performs the steps in the polyphonic character phonetic notation methods described in the foregoing embodiments.

[0259] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in the polyphonic character phonetic notation methods described in the foregoing embodiments.

[0260] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0261] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0262] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0263] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0264] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0265] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0266] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for annotating the pronunciation of polyphonic characters, characterized in that, The method includes: Obtain the phonetic reference text; the phonetic reference text includes the target polyphonic character to be phonetized and the context text of the target polyphonic character; The semantic features corresponding to the phonetic reference text are determined by using a semantic analysis model; The phonetic reference text is segmented to obtain each segment of the phonetic reference text, and the part of speech of each segment of the phonetic reference text is determined; based on each segment of the phonetic reference text and its corresponding part of speech, the segmentation features of the phonetic reference text are determined. Using a pronunciation classification model, the pronunciation results corresponding to the target polyphonic character in the pronunciation reference text are determined based on the semantic features and the word segmentation features.

2. The method according to claim 1, characterized in that, The step of determining the word segmentation features corresponding to the phonetic reference text based on each word segmentation included in the phonetic reference text and its corresponding part of speech includes: For each character in the phonetic reference text, determine the word segment to which the character belongs in the phonetic reference text; determine the positional feature corresponding to the character based on the position of the character in the word segment; determine the phonetic feature corresponding to the character based on the polyphonic character detection result corresponding to the word segment; determine the part-of-speech feature corresponding to the character based on the part of speech corresponding to the word segment. Based on the positional features, phonetic features, and part-of-speech features of each character in the phonetic reference text, the word segmentation features corresponding to the phonetic reference text are determined.

3. The method according to claim 2, characterized in that, The step of determining the positional feature corresponding to the character based on its position in the word segmentation includes: If the character together with other characters constitutes the word segment, then a position identifier used to indicate the position of the character in the word segment is determined as the position feature corresponding to the character; If the characters independently constitute the word segmentation, then a position identifier used to indicate the independent characters is determined as the position feature corresponding to the characters.

4. The method according to claim 2, characterized in that, The step of determining the phonetic features corresponding to the character based on the polyphonic character detection results corresponding to the word segmentation includes: When the detection result of the polyphonic character corresponding to the word segment indicates that the word segment includes polyphonic characters and the word segment can be retrieved in the dictionary, if the character is the polyphonic character in the word segment, then the pronunciation of the character recorded in the dictionary when it belongs to the word segment is determined as the phonetic feature corresponding to the character; if the character is not the polyphonic character in the word segment, then the default phonetic identifier is determined as the phonetic feature corresponding to the character. When the polyphonic character detection result corresponding to the word segment indicates that the word segment does not contain polyphonic characters, or when the polyphonic character detection result corresponding to the word segment indicates that the word segment contains polyphonic characters, but the word segment cannot be retrieved in the dictionary, the default phonetic identifier is determined as the phonetic feature corresponding to the character.

5. The method according to claim 2, characterized in that, The step of determining the part-of-speech feature corresponding to the character based on the part-of-speech tag corresponding to the word segmentation includes: A part-of-speech identifier is determined to indicate the part of speech corresponding to the word segmentation, and this identifier is used as the part-of-speech feature corresponding to the character.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the phonetic annotation result corresponding to the target polyphonic character in the phonetic reference text based on the semantic features and the word segmentation features using a pronunciation classification model includes: Based on the semantic features and the word segmentation features, determine the input feature vector; The gated recurrent unit in the pronunciation classification model processes the input feature vector to obtain the output feature vector. The input feature vector and the output feature vector are fused to obtain a fused feature vector; The classification feature vector is obtained by processing the fused feature vector through the linear layer in the pronunciation classification model; The phonetic annotation result corresponding to the target polyphonic character is determined by classifying the target polyphonic character based on the classification feature vector through the classification layer in the pronunciation classification model.

7. The method according to claim 6, characterized in that, The process of obtaining a classification feature vector by processing the fused feature vector through the linear layer of the pronunciation classification model includes: Based on the index of the target polyphonic character in the phonetic reference text, extract the sub-feature vector corresponding to the target polyphonic character from the fused feature vector; The classification feature vector is obtained by processing the sub-feature vector through the linear layer.

8. The method according to claim 6, characterized in that, The step of classifying the target polyphonic character based on the classification feature vector through the classification layer in the pronunciation classification model to determine the phonetic annotation result includes: For each candidate pronunciation supported by the pronunciation classification model, if the candidate pronunciation is a possible pronunciation of the target polyphonic character, then the initial weight corresponding to the candidate pronunciation is determined to be 1; if the candidate pronunciation is not a possible pronunciation of the target polyphonic character, then the initial weight corresponding to the candidate pronunciation is determined to be 0. The classification layer performs classification based on the classification feature vector to determine the target weight corresponding to each candidate pronunciation supported by the pronunciation classification model. The target weight is determined based on the initial weight. If the initial weight corresponding to the candidate pronunciation is 0, then the target weight corresponding to the candidate pronunciation is also 0. The candidate pronunciation with the highest corresponding target weight is determined as the phonetic result corresponding to the target polyphonic character.

9. The method according to any one of claims 1 to 8, characterized in that, The process of obtaining the phonetic reference text includes: For each clause in the original text, detect whether the clause contains a polyphonic character; if the clause contains the polyphonic character and the polyphonic character independently constitutes a word in the clause, then the polyphonic character is taken as the target polyphonic character, and based on the position of the target polyphonic character in the clause and a preset text extraction window, the phonetic reference text is extracted from the clause.

10. The method according to claim 9, characterized in that, Before obtaining the phonetic reference text, the method further includes: For each sentence in the original text, the length of the sentence is checked to see if it is within a preset length range. If the length of the sentence is less than the lower threshold of the preset length range, the sentence is merged with other adjacent sentences into one sentence. If the length of the sentence is greater than the upper threshold of the preset length range, the sentence is split into multiple sentences. For each clause in the original text, if the clause contains a preset special character, then the special character is deleted, or the special character is replaced with a preset regular character.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: Based on the phonetic transcription results corresponding to the target polyphonic character, a speech synthesis markup language corresponding to the target text to which the target polyphonic character belongs is generated; the speech synthesis markup language is used to indicate the pronunciation mode of the speech when converting the target text into the corresponding speech.

12. A device for annotating the pronunciation of polyphonic characters, characterized in that, The device includes: The acquisition module is used to acquire the phonetic reference text; the phonetic reference text includes the target polyphonic character to be phonetized and the context text of the target polyphonic character; The first feature determination module is used to determine the semantic features corresponding to the phonetic reference text through a semantic analysis model; The second feature determination module is used to perform word segmentation processing on the phonetic reference text to obtain each word included in the phonetic reference text, and determine the part of speech corresponding to each word included in the phonetic reference text; based on each word included in the phonetic reference text and its corresponding part of speech, determine the word segmentation features corresponding to the phonetic reference text. The phonetic annotation module is used to determine the phonetic annotation result corresponding to the target polyphonic character in the phonetic annotation reference text based on the semantic features and the word segmentation features using a pronunciation classification model.

13. A computer device, characterized in that, The device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the polyphonic character phonetic notation method according to any one of claims 1 to 11, based on the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed by an electronic device, implements the method for phonetic notation of polyphonic characters as described in any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for phonetic notation of polyphonic characters as described in any one of claims 1 to 11.

Images