Multi-syllable word disambiguation method, device, equipment, storage medium and program product

Abstract

The application discloses a multi-sound character disambiguation method and device, equipment, storage medium and program product, relates to the technical field of natural language processing, and comprises the following steps: for a target multi-sound character in a target text, based on the association relationship between the plurality of soft connections corresponding to the target multi-sound character and the target text, the plurality of soft connections are coded to obtain target coding features of the plurality of soft connections; each soft connection is composed of at least the target multi-sound character or at least two continuous characters containing the target multi-sound character in the target text; the target multi-sound character is classified based on the target coding features of the soft connections to obtain a probability distribution of the target multi-sound character corresponding to the soft connection; the probability distribution represents the pronunciation of the target multi-sound character in the soft connection; a plurality of pronunciations are all pronunciations of all multi-sound characters in a preset multi-sound character set; and the pronunciations of the target multi-sound character in the target text are determined based on the plurality of probability distributions of the target multi-sound character. The application improves the accuracy of multi-sound character disambiguation.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, apparatus, device, storage medium, and program product for disambiguating polyphonic characters. Background Technology

[0002] Polyphonic characters are a widespread linguistic phenomenon in Chinese. If a Chinese character has multiple pronunciations, then that character is a polyphonic character. In speech synthesis and question-and-answer scenarios, it is necessary to select the correct pronunciation for Chinese characters with multiple pronunciations. This process is called polyphonic character disambiguation.

[0003] Current mainstream solutions for disambiguating polyphonic characters are based on classification models and polyphonic word dictionaries. This approach requires segmenting the text to obtain segments containing polyphonic characters, then searching for explanations of these segments in the polyphonic word dictionary, and finally classifying the polyphonic characters based on these explanations to determine their pronunciation. This approach heavily relies on the accuracy of segmentation and the accuracy of the explanations in the polyphonic word dictionary. In other words, if segmentation errors or incorrect explanations occur, the correct pronunciation of the polyphonic characters cannot be guaranteed. Therefore, the accuracy of polyphonic character disambiguation tasks needs further improvement. Summary of the Invention

[0004] In view of the above problems, this application provides a method, apparatus, device, storage medium, and program product for disambiguating polyphonic characters, so as to improve the accuracy of polyphonic character disambiguation. The specific solution is as follows:

[0005] The first aspect of this application provides a method for disambiguating polyphonic characters, including:

[0006] For a target polyphonic character in the target text, based on the association relationship between multiple soft links corresponding to the target polyphonic character and the target text, each soft link is encoded to obtain the target encoding feature of each soft link; each soft link is composed of the target polyphonic character or composed of at least two consecutive characters in the target text containing the target polyphonic character, and the maximum number of characters contained in each soft link is a preset number;

[0007] Based on the target encoding features of each soft link, the target polyphonic characters in each soft link are classified to obtain the probability distribution of the target polyphonic characters corresponding to each soft link; the probability distribution is the probability of the target polyphonic character belonging to each of several pronunciations, representing the pronunciation of the target polyphonic character in the soft link; the several pronunciations are all pronunciations of all polyphonic characters in a preset set of polyphonic characters;

[0008] Voting is performed based on the probability distribution of the target polyphonic characters corresponding to each soft link to determine the pronunciation of the target polyphonic characters in the target text.

[0009] In one possible implementation, encoding each soft link based on the association between the target polyphonic character and the target text includes:

[0010] Each character in the target text is encoded to obtain the encoding features of each character;

[0011] For each soft link corresponding to the target polyphonic character, the encoding features of each character contained in the soft link are fused to obtain the initial encoding features of the soft link;

[0012] The initial encoding features of the soft link are fused with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft link.

[0013] In one possible implementation, the initial encoding features of the soft link are fused with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft link, including:

[0014] Using the initial encoding features of the soft link and the encoding features of each character in the target text, calculate the attention weight of the soft link to each character in the target text;

[0015] Based on the attention weights of each character in the target text applied by the soft link, the encoding features of each character in the target text are weighted and summed to obtain the target encoding features of the soft link.

[0016] In one possible implementation, the voting based on the probability distribution of the target polyphonic characters corresponding to each of the soft links includes:

[0017] By fusing the probabilities of the target polyphonic characters belonging to the same pronunciation in the probability distribution of each soft link, the target probability of the target polyphonic character belonging to each pronunciation is obtained.

[0018] The pronunciation corresponding to the maximum target probability is determined as the pronunciation of the target polyphonic character in the target text.

[0019] In one possible implementation, the voting based on the probability distribution of the target polyphonic characters corresponding to each of the soft links includes:

[0020] Based on the probability distribution of the target polyphonic character corresponding to each soft link, the pronunciation of the target polyphonic character in each soft link is determined;

[0021] The pronunciations of the target polyphonic character in each of the soft links are statistically analyzed, and the pronunciation with the most occurrences is determined as the pronunciation of the target polyphonic character in the target text.

[0022] One possible implementation also includes:

[0023] If the number of multiple pronunciations of the target polyphonic character is the same, the pronunciation with the highest probability is determined as the pronunciation of the target polyphonic character in the target text;

[0024] Alternatively, if the number of multiple pronunciations of the target polyphonic character is the same, the probabilities of the target polyphonic character belonging to the same pronunciation in the probability distribution of the target polyphonic character corresponding to each soft link are merged to obtain the target probability of the target polyphonic character belonging to each pronunciation; the pronunciation corresponding to the maximum target probability is determined as the pronunciation of the target polyphonic character in the target text.

[0025] In one possible implementation, each character in the target text is encoded to obtain the encoding features of each character. For each soft link corresponding to the target polyphonic character, the initial encoding features of the soft link are fused with the encoding features of each character in the target text based on a cross-attention mechanism to obtain the target encoding features of the soft link. The process of classifying the target encoding features of each soft link includes:

[0026] The encoding module of the classification model encodes each character in the target text to obtain the encoding features of each character;

[0027] For each soft connection corresponding to the target polyphonic character, the connection module of the classification model fuses the encoding features of each character contained in the soft connection to obtain the initial encoding features of the soft connection.

[0028] For each soft connection, the attention module of the classification model fuses the initial encoding features of the soft connection with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft connection.

[0029] The classification module of the classification model classifies the target polyphonic characters in each soft link based on the target encoding features of each soft link, thereby obtaining the probability distribution of the target polyphonic characters corresponding to each soft link.

[0030] In one possible implementation, the classification model is trained as follows:

[0031] The encoding module encodes each character in the text sample to obtain the encoding features of each character;

[0032] For each polyphonic character in the text sample, the connection module obtains the initial encoding features of multiple soft links corresponding to that polyphonic character; wherein each soft link is composed of the polyphonic character or composed of at least two consecutive characters in the text sample containing the polyphonic character, and the maximum number of characters contained in each soft link is a preset number; the initial encoding features of each soft link are obtained by fusing the encoding features of each character contained in the soft link;

[0033] For each soft connection, the attention module fuses the initial encoding features of the soft connection with the encoding features of each character in the text sample based on the cross-attention mechanism to obtain the target encoding features of the soft connection.

[0034] The classification module classifies the polyphonic character in each soft link based on the target encoding features of each soft link, thereby obtaining the probability distribution of the polyphonic character corresponding to the soft link.

[0035] The first pronunciation of each polyphonic character corresponding to each soft link in the text sample is determined based on the probability distribution of each polyphonic character.

[0036] For each polyphonic character in the text sample, the second pronunciation of each polyphonic character in the text sample is determined by voting based on the multiple probability distributions of the polyphonic character determined by each soft connection corresponding to the polyphonic character.

[0037] The parameters of the classification model are updated with the goal of ensuring that the first and second pronunciations of each polyphonic character are close to the pronunciation label of that polyphonic character.

[0038] A second aspect of this application provides a polyphonic character disambiguation device, comprising:

[0039] The soft link unit is used to encode each soft link based on the association relationship between the multiple soft links corresponding to the target polyphonic characters in the target text, thereby obtaining the target encoding features of each soft link; each soft link is composed of the target polyphonic character or is composed of at least two consecutive characters in the target text containing the target polyphonic character, and the maximum number of characters contained in each soft link is a preset number;

[0040] A classification unit is used to classify the target polyphonic characters in each soft link based on the target encoding features of each soft link, and obtain the probability distribution of the target polyphonic characters corresponding to each soft link; the probability distribution is the probability of the target polyphonic character belonging to each of several pronunciations, representing the pronunciation of the target polyphonic character in the soft link; the several pronunciations are all pronunciations of all polyphonic characters in a preset set of polyphonic characters;

[0041] A voting unit is used to vote based on the probability distribution of the target polyphonic characters corresponding to each soft link, so as to determine the pronunciation of the target polyphonic characters in the target text.

[0042] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the polyphonic character disambiguation method described in the first aspect or any implementation thereof.

[0043] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0044] The memory is used to store computer programs;

[0045] The processor is used to execute the computer program so that the electronic device can implement the polyphonic character disambiguation method of the first aspect or any implementation thereof.

[0046] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to perform a polyphonic character disambiguation method as described in the first aspect or any implementation thereof.

[0047] Using the above technical solution, the polyphonic character disambiguation method, apparatus, device, storage medium, and program product provided in this application, for a target polyphonic character in a target text, encodes each soft link based on the association relationship between multiple soft links corresponding to the target polyphonic character and the target text, obtaining target encoding features for each soft link; each soft link consists of the target polyphonic character or consists of at least two consecutive characters in the target text containing the target polyphonic character, and the maximum number of characters contained in each soft link is a preset number; based on the target encoding features of each soft link, the target polyphonic characters in each soft link are classified to obtain the probability distribution of the target polyphonic characters corresponding to each soft link; this probability distribution is the probability of the target polyphonic character belonging to each of several pronunciations, representing the pronunciation of the target polyphonic character in the soft link; several pronunciations are all pronunciations of all polyphonic characters in a preset set of polyphonic characters; voting is performed based on the probability distribution of the target polyphonic characters corresponding to each soft link to determine the pronunciation of the target polyphonic character in the target text. As can be seen, the polyphonic character disambiguation scheme of this application proposes the concept of soft links. Without relying on word segmentation tools, it extracts polyphonic characters and their surrounding context as soft links. Based on the association between each soft link corresponding to the target polyphonic character and the target text, it determines the target encoding features of each soft link corresponding to the target polyphonic character. Based on the target encoding features of each soft link, it classifies the target polyphonic characters in each soft link, obtaining multiple possible classification results for the target polyphonic characters. Based on the classification results, it votes to determine the pronunciation of the target polyphonic character in the target text. This process does not require word segmentation of the target text, thus avoiding disambiguation errors due to word segmentation errors. It also does not require the use of explanatory information from word segmentation containing polyphonic characters, thus avoiding disambiguation errors due to explanatory information errors. Therefore, it avoids dependence on word segmentation and explanatory information for polyphonic character disambiguation, improving the accuracy of polyphonic character disambiguation. Attached Figure Description

[0048] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0049] Figure 1 A flowchart illustrating an implementation of the polyphonic character disambiguation method provided in this application;

[0050] Figure 2 An example of a soft link provided for this application;

[0051] Figure 3 The flowchart below illustrates an implementation of the method provided in this application, which encodes each soft link based on the association between the target polyphonic character and the target text, thereby obtaining the target encoding features of each soft link.

[0052] Figure 4 A flowchart illustrating an implementation of voting based on various probability distributions of the target polyphonic character provided in this application;

[0053] Figure 5 A flowchart illustrating another implementation of voting based on the probability distributions of the target polyphonic characters provided in this application;

[0054] Figure 6 A schematic diagram of the structure of the classification model provided in this application;

[0055] Figure 7 A schematic diagram of a polyphonic character disambiguation device provided in this application;

[0056] Figure 8 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0057] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0058] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0059] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0060] To avoid the strong reliance of polyphonic character disambiguation on the accuracy of word segmentation and word segmentation explanation information, the proposed solution is presented in this application.

[0061] like Figure 1 The diagram shown is a flowchart of one implementation of the polyphonic character disambiguation method provided in this application, which may include:

[0062] Step S101: For the target polyphonic characters in the target text, based on the association between the multiple soft links corresponding to the target polyphonic characters and the target text, each soft link is encoded to obtain the target encoding features of each soft link; each soft link consists of the target polyphonic character or consists of at least two consecutive characters in the target text containing the target polyphonic character, and the maximum number of characters contained in each soft link is a preset number.

[0063] The target text contains at least one polyphonic character, and the target polyphonic character is any polyphonic character in the target text.

[0064] Optionally, in some scenarios, such as question-and-answer scenarios, the target polyphonic character can be specified by the user. For example, a user provides a sentence and asks what the pronunciation of a certain polyphonic character in that sentence is.

[0065] Optionally, in some scenarios, such as speech synthesis, the target polyphonic characters can be automatically identified. For example, for each character in the target text, the system can search for whether the character exists in a preset set of polyphonic characters (also known as a polyphonic character dictionary). If it exists, the character is determined to be a polyphonic character; otherwise, the character is determined not to be a polyphonic character.

[0066] Among the multiple symbolic links corresponding to the target polyphonic character, the number of characters contained in different symbolic links may be the same or different.

[0067] In different symbolic links containing the same number of characters, the target polyphonic character is positioned differently within the symbolic link.

[0068] The soft link with the fewest characters is one soft link containing 1 character (i.e., the target polyphonic character), and the soft link with the most characters is one soft link containing N (i.e., the preset number of characters).

[0069] The target polyphonic character corresponds to n soft links containing n (n=1, 2, 3, ..., N) characters. Based on this, the total number M of soft links corresponding to the target polyphonic character is:

[0070] M = 1 + 2 + 3 + ... + N = N(N+1) / 2

[0071] When the number of characters n in the soft link is greater than 1, one of these n characters is the target polyphonic character. For the other n-1 characters, these n-1 characters can be the n-1 consecutive characters in the target text adjacent to the target polyphonic character; or, the first k characters (k is a positive integer less than n-1) of these n-1 characters are the k characters preceding the target polyphonic character and adjacent to it, and the last n-1-k characters are the n-1-k characters following the target polyphonic character and adjacent to it; or, the last k characters of these n-1 characters are the k consecutive characters following and adjacent to the target polyphonic character in the target text, and the first n-1-k characters are preset information representing blank space, that is, the target polyphonic character is the first character in the target text; or, the first k characters of these n-1 characters... The first n-1-k characters are the k consecutive characters in the target text that precede and are adjacent to the target polyphonic character, and the last n-1-k characters are pre-defined information representing blank spaces. That is, the target polyphonic character is the last character in the target text; or, the last k characters in the n-1 characters are the k consecutive characters in the target text that follow and are adjacent to the target polyphonic character, and the first n-1-k characters are all the characters in the target text that precede and are adjacent to the target polyphonic character, plus the pre-defined information representing blank spaces; or, the first k characters in the n-1 characters are the k consecutive characters in the target text that precede and are adjacent to the target polyphonic character, and the last n-1-k characters are all the characters in the target text that follow and are adjacent to the target polyphonic character, plus the pre-defined information representing blank spaces.

[0072] In n soft links containing n characters, the target polyphonic character is positioned differently in different soft links.

[0073] As an example, N=4. Of course, N can also take other values, such as N=3, or N=5, N=6, etc.

[0074] like Figure 2 The image shows an example of a soft link provided in an embodiment of this application. In this example, the target text consists of 9 characters, from t1 to t9, where t5 is a polyphonic character. The soft link corresponding to this polyphonic character can contain a maximum of 4 characters. Based on this, each polyphonic character corresponds to 10 soft links.

[0075] When encoding each soft link, this application considers the association between the soft link and the target text. Thus, the encoding features of each soft link carry information about the soft link itself, as well as the association between the soft link and the target text (e.g., semantic relevance).

[0076] Step S102: Based on the target encoding features of each soft connection, classify the target polyphonic characters in the soft connection to obtain the probability distribution of the target polyphonic characters corresponding to the soft connection; the probability distribution is the probability of the target polyphonic character belonging to each of several pronunciations, and the probability distribution represents the pronunciation of the target polyphonic character in the soft connection; several pronunciations are all pronunciations of all polyphonic characters in the preset polyphonic character set.

[0077] For the i-th (i=1, 2, 3, ..., M) soft connection corresponding to the target polyphonic character, the target polyphonic character in the i-th soft connection is classified based on the target encoding features of the i-th soft connection, resulting in the probability distribution of the target polyphonic character corresponding to the i-th soft connection (denoted as the i-th probability distribution of the target polyphonic character), which is the i-th probability distribution of the target polyphonic character. Obviously, this application obtains M probability distributions for the target polyphonic character.

[0078] Step S103: Based on the probability distribution of each target polyphonic character corresponding to each soft link, vote to determine the pronunciation of the target polyphonic character in the target text (for ease of description and distinction, it is referred to as the target pronunciation).

[0079] Soft voting can be performed based on the probability distributions of the target polyphonic character to determine the target pronunciation of the target polyphonic character in the target text.

[0080] Alternatively, hard voting can be performed based on the probability distributions of the target polyphonic character to determine its target pronunciation in the target text.

[0081] Alternatively, hard and soft voting can be performed based on the probability distributions of the target polyphonic character to determine its target pronunciation in the target text.

[0082] The polyphonic character disambiguation method provided in this application proposes the concept of soft links. Without relying on word segmentation tools, it extracts polyphonic characters and their surrounding context as soft links. Based on the association between each soft link corresponding to the target polyphonic character and the target text, it determines the target encoding features of each soft link. Based on the target encoding features of each soft link, it classifies the target polyphonic characters in each soft link, obtaining multiple possible classification results. Based on the classification results, it votes to determine the pronunciation of the target polyphonic character in the target text. This process does not require word segmentation of the target text, thus avoiding disambiguation errors due to word segmentation errors. It also does not require the use of explanatory information from word segmentation containing polyphonic characters, thus avoiding disambiguation errors due to explanatory information errors. Therefore, it avoids dependence on word segmentation and explanatory information for polyphonic character disambiguation, improving the accuracy of polyphonic character disambiguation.

[0083] Furthermore, in polyphonic word disambiguation schemes based on polyphonic word dictionaries, as the number of words containing polyphonic characters in the dictionary increases, the efficiency of word search (i.e., word segmentation containing polyphonic characters) decreases, ultimately directly affecting the efficiency of the polyphonic word disambiguation task. This application, however, does not require a polyphonic word dictionary; therefore, it is not affected by word search efficiency and can improve the accuracy of polyphonic word disambiguation while ensuring the execution efficiency of the task.

[0084] In an optional embodiment, the flowchart of one method for encoding each soft link based on the association relationship between the target polyphonic character and the target text to obtain the target encoding features of each soft link is shown below. Figure 3 As shown, it may include:

[0085] Step S301: Encode each character in the target text to obtain the encoding features of each character.

[0086] As an example, an encoding network can be used to encode each character in the target text, obtaining the encoding features of each character. The encoding features of each character are word embeddings that incorporate contextual information. The structure of this encoding network can adopt the network structure of the encoding module in a pre-trained language model. The language model can be, but is not limited to, any of the following:

[0087] BERT (Bidirectional Encoder Representation from Transformers) model, ELMo (Embeddings from Language Models) model, GPT (Generative Pre-trained Transformer) model, T5 (Text-to-Text Transfer Transformer) model, etc.

[0088] Step S302: For each soft link corresponding to the target polyphonic character, fuse the encoding features of each character contained in the soft link to obtain the initial encoding features of the soft link.

[0089] When a soft link contains preset information representing blanks, the encoding features of the preset information are pre-defined encoding features, such as an all-zero vector.

[0090] Optionally, for the i-th soft link, the weighted sum of the encoding features of each word contained in the i-th soft link can be used to obtain the initial encoding features of the i-th soft link. The initial encoding features of each soft link are a vector.

[0091] As an example, the sum of the weights of each word in the i-th soft link is 1, and the weights of each word are the same. That is, the average value of the encoding features of each word contained in the i-th soft link is calculated to obtain the initial encoding features of the i-th soft link.

[0092] As an example, the sum of the weights of all characters in the i-th soft link is 1, and the weights of different characters may be the same or different. For example, the target polyphonic character has the largest weight, and the farther away from the target polyphonic character, the smaller the weight.

[0093] Step S303: Based on the cross-attention mechanism, the initial encoding features of the soft connection are fused with the encoding features of each character in the target text to obtain the target encoding features of the soft connection.

[0094] In other words, the target encoding features of the i-th soft link not only include the information of each word in the i-th soft link, but also the information of each word in the target text, as well as the association between the i-th soft link and each word in the target text.

[0095] Based on the attention weights of the i-th soft connection to each character in the target text, the initial encoding features of the i-th soft connection can be fused with the encoding features of each character in the target text to obtain the target encoding features of the soft connection.

[0096] Optionally, the initial encoding features of the i-th soft link can be fused with the encoding features of each character in the target text to obtain the target encoding features of the i-th soft link in the following manner:

[0097] Using the initial encoding features of the i-th soft connection and the encoding features of each character in the target text, calculate the attention weight of the i-th soft connection to each character in the target text.

[0098] For the j-th character (j=1, 2, 3, ..., J; J is the total number of characters in the target text) in the target text, the attention weight of the i-th soft connection to the j-th character can be calculated based on the initial encoding features of the i-th soft connection and the encoding features of the j-th character. As an example, the similarity between the initial encoding features of the i-th soft connection and the encoding features of the j-th character can be calculated, and after reducing the similarity, it can be normalized to obtain the attention weight of the i-th soft connection to the j-th character in the target text.

[0099] The target encoding features and character encoding features of a soft link are vectors of the same length. The similarity between the initial encoding features of the i-th soft link and the encoding features of the j-th character can be multiplied by a reduction factor (a positive number less than 1) to reduce the similarity. The reduction factor can be determined based on the length of the target encoding features of the soft link. For example, the reduction factor can be the reciprocal of the square root of the target encoding feature length. The reduced similarity can then be normalized using the softmax function to obtain the attention weight of the i-th soft link to the j-th character in the target text.

[0100] Based on the attention weights of each character in the target text by the i-th soft connection, the encoding features of each character in the target text are weighted and summed to obtain the target encoding features of the soft connection.

[0101] The weight of the encoded feature of the j-th character is the attention weight of the i-th soft connection to the j-th character.

[0102] In an optional embodiment, when voting based on the probability distribution of the target polyphonic characters corresponding to each soft link, a soft voting method can be used, a hard voting method can be used, or a combination of the two voting methods can be used.

[0103] Optionally, when using soft voting, the flowchart for one implementation of voting based on the probability distribution of the target polyphonic characters corresponding to each soft link is as follows: Figure 4 As shown, it may include:

[0104] Step S401: Combine the probabilities of the target polyphonic character belonging to the same pronunciation in the probability distribution of each soft link to obtain the target probability of the target polyphonic character belonging to each pronunciation.

[0105] Assuming the target polyphonic character corresponds to M soft links, and the number of all pronunciations of all polyphonic characters in the pre-defined polyphonic character set is D, then for each of the M soft links, a probability distribution is obtained for the target polyphonic character. Therefore, a total of M probability distributions are obtained for the target polyphonic character. The i-th probability distribution (i=1, 2, 3, ..., M) represents the probability that the target polyphonic character corresponding to the i-th soft link belongs to any of the D pronunciations. In other words, for the target polyphonic character and the d-th (d=1, 2, 3, ..., D) pronunciation among the D pronunciations, this application obtains M probabilities that the target polyphonic character belongs to the d-th pronunciation.

[0106] This application fuses the M probabilities corresponding to the d-th pronunciation from M probability distributions to obtain the target probability that the target polyphonic character belongs to the d-th pronunciation. Optionally, the average of the M probabilities corresponding to the d-th pronunciation can be taken to obtain the target probability that the target polyphonic character belongs to the d-th pronunciation.

[0107] Step S402: Determine the pronunciation corresponding to the maximum target probability as the pronunciation of the target polyphonic character in the target text.

[0108] Optionally, in the case of hard voting, another implementation flowchart of voting based on the probability distribution of the target polyphonic characters corresponding to each soft link is as follows: Figure 5 As shown, it may include:

[0109] Step S501: Based on the probability distribution of the target polyphonic character corresponding to each soft link, determine the pronunciation of the target polyphonic character in each soft link.

[0110] For the i-th probability distribution among M probability distributions, the pronunciation corresponding to the highest probability in the i-th probability distribution can be determined as one pronunciation of the target polyphonic character (that is, the pronunciation of the target polyphonic character in the i-th soft link can be recorded as the i-th pronunciation of the target polyphonic character). Thus, M pronunciations are determined for the corresponding target polyphonic character.

[0111] Step S502: Statistically analyze the pronunciations of the target polyphonic character in each soft link, and determine the pronunciation with the most occurrences as the pronunciation of the target polyphonic character in the target text.

[0112] The M pronunciations are statistically analyzed to determine the number of identical pronunciations (i.e., the number of votes for each pronunciation of the target polyphonic character). The pronunciation with the most votes is then determined as the pronunciation of the target polyphonic character in the target text.

[0113] Optionally, in some cases, the different pronunciations of the target polyphonic character may have the same number of votes. For example, if the target polyphonic character has two pronunciations and corresponds to 10 soft links, the two pronunciations may both have 5 or 4 votes. In this case, the maximum probability corresponding to the two pronunciations with the most votes can be compared, and the pronunciation corresponding to the larger probability of the two maximum probabilities can be determined as the pronunciation of the target polyphonic character in the target text.

[0114] Optionally, if the number of votes for different pronunciations of a target polyphonic character is the same, a soft voting method can be used to determine the pronunciation of the target polyphonic character in the target text. That is, the probabilities of the target polyphonic character belonging to the same pronunciation in the probability distribution of each soft link are merged to obtain the target probability of the target polyphonic character belonging to each pronunciation; the pronunciation corresponding to the highest target probability is determined as the pronunciation of the target polyphonic character in the target text.

[0115] In an optional embodiment, the above-described encoding of each character in the target text yields the encoding features of each character. For each soft connection corresponding to a target polyphonic character, the initial encoding features of the soft connection are fused with the encoding features of each character in the target text based on a cross-attention mechanism to obtain the target encoding features of the soft connection. The process of classifying the encoding features of each soft connection can be implemented using a classification model, such as... Figure 6 The diagram shown is a structural schematic of a classification model provided in an embodiment of this application, which may include:

[0116] Encoding module 601, connection module 602, attention module 603 and classification module 604;

[0117] The encoding module 601 is used to encode each character in the target text to obtain the encoding features of each character.

[0118] The connection module 602 is used to: for each soft connection corresponding to the target polyphonic character, fuse the encoding features of each character contained in the soft connection to obtain the initial encoding features of the soft connection.

[0119] Attention module 603 is used to: for each soft connection, fuse the initial encoding features of the soft connection with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft connection.

[0120] The classification module 604 is used to classify the target polyphonic characters in each soft connection based on the target encoding features of each soft connection, and to obtain the probability distribution of the target polyphonic characters corresponding to the soft connection.

[0121] Optionally, the classification module 604 may employ a deep neural network. As an example, the classification module 604 includes multiple fully connected layers.

[0122] Optionally, the classification model can be trained in the following ways:

[0123] The encoding module 601 encodes each character in the text sample to obtain the encoding features of each character.

[0124] For each polyphonic character in the text sample, the initial encoding features of multiple soft links corresponding to the polyphonic character are obtained through the connection module 602; wherein, each soft link is composed of the polyphonic character or composed of at least two consecutive characters in the text sample containing the polyphonic character, and the maximum number of characters contained in each soft link is a preset number; the initial encoding features of each soft link are obtained by fusing the encoding features of each character contained in the soft link;

[0125] For each soft connection, the attention module 603 fuses the initial encoding features of the soft connection with the encoding features of each character in the text sample based on the cross-attention mechanism to obtain the target encoding features of the soft connection.

[0126] The classification module 604 classifies the polyphonic character in each soft link based on the target encoding features of each soft link, and obtains the probability distribution of the polyphonic character corresponding to the soft link.

[0127] The classification model's parameters are updated with the goal of ensuring that the pronunciation of each polyphonic character corresponding to a soft link in the text sample, represented by the probability distribution, approximates the pronunciation label of that polyphonic character. Additionally, the model aims to ensure that the pronunciation of the same polyphonic character, determined by voting based on the probability distributions of each soft link corresponding to the same polyphonic character in the text sample, approximates the pronunciation label of the same polyphonic character in the text sample. In other words, the pronunciation of each polyphonic character corresponding to each soft link in the text sample can be determined based on the probability distribution of each polyphonic character (referred to as the first pronunciation for ease of description and distinction). For each polyphonic character in the text sample, the pronunciation of each polyphonic character in the text sample is determined by voting based on the multiple probability distributions of that polyphonic character determined by its corresponding soft links (referred to as the second pronunciation for ease of description and distinction). The parameters of the classification model are updated with the goal that both the first and second pronunciations of each polyphonic character approximate the pronunciation label of that polyphonic character.

[0128] As can be seen from the foregoing, each polyphonic character corresponds to M soft connections, which in turn yields M probability distributions for that polyphonic character. This application aims to update the parameters of the classification model by ensuring that the pronunciation represented by each probability distribution is close to the pronunciation label of the polyphonic character, and that the pronunciation of the polyphonic character determined by voting based on the M probability distributions is close to the pronunciation label of the polyphonic character.

[0129] As an example, for the q-th polyphonic character in the text sample (q=1, 2, 3, ..., Q; Q is the number of polyphonic characters in the text sample), the cross-entropy loss between the i-th probability distribution of the q-th polyphonic character and its pronunciation label can be calculated. The average of the M cross-entropy losses is used to obtain the first loss of the q-th polyphonic character. Based on the M probability distributions of the q-th polyphonic character, soft voting is performed (e.g., the average of the M probability distributions) to obtain the target probability distribution of the q-th polyphonic character. The cross-entropy loss between the target probability distribution of the q-th polyphonic character and its pronunciation label is calculated as the second loss. The first and second losses are weighted and summed to obtain the comprehensive loss corresponding to the q-th polyphonic character. The parameters of the classification model are updated with the goal of reducing the comprehensive loss (i.e., the comprehensive loss calculated after updating the parameters of the classification model is less than the comprehensive loss calculated before updating the parameters of the classification model).

[0130] In the example above, for each polyphonic character, two classification losses based on voting mechanisms were used as the overall loss for the polyphonic character. The first loss was based on a hard voting mechanism, and the second loss was based on a soft voting mechanism. Based on this, the robustness and classification accuracy of the classification model were improved.

[0131] The weights of the first loss and the second loss can be the same or different. The sum of the weights of the first loss and the second loss is 1.

[0132] In the case where the classification module 604 includes multiple fully connected layers, some of these layers are dropout layers during training. This means that during training, the outputs of some neurons in the dropout layers are randomly set to zero, reducing dependencies between neurons. During training, each neuron in the dropout layer has a certain probability of being "dropped" (set to zero), meaning it doesn't participate in forward or backward propagation in the current training batch. This prevents the classification model from over-relying on certain specific neurons, improving its generalization ability.

[0133] It is important to note that dropout layers typically only "discard" the outputs of some neurons during training. After training is complete, the outputs of all neurons will be used in the prediction phase.

[0134] Corresponding to the method embodiments, this application also provides a polyphonic character disambiguation device. A schematic diagram of one structure of the polyphonic character disambiguation device provided in this application is shown below. Figure 7 As shown, it may include:

[0135] The soft connection unit 701, the classification unit 702, and the voting unit 703;

[0136] The soft link unit 701 is used to encode each soft link based on the association relationship between the multiple soft links corresponding to the target polyphonic characters in the target text, thereby obtaining the target encoding features of each soft link; each soft link is composed of the target polyphonic character or is composed of at least two consecutive characters in the target text that contain the target polyphonic character, and the maximum number of characters contained in each soft link is a preset number;

[0137] The classification unit 702 is used to classify the target polyphonic characters in each soft link based on the target encoding features of each soft link, and obtain the probability distribution of the target polyphonic characters corresponding to each soft link; the probability distribution is the probability of the target polyphonic character belonging to each of several pronunciations, representing the pronunciation of the target polyphonic character in the soft link; the several pronunciations are all pronunciations of all polyphonic characters in a preset set of polyphonic characters;

[0138] The voting unit 703 is used to vote based on the probability distribution of the target polyphonic characters corresponding to each soft link, so as to determine the pronunciation of the target polyphonic characters in the target text.

[0139] The polyphonic character disambiguation device provided in this application proposes the concept of soft links. Without relying on word segmentation tools, it extracts polyphonic characters and their surrounding context as soft links. Based on the association between each soft link corresponding to the target polyphonic character and the target text, it determines the target encoding features of each soft link corresponding to the target polyphonic character. Based on the target encoding features of each soft link, it classifies the target polyphonic characters in each soft link, obtaining multiple possible classification results for the target polyphonic characters. Based on the classification results, it votes to determine the pronunciation of the target polyphonic character in the target text. This process does not require word segmentation of the target text, thus avoiding polyphonic character disambiguation errors due to word segmentation errors. It also does not require the use of explanatory information from word segmentation containing polyphonic characters, thus avoiding polyphonic character disambiguation errors due to explanatory information errors. Therefore, it avoids dependence on word segmentation and explanatory information for polyphonic character disambiguation, improving the accuracy of polyphonic character disambiguation.

[0140] Furthermore, in polyphonic character disambiguation schemes based on polyphonic word dictionaries, the efficiency of word lookup decreases as the number of polyphonic words in the dictionary increases, ultimately directly affecting the efficiency of the polyphonic character disambiguation task. This application, however, does not require a polyphonic word dictionary; therefore, it can improve the accuracy of polyphonic character disambiguation while ensuring the efficiency of the task.

[0141] In an optional embodiment, when the soft link unit 701 encodes each soft link based on the association relationship between the multiple soft links corresponding to the target polyphonic character and the target text, it is used for:

[0142] Each character in the target text is encoded to obtain the encoding features of each character;

[0143] For each soft link corresponding to the target polyphonic character, the encoding features of each character contained in the soft link are fused to obtain the initial encoding features of the soft link;

[0144] The initial encoding features of the soft link are fused with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft link.

[0145] In an optional embodiment, when the soft link unit 701 fuses the encoding features of each word contained in the soft link to obtain the initial encoding features of the soft link, it is used for:

[0146] The initial encoding features of the soft link are obtained by weighted summation of the encoding features of each word contained in the soft link.

[0147] In an optional embodiment, when the soft connection unit 701 fuses the initial encoding features of the soft connection with the encoding features of each character in the target text based on a cross-attention mechanism to obtain the target encoding features of the soft connection, it is used for:

[0148] Using the initial encoding features of the soft link and the encoding features of each character in the target text, calculate the attention weight of the soft link to each character in the target text;

[0149] Based on the attention weights of each character in the target text applied by the soft link, the encoding features of each character in the target text are weighted and summed to obtain the target encoding features of the soft link.

[0150] In an optional embodiment, when the voting unit 703 votes based on the probability distribution of the target polyphonic characters corresponding to each soft link, it is used to:

[0151] By fusing the probabilities of the target polyphonic characters belonging to the same pronunciation in the probability distribution of each soft link, the target probability of the target polyphonic character belonging to each pronunciation is obtained.

[0152] The pronunciation corresponding to the maximum target probability is determined as the pronunciation of the target polyphonic character in the target text.

[0153] In an optional embodiment, when the voting unit 703 votes based on the probability distribution of the target polyphonic characters corresponding to each soft link, it is used to:

[0154] Based on the probability distribution of the target polyphonic character corresponding to each soft link, the pronunciation of the target polyphonic character in each soft link is determined;

[0155] The pronunciations of the target polyphonic character in each of the soft links are statistically analyzed, and the pronunciation with the most occurrences is determined as the pronunciation of the target polyphonic character in the target text.

[0156] In an optional embodiment, the voting unit 703 is further configured to:

[0157] If the number of multiple pronunciations of the target polyphonic character is the same, the pronunciation with the highest probability is determined as the pronunciation of the target polyphonic character in the target text;

[0158] Alternatively, if the number of multiple pronunciations of the target polyphonic character is the same, the probabilities of the target polyphonic character belonging to the same pronunciation in the probability distribution of the target polyphonic character corresponding to each soft link are merged to obtain the target probability of the target polyphonic character belonging to each pronunciation; the pronunciation corresponding to the maximum target probability is determined as the pronunciation of the target polyphonic character in the target text.

[0159] In an optional embodiment, the soft connection unit 701 encodes each character in the target text to obtain the encoding features of each character. For each soft connection corresponding to the target polyphonic character, the initial encoding features of the soft connection are fused with the encoding features of each character in the target text based on a cross-attention mechanism to obtain the target encoding features of the soft connection. The process of the classification unit 702 classifying the target encoding features of each soft connection includes:

[0160] The soft connection unit 701 encodes each character in the target text through the encoding module of the classification model to obtain the encoding features of each character;

[0161] The soft connection unit 701 uses the connection module of the classification model to fuse the encoding features of each character contained in the soft connection corresponding to the target polyphonic character to obtain the initial encoding features of the soft connection.

[0162] The soft connection unit 701, through the attention module of the classification model, fuses the initial encoding features of each soft connection with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft connection;

[0163] The classification unit 702 classifies the target polyphonic characters in each soft link based on the target encoding features of each soft link through the classification module of the classification model, and obtains the probability distribution of the target polyphonic characters corresponding to each soft link.

[0164] In an optional embodiment, the polyphonic character disambiguation device further includes a training module for training a classification model. When training the classification model, the training module is used to:

[0165] The encoding module encodes each character in the text sample to obtain the encoding features of each character;

[0166] For each polyphonic character in the text sample, the connection module obtains the initial encoding features of multiple soft links corresponding to that polyphonic character; wherein, each soft link is composed of the polyphonic character or is composed of at least two consecutive characters in the text sample containing the polyphonic character, and the maximum number of characters contained in each soft link is a preset number; the initial encoding features of each soft link are obtained by fusing the encoding features of each character contained in the soft link;

[0167] For each soft connection, the attention module fuses the initial encoding features of the soft connection with the encoding features of each character in the text sample based on the cross-attention mechanism to obtain the target encoding features of the soft connection.

[0168] The classification module classifies the polyphonic characters in each soft connection based on the target encoding features of each soft connection, thereby obtaining the probability distribution of the polyphonic character corresponding to the soft connection.

[0169] The first pronunciation of each polyphonic character corresponding to each soft link in the text sample is determined based on the probability distribution of each polyphonic character.

[0170] For each polyphonic character in the text sample, the second pronunciation of each polyphonic character in the text sample is determined by voting based on the multiple probability distributions of the polyphonic character determined by each soft connection corresponding to the polyphonic character.

[0171] The parameters of the classification model are updated with the goal of ensuring that the first and second pronunciations of each polyphonic character are close to the pronunciation label of that polyphonic character.

[0172] This application also provides an electronic device in its embodiments. (See reference...) Figure 8 As shown, it illustrates a structural schematic diagram of an electronic device suitable for implementing the embodiments of this application. The electronic device in the embodiments of this application can be a terminal device (e.g., a vehicle-mounted system, a large-screen device, a smart home device, a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) or a server (which can be a single server, a server cluster, or a cloud server, etc.). Figure 8 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0173] like Figure 8As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. When the electronic device is powered on, the RAM 803 also stores various programs and data required for the operation of the electronic device. The processing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0174] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, memory cards, hard drives, etc.; and communication devices 809. Communication device 809 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0175] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the polyphonic character disambiguation methods provided in this application.

[0176] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the polyphonic character disambiguation methods provided in this application.

[0177] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0179] In the above embodiments, the functionality can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially as a computer program product. Those skilled in the art can use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of this application.

[0180] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0181] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0182] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for disambiguating polyphonic characters, characterized in that, include: For a target polyphonic character in the target text, based on the association relationship between multiple soft links corresponding to the target polyphonic character and the target text, each soft link is encoded to obtain the target encoding feature of each soft link; each soft link is composed of the target polyphonic character or composed of at least two consecutive characters in the target text containing the target polyphonic character, and the maximum number of characters contained in each soft link is a preset number N; The number of soft links corresponding to the target polyphonic character is N(N+1) / 2; Based on the target encoding features of each soft link, the target polyphonic characters in each soft link are classified to obtain the probability distribution of the target polyphonic characters corresponding to each soft link; The probability distribution represents the probability of the target polyphonic character belonging to each of several pronunciations, and characterizes the pronunciation of the target polyphonic character in the soft link; the several pronunciations are all pronunciations of all polyphonic characters in a preset set of polyphonic characters; Voting is performed based on the probability distribution of the target polyphonic characters corresponding to each soft link to determine the pronunciation of the target polyphonic characters in the target text.

2. The method according to claim 1, characterized in that, The encoding of each soft link based on the association relationship between the target polyphonic character and the target text includes: Each character in the target text is encoded to obtain the encoding features of each character; For each soft link corresponding to the target polyphonic character, the encoding features of each character contained in the soft link are fused to obtain the initial encoding features of the soft link; The initial encoding features of the soft link are fused with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft link.

3. The method according to claim 2, characterized in that, The initial encoding features of the soft link are fused with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft link, including: Using the initial encoding features of the soft link and the encoding features of each character in the target text, calculate the attention weight of the soft link to each character in the target text; Based on the attention weights of each character in the target text applied by the soft link, the encoding features of each character in the target text are weighted and summed to obtain the target encoding features of the soft link.

4. The method according to claim 1, characterized in that, The voting based on the probability distribution of the target polyphonic characters corresponding to each of the soft links includes: By fusing the probabilities of the target polyphonic characters belonging to the same pronunciation in the probability distribution of each soft link, the target probability of the target polyphonic character belonging to each pronunciation is obtained. The pronunciation corresponding to the maximum target probability is determined as the pronunciation of the target polyphonic character in the target text.

5. The method according to claim 1, characterized in that, The voting based on the probability distribution of the target polyphonic characters corresponding to each of the soft links includes: Based on the probability distribution of the target polyphonic character corresponding to each soft link, the pronunciation of the target polyphonic character in each soft link is determined; The pronunciations of the target polyphonic character in each of the soft links are statistically analyzed, and the pronunciation with the most occurrences is determined as the pronunciation of the target polyphonic character in the target text.

6. The method according to claim 5, characterized in that, Also includes: If the number of multiple pronunciations of the target polyphonic character is the same, the pronunciation with the highest probability is determined as the pronunciation of the target polyphonic character in the target text; Alternatively, if the number of multiple pronunciations of the target polyphonic character is the same, the probabilities of the target polyphonic character belonging to the same pronunciation in the probability distribution of the target polyphonic character corresponding to each soft link are merged to obtain the target probability of the target polyphonic character belonging to each pronunciation; the pronunciation corresponding to the maximum target probability is determined as the pronunciation of the target polyphonic character in the target text.

7. The method according to claim 2, characterized in that, Encoding each character in the target text yields its encoding features. For each soft link corresponding to the target polyphonic character, the initial encoding features of the soft link are fused with the encoding features of each character in the target text based on a cross-attention mechanism to obtain the target encoding features of the soft link. The process of classifying the target encoding features of each soft link includes: The encoding module of the classification model encodes each character in the target text to obtain the encoding features of each character; For each soft connection corresponding to the target polyphonic character, the connection module of the classification model fuses the encoding features of each character contained in the soft connection to obtain the initial encoding features of the soft connection. For each soft connection, the attention module of the classification model fuses the initial encoding features of the soft connection with the encoding features of each character in the target text based on the cross-attention mechanism to obtain the target encoding features of the soft connection. The classification module of the classification model classifies the target polyphonic characters in each soft link based on the target encoding features of each soft link, thereby obtaining the probability distribution of the target polyphonic characters corresponding to each soft link.

8. The method according to claim 7, characterized in that, The classification model is trained in the following manner: The encoding module encodes each character in the text sample to obtain the encoding features of each character; For each polyphonic character in the text sample, the connection module obtains the initial encoding features of multiple soft links corresponding to that polyphonic character; wherein each soft link is composed of the polyphonic character or composed of at least two consecutive characters in the text sample containing the polyphonic character, and the maximum number of characters contained in each soft link is a preset number; the initial encoding features of each soft link are obtained by fusing the encoding features of each character contained in the soft link; For each soft connection, the attention module fuses the initial encoding features of the soft connection with the encoding features of each character in the text sample based on the cross-attention mechanism to obtain the target encoding features of the soft connection. The classification module classifies the polyphonic character in each soft link based on the target encoding features of each soft link, thereby obtaining the probability distribution of the polyphonic character corresponding to the soft link. The first pronunciation of each polyphonic character corresponding to each soft link in the text sample is determined based on the probability distribution of each polyphonic character. For each polyphonic character in the text sample, the second pronunciation of each polyphonic character in the text sample is determined by voting based on the multiple probability distributions of the polyphonic character determined by each soft connection corresponding to the polyphonic character. The parameters of the classification model are updated with the goal of ensuring that the first and second pronunciations of each polyphonic character are close to the pronunciation label of that polyphonic character.

9. A polyphonic character disambiguation device, characterized in that, include: A soft link unit is used to encode each soft link based on the association relationship between multiple soft links corresponding to the target polyphonic characters in the target text, thereby obtaining the target encoding features of each soft link; each soft link consists of the target polyphonic character or consists of at least two consecutive characters in the target text containing the target polyphonic character, and the maximum number of characters contained in each soft link is a preset number N; the number of soft links corresponding to the target polyphonic character is N(N+1) / 2; A classification unit is used to classify the target polyphonic characters in each soft connection based on the target encoding features of each soft connection, so as to obtain the probability distribution of the target polyphonic characters corresponding to each soft connection; The probability distribution represents the probability of the target polyphonic character belonging to each of several pronunciations, and characterizes the pronunciation of the target polyphonic character in the soft link; the several pronunciations are all pronunciations of all polyphonic characters in a preset set of polyphonic characters; A voting unit is used to vote based on the probability distribution of the target polyphonic characters corresponding to each soft link, so as to determine the pronunciation of the target polyphonic characters in the target text.

10. A computer program product, characterized in that, Includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the polyphonic character disambiguation method as described in any one of claims 1 to 8.

11. An electronic device, characterized in that, The electronic device includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the polyphonic character disambiguation method as described in any one of claims 1 to 8.

12. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the polyphonic character disambiguation method as described in any one of claims 1 to 8.

Images