Multi-syllable word prediction method, device and equipment and readable storage medium

Abstract

The application provides a multi-sound character prediction method, device and equipment and a readable storage medium. The method uses a pronunciation prediction network that learns semantic information and pronunciation features of multi-sound characters under different segmentation modes, extracts context embedding of a multi-sound character to be tested for a segmentation sequence of a processing text containing the multi-sound character to be tested, uses a linear projection matrix trained in the pronunciation prediction network, fuses a target segmentation and its part-of-speech information to determine a conditional weight matrix, dynamically adjusts the prediction probability of different pronunciation labels in the network output process by using the conditional weight matrix, so that the pronunciation prediction network can better understand the context of the multi-sound character, and the accuracy and robustness of multi-sound character prediction are improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing, and in particular to a method, apparatus, device, and readable storage medium for predicting polyphonic characters. Background Technology

[0002] In the field of natural language processing, in the process of accurately converting Chinese text into its corresponding pronunciation representation, namely the Mandarin text-to-speech technology, there are many instances of multiple pronunciations for a single Chinese character, and their pronunciation is often affected by the context. Therefore, it is necessary to accurately determine the correct pronunciation of each Chinese character in a specific context.

[0003] Traditional methods for handling polyphonic characters mainly rely on rule-based approaches. These methods construct detailed dictionaries and pre-set complex rule sets to segment text into word fragments and eliminate pronunciation ambiguity based on pronunciation entries in the dictionary and the rule set.

[0004] However, this method is highly dependent on the completeness of the dictionary and the accuracy of the rules. Since dictionary updates often lag behind language development and the rules are difficult to cover all complex language phenomena, and the rules are static and fixed, they cannot be flexibly adjusted according to the dynamic semantic environment of the text. When processing texts with complex semantic relationships, prediction errors are prone to occur, and there are problems of low accuracy and poor flexibility in handling polyphonic characters. Summary of the Invention

[0005] In view of this, in order to solve the above-mentioned technical problems, this application provides a method, apparatus, device and readable storage medium for predicting polyphonic characters.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] According to a first aspect of the embodiments of this application, a method for predicting polyphonic characters is provided, applied to a pre-trained pronunciation prediction network, wherein the pronunciation prediction network is provided with a pronunciation label set corresponding to all pronunciations of each polyphonic character; the method includes:

[0008] For the processed text containing polyphonic characters to be tested, obtain the initial word segmentation sequence obtained by segmenting words into units;

[0009] Based on the initial word segmentation sequence, obtain the context embedding of the polyphonic character to be tested;

[0010] Using a linear projection matrix, a feature linear projection is performed on the target word containing the polyphonic character to be tested in the initial word segmentation sequence, the part-of-speech tag of the target word, and at least one of the cross features of the target word and the part-of-speech tag, to generate a conditional weight matrix; the length of the conditional weight matrix is ​​the same as the length of the pronunciation tag set.

[0011] By applying the output activation function of the conditional weight matrix, a predicted probability distribution for the pronunciation tag set is generated based on the context embedding, thereby determining the pronunciation of the polyphonic character to be tested.

[0012] The pronunciation prediction network is trained by acquiring the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods and minimizing the weighted result of the loss function corresponding to each predicted pronunciation.

[0013] Optionally, obtaining the initial word segmentation sequence obtained by segmenting words into units includes:

[0014] The processed text is segmented using a word-level word segmentation tool to obtain the first segmentation sequence;

[0015] For each first word in the first word segmentation sequence, if the first word fails to match the preset vocabulary, the first word is updated to a second word; the second word is obtained by segmenting the first word using the word segmentation function set in the pronunciation prediction network.

[0016] The first segmented word sequence after the update is completed is determined as the initial segmented word sequence.

[0017] Optionally, obtaining the context embedding of the polyphonic character to be tested based on the initial word segmentation sequence includes:

[0018] Each word in the initial word segmentation sequence is encoded to obtain an initial embedding vector for each word, including a word embedding vector and a position embedding vector; the word embedding vector is used to represent the semantic information of the word itself; the position embedding vector is used to represent the position information of the word itself.

[0019] Based on the initial embedding vector of each word segment, the context information is fused to the initial embedding vector of the target word segment through an attention mechanism to generate the context embedding of the polyphonic character to be tested.

[0020] Optionally, the generation of the conditional weight matrix includes:

[0021] Obtain the character encoding vector of the target word segment and the part-of-speech encoding vector of the part-of-speech tag;

[0022] Extract the cross-feature vector of the character encoding vector and the part-of-speech encoding vector, and use the first projection matrix in the linear projection matrix to obtain the first projection result of the cross-feature;

[0023] The second projection result of the character encoding vector is obtained by using the second projection matrix in the linear projection matrix;

[0024] The third projection result of the part-of-speech encoding vector is obtained by using the second projection matrix in the linear projection matrix;

[0025] The conditional weight matrix is ​​generated based on the first projection result, the second projection result, and the third projection result.

[0026] Optionally, the extraction of the cross features between the character encoding vector and the part-of-speech encoding vector includes at least one of the following:

[0027] Obtain the tensor product of the character encoding vector and the part-of-speech encoding vector as the cross feature;

[0028] Alternatively, the element-wise sum of the character encoding vector and the part-of-speech encoding vector can be obtained as the cross-feature.

[0029] Optionally, generating the conditional weight matrix based on the first projection result, the second projection result, and the third projection result includes:

[0030] Based on the three set hyperparameters, the first projection result, the second projection result, and the third projection result are weighted respectively to obtain the corresponding three weighted projection results;

[0031] The three weighted projection results are summed element by element to generate an initial weight matrix;

[0032] Each element in the initial weight matrix is ​​converted into a probability value to obtain the conditional weight matrix; the probability values ​​range from 0 to 1.

[0033] Optionally, after generating the conditional weight matrix, the method further includes:

[0034] Determine candidate pronunciation tags for the polyphonic character to be tested; the candidate pronunciation tags represent all pronunciations of the polyphonic character to be tested;

[0035] For the set of pronunciation tags, set a mask 0 for all pronunciation tags except the candidate pronunciation tags, and set a mask 1 for the candidate pronunciation tags to obtain a mask matrix;

[0036] The mask matrix is ​​multiplied element by element by the conditional weight matrix to obtain the updated conditional weight matrix.

[0037] Optionally, the pronunciation prediction network includes a part-of-speech prediction subnetwork and is equipped with a set of part-of-speech tags; the method further includes:

[0038] The context embedding of the polyphonic character to be tested is mapped into a part-of-speech score vector through the first mapping parameter in the part-of-speech prediction subnetwork; the part-of-speech score vector includes the unnormalized score of each part-of-speech tag in the part-of-speech tag set;

[0039] By using an activation function for part-of-speech classification, a predicted probability distribution for the set of part-of-speech tags is generated based on the part-of-speech score vector, and the predicted part-of-speech of the target word is determined as the part-of-speech of the target word.

[0040] Optionally, generating a predicted probability distribution for the phonetic tag set based on the context embedding by applying the output activation function of the conditional weight matrix includes:

[0041] Based on the second mapping parameters learned during the training phase, the context embedding is mapped into a pronunciation score vector; the pronunciation score vector includes the unnormalized score of each pronunciation tag in the pronunciation tag set;

[0042] For each pronunciation tag, obtain the conditional weight in the conditional weight matrix that is in the same position as the unnormalized score of the pronunciation tag;

[0043] Based on the unnormalized score of the pronunciation tag and the conditional weight, the output activation function is used to perform normalization processing to obtain the probability value of the pronunciation tag.

[0044] Optionally, the normalization process based on the unnormalized score of the pronunciation tag and the conditional weights, performed through the output activation function, includes:

[0045] Determine the exponential result of the unnormalized score of the pronunciation tag;

[0046] The product of the exponential calculation result and the conditional weight of the pronunciation tag is obtained as the target calculation result;

[0047] The target operation result of each pronunciation tag is normalized based on the sum of the target operation results of all pronunciation tags.

[0048] Optionally, the method further includes a training step for a pronunciation prediction network:

[0049] For each sample sentence, a first segmentation sequence obtained by segmenting individual characters is obtained, and a second segmentation sequence obtained by segmenting words is obtained; the sample sentence includes a polyphonic character and a pronunciation tag for the polyphonic character; the second segmentation sequence also includes a part-of-speech tag for the segmentation character containing the polyphonic character;

[0050] The first word segmentation sequence and the second word segmentation sequence are input together into the pronunciation prediction network to obtain the first predicted pronunciation corresponding to the first word segmentation sequence and the second predicted pronunciation corresponding to the second word segmentation sequence; the conditional weight matrix is ​​determined based on the word segmentation of the second word segmentation sequence containing polyphonic characters and the part-of-speech tag;

[0051] Based on the first predicted pronunciation and the second predicted pronunciation, a target loss function is obtained, and the model parameters of the pronunciation prediction network are adjusted by minimizing the target loss function; the target loss includes a first loss between the first predicted pronunciation and the pronunciation marker, and a second loss between the second predicted pronunciation and the pronunciation marker.

[0052] Optionally, the pronunciation prediction network includes a part-of-speech prediction subnetwork, and the method further includes:

[0053] The context embedding of the word segment containing polyphonic characters is determined by the pronunciation prediction network.

[0054] The context embedding is input into the part-of-speech prediction subnetwork to obtain the predicted part-of-speech output by the part-of-speech prediction subnetwork;

[0055] The conditional weight matrix is ​​determined based on the word segmentation of characters containing polyphonic characters and the predicted part-of-speech tags;

[0056] The target loss function also includes a third loss based on the predicted part-of-speech tag and the part-of-speech tag.

[0057] According to a second aspect of the embodiments of this application, a polyphonic character prediction device is provided, applied to a pre-trained pronunciation prediction network. The pronunciation prediction network has a set of pronunciation labels corresponding to all pronunciations of each polyphonic character. The pronunciation prediction network is trained by acquiring the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods and minimizing the weighted result of the loss function corresponding to each predicted pronunciation. The device includes:

[0058] The word segmentation module is used to obtain the initial word segmentation sequence by word segmentation for the processed text containing polyphonic characters to be tested.

[0059] The context acquisition module is used to acquire the context embedding of the polyphonic character to be tested based on the initial word segmentation sequence;

[0060] The conditional weight matrix generation module is used to perform feature linear projection on at least one of the following in the initial word segmentation sequence: the target word containing the polyphonic character to be tested, the part-of-speech tag of the target word, and the cross-feature of the target word and the part-of-speech tag; the length of the conditional weight matrix is ​​the same as the length of the pronunciation tag set.

[0061] The prediction probability and pronunciation determination module is used to determine the pronunciation of the polyphonic character to be tested by applying the output activation function of the conditional weight matrix and generating a prediction probability distribution for the pronunciation tag set based on the context embedding.

[0062] Optionally, the word segmentation module is specifically used for:

[0063] The processed text is segmented using a word-level word segmentation tool to obtain the first segmentation sequence;

[0064] For each first word in the first word segmentation sequence, if the first word fails to match the preset vocabulary, the first word is updated to a second word; the second word is obtained by segmenting the first word using the word segmentation function set in the pronunciation prediction network.

[0065] The first segmented word sequence after the update is completed is determined as the initial segmented word sequence.

[0066] Optionally, the context acquisition module is specifically used for:

[0067] Each word in the initial word segmentation sequence is encoded to obtain an initial embedding vector for each word, including a word embedding vector and a position embedding vector; the word embedding vector is used to represent the semantic information of the word itself; the position embedding vector is used to represent the position information of the word itself.

[0068] Based on the initial embedding vector of each word segment, the context information is fused to the initial embedding vector of the target word segment through an attention mechanism to generate the context embedding of the polyphonic character to be tested.

[0069] Optionally, the conditional weight matrix generation module is specifically used for:

[0070] Obtain the character encoding vector of the target word segment and the part-of-speech encoding vector of the part-of-speech tag;

[0071] Extract the cross-feature vector of the character encoding vector and the part-of-speech encoding vector, and use the first projection matrix in the linear projection matrix to obtain the first projection result of the cross-feature;

[0072] The second projection matrix in the linear projection matrix is ​​used to obtain the second projection result of the character encoding vector; the third projection result of the part-of-speech encoding vector is also used to obtain the third projection result of the part-of-speech encoding vector.

[0073] The conditional weight matrix is ​​generated based on the first projection result, the second projection result, and the third projection result.

[0074] Optionally, the conditional weight matrix generation module, when extracting the cross features between the character encoding vector and the part-of-speech encoding vector, includes at least one of the following methods:

[0075] Obtain the tensor product of the character encoding vector and the part-of-speech encoding vector as the cross feature;

[0076] Alternatively, the element-wise sum of the character encoding vector and the part-of-speech encoding vector can be obtained as the cross-feature.

[0077] Optionally, the conditional weight matrix generation module, when generating the conditional weight matrix based on the first projection result, the second projection result, and the third projection result, includes:

[0078] Based on the three set hyperparameters, the first projection result, the second projection result, and the third projection result are weighted respectively to obtain the corresponding three weighted projection results;

[0079] The three weighted projection results are summed element by element to generate an initial weight matrix;

[0080] Each element in the initial weight matrix is ​​converted into a probability value to obtain the conditional weight matrix; the probability values ​​range from 0 to 1.

[0081] Optionally, after generating the conditional weight matrix, the apparatus further includes:

[0082] Determine candidate pronunciation tags for the polyphonic character to be tested; the candidate pronunciation tags represent all pronunciations of the polyphonic character to be tested;

[0083] For the set of pronunciation tags, set a mask 0 for all pronunciation tags except the candidate pronunciation tags, and set a mask 1 for the candidate pronunciation tags to obtain a mask matrix;

[0084] The mask matrix is ​​multiplied element by element by the conditional weight matrix to obtain the updated conditional weight matrix.

[0085] Optionally, the pronunciation prediction network includes a part-of-speech prediction subnetwork and is equipped with a set of part-of-speech tags; the device further includes:

[0086] The context embedding of the polyphonic character to be tested is mapped into a part-of-speech score vector through the first mapping parameter in the part-of-speech prediction subnetwork; the part-of-speech score vector includes the unnormalized score of each part-of-speech tag in the part-of-speech tag set;

[0087] By using an activation function for part-of-speech classification, a predicted probability distribution for the set of part-of-speech tags is generated based on the part-of-speech score vector, and the predicted part-of-speech of the target word is determined as the part-of-speech of the target word.

[0088] Optionally, the prediction probability and pronunciation determination module is specifically used for:

[0089] Based on the second mapping parameters learned during the training phase, the context embedding is mapped into a pronunciation score vector; the pronunciation score vector includes the unnormalized score of each pronunciation tag in the pronunciation tag set;

[0090] For each pronunciation tag, obtain the conditional weight in the conditional weight matrix that is in the same position as the unnormalized score of the pronunciation tag;

[0091] Based on the unnormalized score of the pronunciation tag and the conditional weight, the output activation function is used to perform normalization processing to obtain the probability value of the pronunciation tag.

[0092] Optionally, the prediction probability and pronunciation determination module, when performing normalization processing through the output activation function based on the unnormalized score of the pronunciation tag and the conditional weights, includes:

[0093] Determine the exponential result of the unnormalized score of the pronunciation tag;

[0094] The product of the exponential calculation result and the conditional weight of the pronunciation tag is obtained as the target calculation result;

[0095] The target operation result of each pronunciation tag is normalized based on the sum of the target operation results of all pronunciation tags.

[0096] Optionally, the device further includes a training step for a pronunciation prediction network:

[0097] For each sample sentence, a first segmentation sequence obtained by segmenting individual characters is obtained, and a second segmentation sequence obtained by segmenting words is obtained; the sample sentence includes a polyphonic character and a pronunciation tag for the polyphonic character; the second segmentation sequence also includes a part-of-speech tag for the segmentation character containing the polyphonic character;

[0098] The first word segmentation sequence and the second word segmentation sequence are input together into the pronunciation prediction network to obtain the first predicted pronunciation corresponding to the first word segmentation sequence and the second predicted pronunciation corresponding to the second word segmentation sequence; the conditional weight matrix is ​​determined based on the word segmentation of the second word segmentation sequence containing polyphonic characters and the part-of-speech tag;

[0099] Based on the first predicted pronunciation and the second predicted pronunciation, a target loss function is obtained, and the model parameters of the pronunciation prediction network are adjusted by minimizing the target loss function; the target loss includes a first loss between the first predicted pronunciation and the pronunciation marker, and a second loss between the second predicted pronunciation and the pronunciation marker.

[0100] Optionally, the pronunciation prediction network includes a part-of-speech prediction sub-network, and the device further includes: determining the context embedding of the word segment containing polyphonic characters through the pronunciation prediction network;

[0101] The context embedding is input into the part-of-speech prediction subnetwork to obtain the predicted part-of-speech output by the part-of-speech prediction subnetwork; the conditional weight matrix is ​​determined based on the word segmentation containing polyphonic characters and the predicted part-of-speech; the target loss function also includes a third loss of the predicted part-of-speech and the part-of-speech tag.

[0102] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a memory and a processor; the memory being used to store a computer program; the processor being used to execute the above-described polyphonic character prediction method by invoking the computer program.

[0103] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the program, when executed by a processor, implements the above-described polyphonic character prediction method.

[0104] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0105] In the technical solution provided in this application, a pronunciation prediction network that learns the semantic information and pronunciation features of polyphonic characters under different word segmentation methods is used. The context embedding of the polyphonic characters to be tested is extracted from the word segmentation sequence of the processed text containing the polyphonic characters to be tested, accurately capturing the pronunciation features of the polyphonic characters in the current context. The linear projection matrix trained in the pronunciation prediction network is used to fuse the target word segmentation and its part-of-speech information to determine the conditional weight matrix. The conditional weight matrix is ​​used to dynamically adjust the prediction probability for different pronunciation tags in the network output process, so that the pronunciation prediction network can better understand the context in which the polyphonic characters are located and make more accurate pronunciation predictions. This improves the accuracy and robustness of polyphonic character prediction, reduces user confusion and inconvenience, and enhances the user experience.

[0106] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Furthermore, no embodiment in this application needs to achieve all the effects described above. Attached Figure Description

[0107] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0108] Figure 1A It is a schematic flowchart of a method for predicting polyphonic characters shown in an exemplary embodiment of this application;

[0109] Figure 1B It is a schematic diagram of the network structure of a pronunciation prediction network shown in an exemplary embodiment of this application;

[0110] Figure 2 It is a flowchart of steps for obtaining an initial word segmentation sequence shown in an exemplary embodiment of this application;

[0111] Figure 3 It is a flowchart of steps for obtaining the context embedding of the to-be-detected polyphonic character shown in an exemplary embodiment of this application;

[0112] Figure 4A It is a flowchart of steps for generating the conditional weight matrix shown in an exemplary embodiment of this application;

[0113] Figure 4B It is a flowchart of steps for obtaining the conditional weight matrix according to three projection results shown in an exemplary embodiment of this application;

[0114] Figure 4C It is another flowchart of steps for generating the conditional weight matrix by fusing specific mask information based on the conditional weight shown in an exemplary embodiment of this application;

[0115] Figure 5 It is a schematic diagram of the training process of a pronunciation prediction network shown in an exemplary embodiment of this application;

[0116] Figure 6 It is a schematic diagram of the structure of a polyphonic character prediction device shown in an exemplary embodiment of this application;

[0117] Figure 7 It is a schematic diagram of the hardware of an electronic device shown in an exemplary embodiment of this application. Detailed implementation manners

[0118] In the field of natural language processing, in the process of accurately converting Chinese text into the corresponding pronunciation representation, that is, the Mandarin text-to-speech technology, due to the existence of a large number of characters with multiple pronunciations in Chinese characters, and the pronunciation of polyphonic characters is often affected by the context. For example, for the polyphonic character "行", the pronunciation is represented as "xing2" in the text "我骑自行车上学" (I go to school by bike) and as "hang2" in the text "去银行" (go to the bank). Therefore, it is necessary to accurately judge the correct pronunciation of each Chinese character in a specific context.

[0119] Traditional methods for handling polyphonic characters rely on rule-based approaches. These methods involve constructing a comprehensive dictionary and pre-setting a complex set of rules. The dictionary contains numerous pronunciation entries for Chinese characters, while the rule set is used to parse the text, segmenting it into word fragments, and determining the pronunciation of each Chinese character based on the pronunciation entries in the dictionary and the rule set, thereby eliminating pronunciation ambiguity in polyphonic characters.

[0120] This method relies heavily on the completeness of the dictionary and the accuracy of the rules. However, with the continuous development of language and the emergence of new vocabulary, dictionary updates are often difficult to keep pace with language development. This results in some newly emerging polyphonic characters or special pronunciations not being accurately identified. Moreover, the rule set based on the complexity of the Chinese language is difficult to cover all possible contexts and pronunciation changes. Furthermore, the rules are static and fixed, and cannot be flexibly adjusted according to the dynamic semantic environment of the text. When processing texts with complex semantic relationships, prediction errors are prone to occur, resulting in low accuracy and poor flexibility in handling polyphonic characters.

[0121] In view of this, this application proposes a method for predicting polyphonic characters using a pre-trained pronunciation prediction network. This network captures the dynamic characteristics and contextual information of polyphonic characters, thereby achieving accurate pronunciation judgment. In this embodiment, the output layer of the pronunciation prediction network includes a pronunciation tag set, comprising pronunciation tags corresponding to all pronunciations of each polyphonic character. For example, if the number of polyphonic characters in Chinese is N, and the number of pronunciations of a polyphonic character is typically between 2 and 5, then the pronunciation tag set contains the pronunciation tags corresponding to all pronunciations of the N polyphonic characters.

[0122] During the training phase, this pronunciation prediction network acquires the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods. It then minimizes the weighted sum of the loss functions corresponding to each predicted pronunciation for training. Specifically, during the training phase, the pronunciation prediction network uses multiple sample word segmentation sequences obtained by processing the same sample sentence with different word segmentation methods. It predicts the pronunciation of polyphonic characters in each sample word segmentation sequence. The network calculates the loss for the predicted pronunciations of the same polyphonic character under different sample word segmentation sequences, and then sums these losses in a weighted manner to obtain a weighted loss. By minimizing this weighted loss, the network's internal parameters are adjusted, enabling the pronunciation prediction network to better learn the contextual dependencies of polyphonic characters, as well as sentence structure and semantic information, during the training phase, thereby enhancing the accuracy and robustness of polyphonic character prediction.

[0123] Based on this pronunciation prediction network, see [link / reference] Figure 1A The flowchart shown illustrates the steps of the polyphonic character prediction method proposed in this embodiment, which includes at least the following steps:

[0124] S101, for the processed text containing polyphonic characters to be tested, obtain the initial word segmentation sequence obtained by segmenting words into units;

[0125] The polyphonic characters to be tested refer to the polyphonic characters in the text whose correct pronunciation needs to be determined. Polyphonic characters are Chinese characters with two or more pronunciations, and their pronunciations are related to the context in which the polyphonic characters are located, such as words or sentences.

[0126] In this embodiment, the processed text containing the polyphonic character to be tested is used to provide contextual information for the character, thereby aiding in its pronunciation. Therefore, the length of the processed text should be sufficient to contain the necessary contextual information for the character, so as to clearly express its linguistic context and semantic relationships. In practical applications, the length of the processed text can be the entire sentence or a part of a sentence, such as a phrase or clause containing the polyphonic character, and can be dynamically adjusted according to the actual situation. For example, if the polyphonic character to be tested is located near the beginning or end of a sentence, the entire sentence or even more contextual sentences can be used as the processed text; if the polyphonic character to be tested is located in the middle of a sentence and the surrounding words clearly indicate its context, the clause or phrase containing the polyphonic character can be used as the processed text.

[0127] Word segmentation refers to the process of dividing continuous text into meaningful, independent lexical units. Depending on the granularity of segmentation, it can be divided into word-level segmentation and single-character-level segmentation. Word-level segmentation divides text according to lexical boundaries, with each segmented unit being a complete word. This segmentation method conforms to human reading and comprehension habits and retains more semantic information. Single-character-level segmentation divides text by character, with each segmented unit being a single character. For example, the processed text "He is a bank accountant." corresponds to word-level segmentation "He / is / bank / accountant" and single-character-level segmentation "He / is / bank / accountant / accountant". In this embodiment, word-level segmentation is performed on the processed text to obtain the corresponding segmented sequence as the initial segmentation sequence.

[0128] S102, Based on the initial word segmentation sequence, obtain the context embedding of the polyphonic character to be tested;

[0129] Contextual embedding is a vector representation of the meaning of words or characters within a specific textual context. It transforms words or characters and their contextual information into fixed-length, high-dimensional vector representations that capture the semantic information and contextual relationships of words or characters within a given text. This contextual embedding varies depending on the context in which the word or character is embedded; the same word may have different embedding representations in different contexts.

[0130] For example, for sentence 1 "He goes to the bank every day." and sentence 2 "He goes for a walk every day." which contain the polyphonic character "行", the context embedding of "行" in sentence 1 tends to mean "noun, financial institution", while the context embedding of "行" in sentence 2 tends to mean "verb, walk".

[0131] In this embodiment, based on the processed text for providing context information for the polyphonic character to be measured, for the polyphonic character to be measured, its context embedding will be constructed according to the initial word segmentation sequence corresponding to the processed text and the sequence position of the polyphonic character to be measured in the initial word segmentation sequence. Regarding the context embedding, the pre-trained word embedding sub-network or the custom embedding layer in the pronunciation prediction network can be used to extract the semantic information and position information of the target word segment containing the polyphonic character to be measured in the initial word segmentation sequence, and combine the self-attention mechanism to fuse the context relationship information between other word segments and the target word segment in the initial word segmentation sequence to generate the context embedding of the target word segment as the context embedding of the polyphonic character to be measured.

[0132] Among them, the position information can include absolute position information and relative position information. The absolute position information provides the exact position of the polyphonic character to be measured in the initial word segmentation sequence to better understand the global structure and long-distance dependence relationship of the sentence, and the relative position information provides the relative distance and order between word segments in the initial word segmentation sequence to capture the local dependence relationship in the sentence. The two position information complement each other to better understand and capture the context information of the polyphonic character to be measured.

[0133] S103, use a linear projection matrix to perform feature linear projection on at least one of the target word segment containing the polyphonic character to be measured, the词性 of the target word segment, and the cross feature of the target word segment and the词性 in the initial word segmentation sequence to generate a conditional weight matrix; the length of the conditional weight matrix is the same as the length of the pronunciation label set;

[0134] The linear projection matrix belongs to the learnable parameters in the pronunciation prediction network and is obtained through the training stage. Based on the pronunciation prediction network being trained by obtaining the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods and minimizing the weighted results of the loss functions corresponding to each predicted pronunciation, therefore, the linear projection matrix is used as a shared parameter for joint training of sample word segmentation sequences under different word segmentation granularities in the training stage, and is obtained through joint training of sample word segmentation sequences of sample sentences under different word segmentation granularities. The linear projection matrix learned in the training stage fuses and learns the influence of different word segmentation granularities on the pronunciation prediction of polyphonic characters, and captures the contribution of the词性 and the characters themselves to the semantics in the context of polyphonic characters.

[0135] The target segmentation refers to the segmentation unit that contains the polyphonic character to be tested among all the segmentation units in the initial segmentation sequence corresponding to the processed text. The part of speech of the target segmentation is used to describe the grammatical function and category of the target segmentation in the processed text. For example, the part of speech may include, but is not limited to, nouns, verbs, adjectives, pronouns, adverbs, prepositions, onomatopoeia, etc.

[0136] In this embodiment, the linear projection matrix is ​​used to project and transform the input features. By performing feature mapping and linear combination on the target word containing the polyphonic character to be tested in the initial word segmentation sequence and the part-of-speech tagging of the target word, a new and more representative comprehensive feature vector is generated to better capture the semantic feature information of the polyphonic character.

[0137] A linear projection matrix can be used to project the original input features, such as the character encoding vector corresponding to the target word segment, the part-of-speech encoding vector corresponding to the part-of-speech tag, or the cross-features of the target word segment and the part-of-speech tag, onto a new feature space. Projection can be achieved through matrix multiplication, where multiplying the original input features by the linear projection matrix yields the projected new feature vector. To generate the conditional weight matrix, the pronunciation prediction network further interacts with the projected new feature vector, converting it into a probability matrix as the conditional weight matrix. The length of this conditional weight matrix is ​​the same as the number of pronunciation tags in the pronunciation tag set set in the pronunciation prediction network, and the elements in the conditional weight matrix correspond one-to-one with the pronunciation tags in the pronunciation tag set.

[0138] The conditional weight matrix is ​​used to adjust the activation function output of the pronunciation prediction model. It assigns different weights to each pronunciation tag in the pronunciation tag set set in the pronunciation prediction model. The weights reflect the probability that the pronunciation tag is predicted to be the pronunciation of the pronunciation of the pronunciation of the target character containing the polyphonic character to be tested and its part-of-speech information. It integrates the contribution of the semantic feature information extracted from the part-of-speech information in the context and the character itself to the pronunciation prediction. Each element in the conditional weight matrix is ​​used to represent the weight or probability of predicting the pronunciation of the polyphonic character to be tested as belonging to the corresponding pronunciation tag based on the semantic feature information extracted from the part-of-speech information in the context and the character itself.

[0139] S104, by applying the output activation function of the conditional weight matrix, a predicted probability distribution for the pronunciation tag set is generated based on the context embedding, and the pronunciation of the polyphonic character to be tested is determined;

[0140] The output activation function is a function used in pronunciation prediction networks to transform linear outputs into non-linear outputs. Applied to the output layer of the pronunciation prediction network, it ensures that the output value is within a suitable range. For classification tasks, the softmax function is typically used to transform the output into a probability distribution corresponding to the class. This output activation function takes the logit value (also called the unnormalized score or linear prediction) generated by the network for each pronunciation tag as input and outputs the probability that the polyphonic character to be tested belongs to that pronunciation tag. The logit value represents the network's original prediction output for each class (in this case, the pronunciation tag). It is the prediction value for each class directly obtained by the network after processing the context embedding of the polyphonic character to be tested. It is usually calculated through a linear layer of the network, such as a fully connected layer, and is therefore also called a linear prediction value. It reflects the linear partitioning of classes by the network in the input feature space. For example, if the network needs to predict one of n pronunciation tags, the output layer of the network has n neurons, each neuron outputting the logit value of a pronunciation tag.

[0141] In this embodiment, after generating the conditional weight matrix using the linear projection matrix, this conditional weight matrix participates in the calculation of the output activation function. The elements in the conditional weight matrix corresponding to the pronunciation tags are used to adjust the predicted probability generated by the output activation function for each pronunciation tag. For example, for the i-th pronunciation tag, during the process of generating the predicted probability Pi corresponding to the i-th pronunciation tag using the output activation function, the calculation of the output activation function is based on the logit value of the network output corresponding to the i-th pronunciation tag and the weight elements in the conditional weight matrix corresponding to the i-th pronunciation tag. These weight elements are used to adjust the probability value determined based on the logit value.

[0142] In pronunciation prediction networks, after obtaining the context embedding of the polyphonic character to be tested, further encoding processing can be performed on the context embedding to extract and integrate its feature information. This encoding processing can be implemented through simple fully connected layers or more complex structures such as recurrent neural networks, long short-term memory networks, gated recurrent units, or Transformer encoders to obtain encoded features. Based on these encoded features, a logit value is generated for each pronunciation tag. The process of generating the logit value involves mapping the encoded features to a space with the same dimension as the number of pronunciation tags in the pronunciation tag set.

[0143] Based on this, for the step of generating a predicted probability distribution for the set of pronunciation tags according to the context embedding, the context embedding can be encoded and mapped to the logit value of each pronunciation tag, and the logit value can be used as the input of the output activation function to obtain the probability of each pronunciation tag output by the output activation function of the conditional weight matrix, thus constituting the predicted probability distribution for the set of pronunciation tags, and then the pronunciation of the polyphonic character to be tested can be determined according to the predicted probability distribution.

[0144] The pronunciation of a polyphonic character to be tested can be determined in either of the following ways: First, the pronunciation label with the highest probability value from the predicted probability distribution can be selected as the target pronunciation label, and the pronunciation corresponding to this target pronunciation label can be determined as the pronunciation of the polyphonic character to be tested. Second, to improve the accuracy and reliability of the prediction and avoid misjudgments due to network uncertainty or noise, the pronunciation label with the highest probability value from the predicted probability distribution, and whose probability value is greater than a set probability threshold, can be selected as the target pronunciation label, and the pronunciation corresponding to this target pronunciation label can be determined as the pronunciation of the polyphonic character to be tested. This set probability threshold is a confidence level cutoff line preset based on task requirements and model performance. It is used to filter out pronunciation labels with sufficiently high confidence. When the highest probability value exceeds this set probability threshold, the network's prediction is considered reliable, and the pronunciation corresponding to this target pronunciation label is determined as the pronunciation of the polyphonic character to be tested.

[0145] In this embodiment, a pronunciation prediction network that learns the semantic information and pronunciation features of polyphonic characters under different word segmentation methods is used to extract the context embedding of the polyphonic characters to be tested from the word segmentation sequence of the processed text containing the polyphonic characters to be tested. This accurately captures the pronunciation features of the polyphonic characters in the current context. The linear projection matrix trained in the pronunciation prediction network is used to fuse the target word segmentation and its part-of-speech information to determine the conditional weight matrix. Based on the conditional weight matrix, the prediction probability for different pronunciation tags in the network output process is dynamically adjusted, enabling the pronunciation prediction network to better understand the context in which the polyphonic characters are located and more accurately capture the pronunciation features of the polyphonic characters in the current context. This achieves more accurate pronunciation prediction, thereby improving the accuracy and robustness of polyphonic character prediction, reducing user confusion caused by pronunciation prediction errors, and improving the user experience.

[0146] Based on the steps of the above polyphonic character prediction method, such as Figure 1BThe exemplary diagram illustrates a pronunciation prediction network structure. The pronunciation prediction network involved in this application may include at least an input layer, an encoding layer, a conditional weighting layer, and an output layer. The input layer receives processed text containing polyphonic characters to be tested and obtains an initial word segmentation sequence obtained by segmenting words into units. The encoding layer obtains the context embedding of the polyphonic characters to be tested based on the initial word segmentation sequence and can also perform further encoding processing on the context embedding. The parameters of the conditional weighting layer include a linear projection matrix, used to process the initial word segmentation sequence based on the linear projection weights, i.e., to determine if the initial word segmentation sequence contains polyphonic characters. The target word segmentation of the polyphonic character to be tested is performed, and at least one of the part-of-speech tag and the cross-feature of the target word segmentation and the part-of-speech tag is linearly projected to generate a conditional weight matrix. The output layer takes the output results of the encoding layer and the conditional weighted layer as input, and obtains the logit value corresponding to each pronunciation tag by mapping the output result of the encoding layer to a space with the same dimension as the number of pronunciation tags. The output activation function of the conditional weight matrix is ​​applied to generate a predicted probability distribution for the pronunciation tag set based on the logit value, and then the pronunciation of the polyphonic character to be tested is output based on the predicted probability distribution.

[0147] The input layer described above can be a simple text processing module that converts a text string into a word segmentation sequence. Alternatively, this input layer can use external tools or additional word segmentation networks to obtain the initial word segmentation sequence. The encoding layer can employ a pre-trained word embedding network to extract the context embedding of the polyphonic character under test. Alternatively, the encoding layer can use structures such as convolutional neural networks, recurrent neural networks, or Transformers to further encode the context embedding to extract deeper semantic information. The conditional weighting layer can contain one or more fully connected layers used to perform feature mapping on the input target word segment and its part-of-speech tag, generating vector representations of the target word segment, part-of-speech tag vector representations, and their cross-features. A linear transformation is then performed using a linear projection matrix to generate a conditional weight matrix with the same dimension as the number of pronunciation tags. In practical applications, the conditional weighting layer can also employ an attention mechanism to dynamically adjust the weights to better capture the pronunciation features of polyphonic characters in different contexts. The output layer can contain at least one fully connected layer and a softmax activation function. The fully connected layer maps the output of the encoding layer to a space with the same dimension as the number of phonetic tags, obtaining the logit value. The softmax activation function converts the logit value into a probability distribution, ensuring that the sum of all probabilities is 1. During the training phase, the output layer can use loss functions such as cross-entropy loss or negative log-likelihood loss to optimize network performance.

[0148] It is understood that the exemplary network hierarchy of the pronunciation prediction network described above can be flexibly adjusted according to design requirements in practical applications, and this application does not limit it in this regard.

[0149] In some embodiments, to improve the accuracy of text segmentation, a pre-set vocabulary is provided in this embodiment. This pre-set vocabulary may include at least commonly used Chinese words, and can be set using the top N most frequent words from the word-level segmentation tool used. For example, if the word-level segmentation tool is jieba, the pre-set vocabulary may include at least 20,000 of the most frequent words from jieba. Regarding the aforementioned step S101, this embodiment proposes a new segmentation method based on this vocabulary, see [link to relevant documentation]. Figure 2 The flowchart shown illustrates the steps involved in this word segmentation method:

[0150] S201, The processed text is segmented using a word-level word segmentation tool to obtain a first word segmentation sequence;

[0151] The word-level segmentation tool refers to a tool in natural language processing technology that can identify and segment words in text. It is trained on a large corpus and can accurately identify word boundaries in Chinese, English, or other languages. In this embodiment, the word-level segmentation tool includes segmentation methods other than those set in the pronunciation prediction network. It can include segmentation tools from related technologies, such as jieba, SnowNLP, LTP (Language Technology Platform), HanLP, NLPIR, THULAC, FoolNLTK, etc. The tool can be dynamically selected based on the actual situation to better suit the current segmentation task.

[0152] In this step, the processed text is used as input to a word-level segmentation tool. The tool, using a pre-trained model or algorithm, identifies and segments words in the text, obtaining the corresponding segmented sequence output by the tool as the first segmented sequence. For example, the processed text "He is a bank accountant" can be represented by the first segmented sequence output by the word-level segmentation tool as "He / is / bank accountant".

[0153] S202, for each first word in the first word segmentation sequence, determine whether the first word segmentation matches a preset vocabulary;

[0154] The preset vocabulary can include a custom vocabulary and the vocabulary of a pre-trained pronunciation prediction network. The custom vocabulary can be determined based on the vocabulary provided by the word-level segmentation tool used in step S201 or other word-level segmentation tools. That is, the performance of the pronunciation prediction network in Chinese word segmentation can be enhanced by adding an additional set number of Chinese words to the vocabulary (vocab.txt) of the pronunciation prediction network.

[0155] In this embodiment, the preset vocabulary is used to determine whether the segmentation result of the word-level segmentation tool meets expectations. By traversing each first segment in the first segmentation sequence, it checks whether the first segment matches any word included in the preset vocabulary, i.e., whether the first segment exists in the preset vocabulary. If the first segment exists in the preset vocabulary, it means that the first segment is a high-frequency word or a known word, and the segmentation result can be directly retained. Conversely, if the first segment does not exist in the preset vocabulary, it means that the first segment may be a low-frequency word, a new word, or a non-word fragment obtained due to inaccurate segmentation by the word-level segmentation tool, and further segmentation processing is required.

[0156] For example, for the first word segmentation sequence "he / is / bank accountant", the three words "he", "is", and "bank accountant" are checked in turn to see if they exist in the preset vocabulary to determine whether the first word segmentation matches the preset vocabulary.

[0157] S2031, if the first word segmentation fails to match the preset vocabulary, the first word segmentation is updated to the second word segmentation; the second word segmentation is obtained by segmenting the first word segmentation using the word segmentation function set in the pronunciation prediction network;

[0158] If the first word segment fails to match the preset vocabulary, it means that the preset vocabulary does not include the first word segment, and further processing is required. In this embodiment, for the first word segment that fails to match, the word segmentation function set in the pronunciation prediction network is called, such as the tokenize function built into the pronunciation prediction network, to re-segment the first word segment, so as to further divide the first word segment into smaller units such as characters or subwords, to obtain the second word segment output by the word segmentation function of the network, and the first word segment in the first word segmentation sequence is updated to the second word segment.

[0159] For example, for the first word segmentation sequence "he / is / bank accountant", the three word segments "he", "is", and "bank accountant" are detected in sequence. If the first word segment "bank accountant" does not exist in the preset vocabulary, that is, the first word segmentation fails to match the preset vocabulary, then the word segmentation function in the pronunciation prediction network is used to segment the first word segment "bank accountant" again. If the second word segmentation function outputs "bank / accountant", then after the first word segmentation is updated to the second word segmentation, the first word segmentation sequence is represented as "he / is / bank / accountant".

[0160] Regarding updating the first segment to the second segment, if the first segment is different from the corresponding second segment, the first segment can be updated to the second segment; otherwise, if the first segment is the same as the corresponding second segment, the first segment can remain unchanged.

[0161] S2032, if the first word segmentation is successfully matched with the preset vocabulary, the next word segmentation of the first word segmentation is matched with the preset vocabulary until all the first word segmentation in the first word segmentation sequence is matched.

[0162] If the first word segment matches a preset vocabulary, it means that the preset vocabulary includes the first word segment. Therefore, the first word segment is skipped, and the next word segment is checked to see if it matches the preset vocabulary. For example, taking the first word segment sequence ":he / is / bank accountant" as an example, if the first word segment "he" matches the preset vocabulary, the check continues to check if the first word segment "is" matches the preset vocabulary, until the matching check of the last word in the first word segment sequence is completed.

[0163] In this embodiment, a vocabulary list containing commonly used Chinese words is pre-set, and initial word segmentation is performed using a word-level word segmentation tool. The segmentation results are then matched against the pre-set vocabulary list. If a match fails, the word segmentation function built into the pronunciation prediction network is used for re-segmentation. This effectively addresses situations where some rare words, new words, or technical terms are not in the pre-set vocabulary list, improving the generalization ability of word segmentation processing. Since the words in the pre-set vocabulary list are more in line with Chinese language habits, the above steps reduce the possibility of word segmentation errors, thus improving the quality of word segmentation while ensuring efficiency.

[0164] In some embodiments, for the aforementioned step S102, obtaining the context embedding of the polyphonic character to be tested based on the initial word segmentation sequence, see [link to relevant documentation]. Figure 3 The exemplary flowchart can be implemented in the following way:

[0165] S301. Encode each token in the initial token sequence to obtain an initial embedding vector corresponding to each token, including a word embedding vector and a position embedding vector. The word embedding vector is used to represent the semantic information of the token itself. The position embedding vector is used to represent the position information of the token itself.

[0166] This initial embedding vector is used to initially capture and represent the semantic information of each token in the token sequence. It includes a word embedding vector and a position embedding vector. Among them, the word embedding vector maps each token unit in the initial token sequence to a vector in the real number space to generate a vector representation. Each token is represented as a vector of a fixed length. This vector can capture the semantic relationship between words and is used to represent the semantic information of the token itself. For example, for the polyphonic character "行" as part of the token "银行", the word embedding vector contains the overall semantic of the word "银行". The position embedding vector is used to introduce the word order information in the initial token sequence. It can be generated in various ways, such as using sine and cosine functions (such as the position encoding in the Transformer model) or a learned position embedding matrix. Since the word embedding itself does not contain the position information of the token unit in the processed text, the position embedding vector is used to supplement this position information. The position embedding vector is used to represent the position information of the token itself.

[0167] The position information represented by the position embedding vector can at least include at least one of relative position encoding and absolute position encoding. Relative position encoding is used to capture the relative position relationship between token units. It is calculated based on the relative position between token units. By calculating the position difference between two token units and applying a learnable function or matrix to generate relative position encoding. Absolute position encoding assigns a fixed encoding to each position, which can compensate for the absolute position information that may be lost when there is only relative position encoding.

[0168] In this embodiment, by combining, such as adding or concatenating, the word embedding vector and the position embedding vector of each token, the initial embedding vector of each token can be obtained. For example, for a token sequence containing polyphonic characters such as "我 / 今天 / 去 / 银行", each token will have an initial embedding vector Mi. Using Ei to represent the word embedding vector and Si to represent the position embedding vector, the corresponding initial embedding vector can be expressed as: "我": M1 = E1 + S1, "今天": M2 = E2 + S2, "去": M3 = E3 + S3, "银行": M4 = E4 + S4.

[0169] S302. According to the initial embedding vector of each token, fuse the context information for the initial embedding vector of the target token through an attention mechanism to generate the context embedding of the待测多音字 (to-be-determined polyphonic character).

[0170] The attention mechanism allows the network to dynamically focus on different parts of the input sequence when processing text, calculates the importance of each token unit in the initial tokenization sequence for the current task, and fuses feature information according to the weights representing the importance.

[0171] In an embodiment, the attention mechanism is used to calculate the correlation or importance between the target token and each token in the initial tokenization sequence. Through the attention weight matrix learned by the network, the query vector of the target token is calculated, and the key vector and value vector of each token unit in the initial tokenization sequence are calculated. The dot product or other similarity metrics are used to calculate the similarity score between the query vector and each key vector as the attention score. The softmax function is used to normalize the attention score into an attention weight, which represents the importance of each token unit in the initial tokenization sequence to the target token. Then, the value vectors of all tokens are weighted and summed according to the attention weights to obtain a vector representation that fuses the context information, which is used as the context embedding of the to-be-detected polyphonic character, so that this context embedding contains the information of the target token and its surrounding tokens, and can reflect the complete semantics of the to-be-detected polyphonic character in the current context.

[0172] For example, taking the initial tokenization sequence "我 / 今天 / 去 / 银行" as an example, the initial embedding vector representations corresponding to the tokens are: "我": M1 = E1S1, "今天": M2 = E2S2, "去": M3 = E3S3, "银行": M4 = E4S4. The attention weight matrix learned in the attention mechanism includes Q, K, and V. Based on the attention mechanism , Q is the query vector, representing the information that needs to be focused on currently; K is the key vector, representing the features of the candidate information; V is the value vector, representing the actual content of the candidate information. dk is the dimension of the key vector, which is used to scale the dot product result to prevent gradient disappearance or explosion. Then the calculation process of the attention mechanism can be expressed as: (1) Calculate the query vector QM4 for the target token "银行"; (2) For each token unit in the initial tokenization sequence, calculate its key vector KMi and value vector VMi; (3) Calculate the similarity score score(QM4, KMi) between the query vector and each key vector; (4) Use the softmax function to normalize the score into the attention weight Pi4; (5) Weight and sum the value vectors VMi of all tokens according to the attention weight Pi4 to obtain the context embedding of the to-be-detected polyphonic character.

[0173] In the embodiments of the present disclosure, by combining the word embedding vector, the position embedding vector, and the attention mechanism, the pronunciation prediction network can dynamically focus on the part related to the to-be-detected polyphonic character in the initial tokenization sequence, and fuse the information of this part to generate the context embedding of the to-be-detected polyphonic character, so that the network can more accurately understand the meaning of the to-be-detected polyphonic character in the current context, thereby improving the accuracy of polyphonic character pronunciation prediction.

[0174] In some embodiments, the linear projection matrix in step S103 may include at least one of a first projection matrix, a second projection matrix, and a third projection matrix. The first projection matrix is ​​used to project the cross-feature formed by the character encoding vector of the target word and the part-of-speech encoding vector corresponding to the part-of-speech of the target word. The second projection matrix is ​​used to project the character encoding vector of the target word. The third projection matrix is ​​used to project the part-of-speech encoding vector corresponding to the part-of-speech of the target word. During the pronunciation prediction network structure setting stage, the setting of the linear projection matrix is ​​determined according to the type of input feature to be projected. For example, when only the cross-feature of the character encoding vector of the target word and the part-of-speech encoding vector corresponding to the part-of-speech of the target word is projected, the linear projection matrix may only include the first projection matrix; when the cross-feature, the target word, and the part-of-speech of the target word are projected simultaneously, the linear projection matrix includes the first, second, and third projection matrices.

[0175] Based on this linear projection matrix, such as Figure 4A The flowchart shown illustrates that, for step S103, which involves performing linear projection of at least one of the following in the initial word segmentation sequence: the target word containing the polyphonic character to be tested, the part-of-speech tag of the target word, and the cross-feature of the target word and the part-of-speech tag, to generate a conditional weight matrix, step S401 can be executed first to obtain the character encoding vector of the target word and the part-of-speech tag encoding vector, and to extract the cross-feature of the character encoding vector and the part-of-speech tag encoding vector. Next, step S4021 can be used to obtain the first projection result of the cross-feature using the first projection matrix in the linear projection matrix, or step S4022 can be used to obtain the second projection result of the character encoding vector using the second projection matrix in the linear projection matrix, or step S4023 can be used to obtain the third projection result of the part-of-speech tag encoding vector using the second projection matrix in the linear projection matrix, and then the conditional weight matrix can be generated based on the obtained projection results. When simultaneously projecting the cross features, character encoding vectors, and part-of-speech encoding vectors, step S403 can be executed to generate the conditional weight matrix based on the first projection result, the second projection result, and the third projection result. Steps S4021, S4022, and S4023 are three parallel steps, and at least one of these projection processes can be selected to obtain the corresponding projection result as needed.

[0176] The cross-feature is a richer feature representation generated by combining character-level information and part-of-speech (POS) information. It is used to capture the interaction between character encoding vectors and POS encoding vectors. This can be understood as combining character encoding vectors and POS encoding vectors in some form to capture their interactive information. In this embodiment, the character encoding vectors and POS encoding vectors can be processed using element-wise multiplication, element-wise addition, concatenation, linear transformation, etc., to obtain the cross-feature. It can be selected and used according to the specific task requirements to improve the network's representation ability and prediction performance.

[0177] Element-wise multiplication generates a new vector as the cross feature by multiplying the character encoding vector and the part-of-speech encoding vector element-wise. This method emphasizes the common features between the target word segments and their parts of speech. Assuming the character encoding vector is c and the part-of-speech encoding vector is p, the cross feature vector f can be expressed as: f = c ⊙ p, where ⊙ represents element-wise multiplication. Correspondingly, element-wise addition generates a new vector as the cross feature by adding the character encoding vector and the part-of-speech encoding vector element-wise. This method combines the information from both vectors while retaining their respective characteristics.

[0178] Concatenation refers to combining character encoding vectors and part-of-speech encoding vectors to generate a higher-dimensional vector. This method preserves the complete information of both, such as f=[c, p]. Linear transformation refers to combining character encoding vectors and part-of-speech encoding vectors through linear transformations (such as fully connected layers) to generate a new vector. This method can learn more complex interaction relationships. For example, assuming W is the weight matrix and b represents the bias vector, the cross feature vector f can be expressed as f=W[cp]+b.

[0179] In some embodiments, the step of extracting the cross features of the character encoding vector and the part-of-speech encoding vector can be implemented in any of the following ways: obtaining the tensor product of the character encoding vector and the part-of-speech encoding vector as the cross feature; or obtaining the element-wise sum of the character encoding vector and the part-of-speech encoding vector as the cross feature.

[0180] Regarding obtaining the tensor product of the character encoding vector and the part-of-speech encoding vector, the tensor product of two vectors (also known as the outer product, cross product, or vector product) is an important and useful concept in linear algebra and vector analysis. Let there be two three-dimensional vectors a = (a1, a2, a3) and b = (b1, b2, b3). Then the tensor product is a new vector f = a × b, which can be defined as f = (a2b3 - a3b2, a3b1 - a1b3, a1b2 - a2b1). Based on the principle of tensor product calculation for three-dimensional vectors, the tensor product of the character encoding vector and the part-of-speech encoding vector can be generated. It can be understood that the essence of the tensor product is to map the character encoding vector and the part-of-speech encoding vector to generate new, higher-dimensional vectors and matrices. Therefore, the calculation method for the tensor product can also be defined by the developers themselves, using non-standard tensor product calculations, such as bilinear transformation.

[0181] In this embodiment of the disclosure, by obtaining the character encoding vector and part-of-speech encoding vector of the target word segment and extracting their cross features, the characteristics of the polyphonic character to be tested in different contexts can be captured more comprehensively. By using a linear projection matrix to perform feature linear projection on the character encoding vector, part-of-speech encoding vector and their cross features, a conditional weight matrix is ​​generated. This not only reduces the feature dimension and the amount of computation, but also effectively learns and integrates the part-of-speech and semantic information of the polyphonic character to be tested in the context and the word segmentation itself through the learning ability of the projection matrix, making the features more distinguishable in the projected space.

[0182] In some embodiments, when the cross features, character encoding vectors and part-of-speech encoding vectors are projected simultaneously in the foregoing embodiments to obtain a first projection result, a second projection result and a third projection result, in order to more flexibly adjust the contribution of these projection results in generating the conditional weight matrix, this embodiment proposes three hyperparameters as weight coefficients for these three projection results to dynamically adjust the importance of each projection result during the training or inference process of the pronunciation prediction network.

[0183] Based on this, see Figure 4BAs shown, in the aforementioned embodiment, step S403, generating the conditional weight matrix based on the first projection result, the second projection result, and the third projection result, can be implemented in the following manner: S4031, weighting the first projection result, the second projection result, and the third projection result according to three set hyperparameters to obtain three corresponding weighted projection results; S4032, adding the three weighted projection results element-wise to generate an initial weight matrix; S4033, converting each element in the initial weight matrix into a probability value to obtain the conditional weight matrix; the probability value ranges from 0 to 1. During the element-wise addition of the three weighted projection results, a bias term can be set to adjust the addition result.

[0184] To obtain the conditional weight matrix by converting each element in the initial weight matrix into a probability value, the initial weight matrix can be activated using the sigmoid function to convert each element in the initial weight matrix into a probability value.

[0185] For example, if hyperparameters α1, α2, and α3 are set sequentially for the first, second, and third projection results, then after obtaining the three projection results r1, r2, and r3, the corresponding three weighted projection results can be represented as α1*r1, α2*r2, and α3*r3. The initial weight matrix generated by adding the elements of the three weighted projection results can be represented as f'=α1*r1+α2*r2+α3*r3. Then, each element in f' is converted into a probability value to obtain the conditional weight matrix f.

[0186] In this embodiment, by introducing hyperparameters as weight coefficients, the contributions of cross features, character encoding vectors, and part-of-speech encoding vectors to the generation of the conditional weight matrix are dynamically adjusted. This enables the network to more accurately capture the pronunciation changes of polyphonic characters in different contexts, thereby improving the accuracy and reliability of prediction. Furthermore, by adjusting the weight coefficients, the pronunciation prediction network can better adapt to different training data and testing environments, improving the network's generalization ability and robustness.

[0187] In some embodiments, after generating the conditional weight matrix, in order to more effectively utilize the matrix to focus on the candidate pronunciations of the polyphonic character to be tested and reduce interference with other non-candidate pronunciations, this embodiment proposes another processing step based on fusing specific mask information with the conditional weights. For example... Figure 4C As shown, after generating the conditional weight matrix, the following steps may also be included:

[0188] S404, determine the candidate pronunciation tags of the polyphonic character to be tested; the candidate pronunciation tags represent all pronunciations of the polyphonic character to be tested;

[0189] The candidate pronunciation labels are all the labels selected from the set of pronunciation labels set in the pronunciation prediction network, and the pronunciations represented by the pronunciation labels belong to the pronunciations of the polyphonic character to be measured. For example, if the polyphonic character to be measured is "行", the candidate pronunciation labels refer to all the pronunciation labels in the set of pronunciation labels that represent the polyphonic pronunciations of "行".

[0190] In this embodiment, the mapping relationship between the polyphonic character and all its pronunciations can be pre-constructed and stored. For the polyphonic character to be measured, the mapping record containing the polyphonic character to be measured is searched from the mapping relationship, and all the pronunciations of the polyphonic character to be measured are extracted from the mapping record. Then, the label representing the pronunciation of the polyphonic character to be measured is determined from all the pronunciation labels included in the set of pronunciation labels as the candidate pronunciation label.

[0191] S405. For the set of pronunciation labels, set the mask 0 for other pronunciation labels except the candidate pronunciation labels, and set the mask 1 for the candidate pronunciation labels to obtain a mask matrix;

[0192] The mask matrix is a matrix with the same shape as the conditional weight matrix. The length of the matrix is the same as the number of pronunciation labels included in the set of pronunciation labels. Its element values are 0 or 1, and are used to indicate which elements in the conditional weight matrix should be retained (elements with a value of 1), and which elements should be ignored or set to zero (elements with a value of 0).

[0193] That is, for each pronunciation label in the set of pronunciation labels set in the pronunciation prediction network, if the pronunciation represented by the pronunciation label belongs to one of the pronunciations of the polyphonic character to be measured, set the value of the element at the corresponding element position in the mask matrix to 1. Conversely, if the pronunciation represented by the pronunciation label does not belong to one of the pronunciations of the polyphonic character to be measured, set the value of the element at the corresponding element position in the mask matrix to 0.

[0194] For example, if there are 100 pronunciation labels in the set of pronunciation labels, the 51st pronunciation label represents the pronunciation "hang2", and the 77th pronunciation label represents the pronunciation "xing2", then for the polyphonic character "行" to be measured, the length of this mask matrix is 100, including 100 elements. Except for the 51st and 77th elements with a value of 1, the values of the elements in other positions are 0.

[0195] S406. Multiply the mask matrix and the conditional weight matrix element by element to obtain an updated conditional weight matrix.

[0196] The mask matrix ensures that only the conditional weights corresponding to the candidate pronunciation tags of the polyphonic character under test are retained, while the conditional weights of non-candidate pronunciation tags are set to 0. This allows the updated conditional weight matrix to dynamically adjust the prediction probabilities of the candidate pronunciation tags of the polyphonic character under test based on contextual information during pronunciation prediction. Element-wise multiplication of the mask matrix and the conditional weight matrix can be performed using matrix operation libraries such as NumPy, TensorFlow, or PyTorch. The element-wise multiplication at each corresponding position yields a new matrix that serves as the updated conditional weight matrix.

[0197] In this embodiment of the disclosure, by introducing a mask matrix, the conditional weights corresponding to the candidate pronunciation labels of the polyphonic characters to be tested are retained, while the conditional weights of the non-candidate pronunciation labels are set to 0. This effectively reduces the interference of other non-candidate pronunciations on the prediction results, allowing the network to focus more on the prediction of candidate pronunciations, thereby improving the accuracy and reliability of the prediction.

[0198] For the part-of-speech (POS) of target words containing polyphonic characters in the initial segmentation sequence described in the foregoing embodiments, it can be determined by generating the initial segmentation sequence and then POS-labeling the target word using relevant techniques. However, to further improve the accuracy and efficiency of POS labeling, especially when processing target words containing polyphonic characters, this embodiment proposes a method to obtain the POS of the target word based on a POS prediction subnetwork set in a pronunciation prediction network. Specifically, the pronunciation prediction network in this embodiment not only includes a part for predicting the pronunciation of polyphonic characters, but also additionally sets up a POS prediction subnetwork, which can use contextual information related to the target word, such as preceding and following vocabulary and grammatical structure, to predict the POS of the target word. The specific implementation of this POS prediction subnetwork can be a multilayer perceptron, convolutional neural network, recurrent neural network or its variants such as long short-term memory network, gated recurrent unit, etc., or a combination of these networks. The POS prediction subnetwork receives the contextual embedding information of the polyphonic character to be tested and the target word, and through a series of calculations and transformations, finally outputs the POS prediction result of the target word.

[0199] In this embodiment, a part-of-speech (POS) prediction subnetwork is set up in the pronunciation prediction network, which is trained along with the pronunciation prediction network. The classification category of this POS prediction subnetwork is part-of-speech classification. This subnetwork has a pre-set set of POS tags, including multiple POS tags representing different parts of speech. For example, the parts of speech represented by the POS tags may include, but are not limited to, unknown tags, nouns, verbs, adjectives, numerals, pronouns, adverbs, prepositions, conjunctions, auxiliary words, interjections, onomatopoeia, etc. Based on this POS prediction subnetwork, the method may further include the following step of obtaining the POS of the target word segment:

[0200] The context embedding of the polyphonic character to be tested is mapped into a part-of-speech score vector through the first mapping parameter in the part-of-speech prediction subnetwork; the part-of-speech score vector includes the unnormalized score of each part-of-speech tag in the part-of-speech tag set;

[0201] By using an activation function for part-of-speech classification, a predicted probability distribution for the set of part-of-speech tags is generated based on the part-of-speech score vector, and the predicted part-of-speech of the target word is determined as the part-of-speech of the target word.

[0202] The first mapping parameter refers to the weights and biases learned by the part-of-speech prediction subnetwork during training, which are used to map the input to the part-of-speech score vector. Each element in the part-of-speech score vector corresponds to an unnormalized score (i.e., logit value) of a part-of-speech tag. This unnormalized score represents the probability that the target word belongs to that part of speech, but it has not been normalized and therefore cannot be directly used as a probability.

[0203] The activation function is used to introduce nonlinear factors, enabling the part-of-speech prediction subnetwork to learn complex mapping relationships. In the part-of-speech classification task of this embodiment, the softmax function can be used as the activation function to convert the part-of-speech score vector into a predicted probability distribution. In this predicted probability distribution, each part-of-speech tag corresponds to a probability value, which represents the likelihood that the target word belongs to that part of speech.

[0204] Based on the predicted probability distribution, the part-of-speech tag with the highest probability value can be directly selected as the predicted part-of-speech tag of the target word. This tag represents the most likely part-of-speech tag of the target word determined by the part-of-speech prediction sub-network in the given context of the processed text.

[0205] In this embodiment, a part-of-speech prediction subnetwork is used to accurately predict the part of speech of the polyphonic character to be tested, thereby providing useful information for subsequent tasks such as polyphonic character pronunciation prediction. Furthermore, since the part-of-speech prediction subnetwork and the pronunciation prediction network share some input features or contextual information, information sharing and collaborative optimization between the two can be achieved to a certain extent.

[0206] In some embodiments, the generation of a predicted probability distribution for the pronunciation tag set by applying the output activation function of the conditional weight matrix in step S104 can be specifically implemented through the following steps a1-a3:

[0207] a1, based on the second mapping parameters learned during the training phase, the context embedding is mapped into a pronunciation score vector; the pronunciation score vector includes the unnormalized score of each pronunciation tag in the pronunciation tag set;

[0208] The second mapping parameter refers to the weights and biases in the pronunciation prediction network used to map the context embedding of the polyphonic character to be tested, or the encoded features generated by encoding the context embedding, to the output such as the pronunciation score vector. It is understood that the term "second" has no specific indicative meaning and is only used to distinguish it from other mapping parameters of the same type in this application.

[0209] The pronunciation score vector is a vector output by the pronunciation prediction network. Each element corresponds to the unnormalized score of a pronunciation label. This score represents the probability that the context of the polyphonic character being tested corresponds to that pronunciation label, but it has not been normalized and therefore cannot be directly used as a probability. The unnormalized score of this pronunciation label is the raw, unnormalized prediction value output by the pronunciation prediction network, corresponding to the linear prediction result of each pronunciation label. It will then be converted into a probability value through normalization by the output activation function (such as the softmax function). For example, in a three-class classification problem (assuming the categories are cat, dog, and bird), the network output layer may have three neurons, corresponding to the logits values ​​of the three categories respectively. For the i-th label "dog", the unnormalized score of the i-th label is the output value of the neuron in the network output layer corresponding to the "dog" category.

[0210] b2, for each pronunciation tag, obtain the conditional weight in the conditional weight matrix that is in the same position as the unnormalized score of the pronunciation tag;

[0211] Since the length of the conditional weight matrix is ​​the same as the number of pronunciation tags included in the pronunciation tag set, for each element in the pronunciation score vector—that is, the unnormalized score of each pronunciation tag—a conditional weight can be found in the conditional weight matrix at the same position. This conditional weight reflects the relative importance of each pronunciation tag within the context of the currently processed text.

[0212] b3. Based on the unnormalized score of the pronunciation tag and the conditional weight, the output activation function is used to normalize the score to obtain the probability value of the pronunciation tag.

[0213] In this embodiment, the normalization process using the output activation function can be implemented as follows: First, determine the exponential result of the unnormalized score of the pronunciation tag. For example, using the natural constant e as the base and the unnormalized score logit of the pronunciation tag as the exponent, the exponential result corresponding to the pronunciation tag can be expressed as exp{logit}, i.e., e raised to the power of logit. Next, obtain the product of the exponential result and the conditional weight of the pronunciation tag as the target result. Assuming the conditional weight of the pronunciation tag is Wi, the target result can be expressed as... Then, based on the sum of the target operation results of all pronunciation tags, the target operation result of each pronunciation tag is normalized. That is, the sum of the target operation results of all pronunciation tags is first calculated, which can be expressed as: N represents the number of pronunciation tags in the pronunciation tag set. Let represent the unnormalized score of the j-th pronunciation tag. Then, calculate the ratio of the target operation result of each pronunciation tag to the sum of the target operation results of all pronunciation tags, and use this ratio as the normalized result of that pronunciation tag. That is, the normalized result of the i-th pronunciation tag can be expressed as: The normalized result is used as the probability value of the pronunciation tag.

[0214] In some embodiments, the polyphonic character prediction method provided in this application is implemented by a pronunciation prediction network. This pronunciation prediction network is trained by acquiring the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods and minimizing the weighted result of the loss function corresponding to each predicted pronunciation. Based on this, the pronunciation prediction network can be trained at least through the following steps b1-b3:

[0215] b1. For each sample sentence, obtain the first segmentation sequence obtained by segmenting characters as units, and obtain the second segmentation sequence obtained by segmenting words as units; the sample sentence includes a polyphonic character and the pronunciation tag of the polyphonic character; the second segmentation sequence also includes the part-of-speech tag of the segmentation character containing the polyphonic character;

[0216] In the training process described in this embodiment, the same sample sentence will be segmented using both single-character segmentation and word segmentation methods. The first segmentation sequence obtained by segmenting single characters is considered as an independent word for each character. The second segmentation sequence obtained by segmenting words is further labeled with the true part-of-speech tag for the segmented words containing the polyphonic characters to be tested in the sample sentence.

[0217] See Figure 5 The training process diagram shown below, taking the sample sentence "He went to the bank today" as an example, the first word segmentation sequence is represented as "He / today / go / to / bank", the second word segmentation sequence is represented as "He / today / go / to / bank", and the second word segmentation sequence carries the part-of-speech tag "noun" for the word "bank".

[0218] b2, the first word segmentation sequence and the second word segmentation sequence are input together into the pronunciation prediction network to obtain the first predicted pronunciation corresponding to the first word segmentation sequence and the second predicted pronunciation corresponding to the second word segmentation sequence; the conditional weight matrix is ​​determined based on the word segmentation of the second word segmentation sequence containing polyphonic characters and the part-of-speech tag;

[0219] During the training process of this embodiment, the first word segmentation sequence and the second word segmentation sequence of the same sample sentence are simultaneously input into the pronunciation prediction network for processing. The process of obtaining the first predicted pronunciation of the first word segmentation sequence and the second word segmentation sequence is the same. As Figure 5 shown, taking the first word segmentation sequence as an example, that is, the input layer is used to obtain the context embedding of the polyphonic character to be measured according to the initial word segmentation sequence, and this context embedding is input into the encoding layer to obtain the encoded processing feature output by the encoding layer. This encoded processing feature is input into the output layer, and through the output activation function of the conditional weight matrix, the predicted probability distribution based on the first word segmentation sequence is obtained. Furthermore, the pronunciation corresponding to the pronunciation label with the highest probability is selected as the first predicted pronunciation.

[0220] Among them, the conditional weight matrix is determined based on the word segmentation containing the polyphonic character and the词性标记 (lexical category marker) in the second word segmentation sequence. Therefore, after the second word segmentation sequence is input into the pronunciation prediction network and the context embedding output by the input layer is obtained, this context embedding is input into the conditional weighting layer to utilize this conditional weighting layer to combine the target word segmentation "银行" (bank) containing the polyphonic character "行" to be measured and the lexical category of the target word segmentation, "noun", to generate a conditional weight matrix through a learnable linear projection matrix.

[0221] For example, for the first word segmentation sequence represented as "他 / 今 / 天 / 去 / 银 / 行" (he / today / go / to / bank), the first predicted pronunciation output by the network is "xing2", and for the second word segmentation sequence represented as "他 / 今天 / 去 / 银行" (he / today / go / to / bank), the first predicted pronunciation output by the network is "hang2".

[0222] b3. According to the first predicted pronunciation and the second predicted pronunciation, obtain the target loss function, and adjust the model parameters of the pronunciation prediction network by minimizing the target loss function; the target loss includes the first loss between the first predicted pronunciation and the pronunciation marker, and the second loss between the second predicted pronunciation and the pronunciation marker.

[0223] In machine learning, a loss function is used to quantify the difference between the network's prediction and the actual result. The process of training the network is to continuously adjust the network parameters to minimize this loss function. In this embodiment, the target loss function used in the training phase consists of a weighted sum of two loss functions: the first part is the loss function based on the first predicted pronunciation of the first word segmentation sequence, that is, calculating the first loss of the first predicted pronunciation and the pronunciation marker of the polyphonic character to be tested in the sentence. Assuming the first predicted pronunciation is represented as Ph1 and the pronunciation marker is represented as Ph0, then the first loss can be expressed as L1=L(Ph0,Ph1); the second part is the loss function based on the second predicted pronunciation of the second word segmentation sequence, that is, calculating the second loss of the second predicted pronunciation and the pronunciation marker of the polyphonic character to be tested in the sentence. Assuming the second predicted pronunciation is represented as Ph2, then the second loss can be expressed as L2=L(Ph0,Ph2); setting the weight coefficient of the first loss as β1 and the weight coefficient of the second loss as β2, and the sum of β1 and β2 as 1, then the target loss function can be expressed as L=β1*L1+β2*L2. Regarding the first loss or the second loss, any loss function suitable for calculating pronunciation loss, such as mean square error, root mean square error, or cross-entropy loss, can be used, and this application does not limit it in this regard.

[0224] Since the specific processing logic of the pronunciation prediction network during training is the same as that of the pronunciation prediction network in the aforementioned pronunciation prediction method, this embodiment will not repeat it. For the specific process, please refer to the description of the aforementioned embodiment.

[0225] In some embodiments, when a part-of-speech (POS) prediction subnetwork is set in the pronunciation prediction network to predict the POS of the target word segment during the inference phase, the loss of the POS prediction subnetwork also needs to be considered during training. Based on this POS prediction subnetwork, the training process of the pronunciation prediction network may further include the following steps:

[0226] The pronunciation prediction network determines the context embedding of the word segment containing polyphonic characters; the context embedding is input into the part-of-speech prediction sub-network to obtain the predicted part-of-speech output by the part-of-speech prediction sub-network; the conditional weight matrix is ​​determined based on the word segment containing polyphonic characters and the predicted part-of-speech; the target loss function also includes a third loss of the predicted part-of-speech and the part-of-speech tag.

[0227] That is, when a part-of-speech prediction subnetwork is set up, the target loss function used in the training process will consist of three parts, including the first loss of the first predicted pronunciation and the pronunciation mark, the second loss of the second predicted pronunciation and the pronunciation mark, and the third loss of the part-of-speech prediction subnetwork predicting the part-of-speech and the part-of-speech mark. Assuming the third loss is represented as L3, the target loss function can be represented as L=γ1*L1+γ2*L2+γ3*L3, where γ1, γ2, and γ3 are three hyperparameters used to control the contribution of different loss functions to the target loss function.

[0228] In this embodiment, by incorporating part-of-speech (POS) prediction into the training process, the pronunciation prediction network can learn more contextual information. Since POS not only reflects the grammatical role of words in a sentence but also relates to their pronunciation, POS prediction helps the model more accurately predict the pronunciation of polyphonic characters, thereby improving overall prediction accuracy. Simultaneously, the POS prediction sub-network provides an additional supervisory signal, which, together with the pronunciation prediction loss, guides the parameter updates of the pronunciation prediction network, helping it learn more robust and comprehensive feature representations.

[0229] Corresponding to the aforementioned embodiments of the polyphonic character prediction method, see [link to relevant documentation]. Figure 6 As shown, this application also provides an embodiment of a polyphonic character prediction device, applied to a pre-trained pronunciation prediction network. The pronunciation prediction network has a set of pronunciation labels corresponding to all pronunciations of each polyphonic character. The pronunciation prediction network is trained by acquiring the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods and minimizing the weighted result of the loss function corresponding to each predicted pronunciation. The device includes:

[0230] The word segmentation processing module 601 is used to obtain the initial word segmentation sequence obtained by segmenting words into units for the processed text containing polyphonic characters to be tested.

[0231] The context acquisition module 602 is used to acquire the context embedding of the polyphonic character to be tested based on the initial word segmentation sequence;

[0232] The conditional weight matrix generation module 603 is used to perform feature linear projection on at least one of the target word containing the polyphonic character to be tested in the initial word segmentation sequence, the part-of-speech of the target word, and the cross feature of the target word and the part-of-speech, using a linear projection matrix to generate a conditional weight matrix; the length of the conditional weight matrix is ​​the same as the length of the pronunciation tag set;

[0233] The prediction probability and pronunciation determination module 604 is used to determine the pronunciation of the polyphonic character to be tested by applying the output activation function of the conditional weight matrix and generating a prediction probability distribution for the pronunciation tag set based on the context embedding.

[0234] In some embodiments, the word segmentation module is specifically used for:

[0235] The processed text is segmented using a word-level word segmentation tool to obtain the first segmentation sequence;

[0236] For each first word in the first word segmentation sequence, if the first word fails to match the preset vocabulary, the first word is updated to a second word; the second word is obtained by segmenting the first word using the word segmentation function set in the pronunciation prediction network.

[0237] The first segmented word sequence after the update is completed is determined as the initial segmented word sequence.

[0238] In some embodiments, the context acquisition module is specifically used for:

[0239] Each word in the initial word segmentation sequence is encoded to obtain an initial embedding vector for each word, including a word embedding vector and a position embedding vector; the word embedding vector is used to represent the semantic information of the word itself; the position embedding vector is used to represent the position information of the word itself.

[0240] Based on the initial embedding vector of each word segment, the context information is fused to the initial embedding vector of the target word segment through an attention mechanism to generate the context embedding of the polyphonic character to be tested.

[0241] In some embodiments, the conditional weight matrix generation module is specifically used for:

[0242] Obtain the character encoding vector of the target word segment and the part-of-speech encoding vector of the part-of-speech tag;

[0243] Extract the cross-feature vector of the character encoding vector and the part-of-speech encoding vector, and use the first projection matrix in the linear projection matrix to obtain the first projection result of the cross-feature;

[0244] The second projection matrix in the linear projection matrix is ​​used to obtain the second projection result of the character encoding vector; the third projection result of the part-of-speech encoding vector is also used to obtain the third projection result of the part-of-speech encoding vector.

[0245] The conditional weight matrix is ​​generated based on the first projection result, the second projection result, and the third projection result.

[0246] In some embodiments, the conditional weight matrix generation module, when extracting the cross features of the character encoding vector and the part-of-speech encoding vector, includes at least one of the following methods:

[0247] Obtain the tensor product of the character encoding vector and the part-of-speech encoding vector as the cross feature;

[0248] Alternatively, the element-wise sum of the character encoding vector and the part-of-speech encoding vector can be obtained as the cross-feature.

[0249] In some embodiments, the conditional weight matrix generation module, when generating the conditional weight matrix based on the first projection result, the second projection result, and the third projection result, includes:

[0250] Based on the three set hyperparameters, the first projection result, the second projection result, and the third projection result are weighted respectively to obtain the corresponding three weighted projection results;

[0251] The three weighted projection results are summed element by element to generate an initial weight matrix;

[0252] Each element in the initial weight matrix is ​​converted into a probability value to obtain the conditional weight matrix; the probability values ​​range from 0 to 1.

[0253] In some embodiments, after generating the conditional weight matrix, the apparatus further includes:

[0254] Determine candidate pronunciation tags for the polyphonic character to be tested; the candidate pronunciation tags represent all pronunciations of the polyphonic character to be tested;

[0255] For the set of pronunciation tags, set a mask 0 for all pronunciation tags except the candidate pronunciation tags, and set a mask 1 for the candidate pronunciation tags to obtain a mask matrix;

[0256] The mask matrix is ​​multiplied element by element by the conditional weight matrix to obtain the updated conditional weight matrix.

[0257] In some embodiments, the pronunciation prediction network includes a part-of-speech prediction subnetwork and is provided with a set of part-of-speech tags; the apparatus further includes:

[0258] The context embedding of the polyphonic character to be tested is mapped into a part-of-speech score vector through the first mapping parameter in the part-of-speech prediction subnetwork; the part-of-speech score vector includes the unnormalized score of each part-of-speech tag in the part-of-speech tag set;

[0259] By using an activation function for part-of-speech classification, a predicted probability distribution for the set of part-of-speech tags is generated based on the part-of-speech score vector, and the predicted part-of-speech of the target word is determined as the part-of-speech of the target word.

[0260] In some embodiments, the prediction probability and pronunciation determination module is specifically used for:

[0261] Based on the second mapping parameters learned during the training phase, the context embedding is mapped into a pronunciation score vector; the pronunciation score vector includes the unnormalized score of each pronunciation tag in the pronunciation tag set;

[0262] For each pronunciation tag, obtain the conditional weight in the conditional weight matrix that is in the same position as the unnormalized score of the pronunciation tag;

[0263] Based on the unnormalized score of the pronunciation tag and the conditional weight, the output activation function is used to perform normalization processing to obtain the probability value of the pronunciation tag.

[0264] In some embodiments, the prediction probability and pronunciation determination module, when performing normalization processing through the output activation function based on the unnormalized score of the pronunciation tag and the conditional weights, includes:

[0265] Determine the exponential result of the unnormalized score of the pronunciation tag;

[0266] The product of the exponential calculation result and the conditional weight of the pronunciation tag is obtained as the target calculation result;

[0267] The target operation result of each pronunciation tag is normalized based on the sum of the target operation results of all pronunciation tags.

[0268] In some embodiments, the apparatus further includes a training step for a pronunciation prediction network:

[0269] For each sample sentence, a first segmentation sequence obtained by segmenting individual characters is obtained, and a second segmentation sequence obtained by segmenting words is obtained; the sample sentence includes a polyphonic character and a pronunciation tag for the polyphonic character; the second segmentation sequence also includes a part-of-speech tag for the segmentation character containing the polyphonic character;

[0270] The first word segmentation sequence and the second word segmentation sequence are input together into the pronunciation prediction network to obtain the first predicted pronunciation corresponding to the first word segmentation sequence and the second predicted pronunciation corresponding to the second word segmentation sequence; the conditional weight matrix is ​​determined based on the word segmentation of the second word segmentation sequence containing polyphonic characters and the part-of-speech tag;

[0271] Based on the first predicted pronunciation and the second predicted pronunciation, a target loss function is obtained, and the model parameters of the pronunciation prediction network are adjusted by minimizing the target loss function; the target loss includes a first loss between the first predicted pronunciation and the pronunciation marker, and a second loss between the second predicted pronunciation and the pronunciation marker.

[0272] In some embodiments, the pronunciation prediction network includes a part-of-speech prediction subnetwork, and the apparatus further includes: determining the context embedding of the word segment containing polyphonic characters through the pronunciation prediction network;

[0273] The context embedding is input into the part-of-speech prediction subnetwork to obtain the predicted part-of-speech output by the part-of-speech prediction subnetwork; the conditional weight matrix is ​​determined based on the word segmentation containing polyphonic characters and the predicted part-of-speech; the target loss function also includes a third loss of the predicted part-of-speech and the part-of-speech tag.

[0274] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0275] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0276] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 7 As shown, the electronic device 700 includes at least one processor 701, a memory 702, and a bus 703. At least one processor 701 is electrically connected to the memory 702. The memory 702 is configured to store at least one computer-executable instruction, and the processor 701 is configured to execute the at least one computer-executable instruction to perform the steps of any polyphonic character prediction method provided in any embodiment or optional implementation of this application.

[0277] Furthermore, the processor 701 can be an FPGA (Field-Programmable Gate Array) or other devices with logic processing capabilities, such as an MCU (Microcontroller Unit) or a CPU (Central Processing Unit).

[0278] This application also provides another readable storage medium storing a computer program that, when executed by a processor, implements the steps of any polyphonic character prediction method provided in any embodiment or optional implementation of this application.

[0279] The readable storage media provided in this application include, but are not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, readable storage media include any medium by which a device (e.g., a computer) stores or transmits information in a readable form.

[0280] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0281] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0282] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A multi-phoneme prediction method, characterized by, A pronunciation prediction network is applied to a pre-trained network, wherein the network has a set of pronunciation labels corresponding to all pronunciations of each polyphonic character; the network is trained by acquiring the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods and minimizing the weighted result of the loss function corresponding to each predicted pronunciation; the method includes: For the processed text containing polyphonic characters to be tested, obtain the initial word segmentation sequence obtained by segmenting words into units; Based on the initial word segmentation sequence, obtain the context embedding of the polyphonic character to be tested; Using a linear projection matrix, a feature linear projection is performed on the target word containing the polyphonic character to be tested in the initial word segmentation sequence, the part-of-speech tag of the target word, and at least one of the cross features of the target word and the part-of-speech tag, to generate a conditional weight matrix; the length of the conditional weight matrix is ​​the same as the length of the pronunciation tag set. By applying the output activation function of the conditional weight matrix, a predicted probability distribution for the pronunciation tag set is generated based on the context embedding, thereby determining the pronunciation of the polyphonic character to be tested. The generated conditional weight matrix includes: Obtain the character encoding vector of the target word segment and the part-of-speech encoding vector of the part-of-speech tag; Extract the cross-feature vector of the character encoding vector and the part-of-speech encoding vector, and use the first projection matrix in the linear projection matrix to obtain the first projection result of the cross-feature; The second projection result of the character encoding vector is obtained by using the second projection matrix in the linear projection matrix; The third projection result of the part-of-speech encoding vector is obtained by using the second projection matrix in the linear projection matrix; The conditional weight matrix is ​​generated based on the first projection result, the second projection result, and the third projection result.

2. The method of claim 1, wherein, The process of obtaining the initial word segmentation sequence obtained by segmenting words into units includes: The processed text is segmented using a word-level word segmentation tool to obtain the first segmentation sequence; For each first word in the first word segmentation sequence, if the first word fails to match the preset vocabulary, the first word is updated to a second word; the second word is obtained by segmenting the first word using the word segmentation function set in the pronunciation prediction network. The first segmented word sequence after the update is completed is determined as the initial segmented word sequence.

3. The method of claim 1, wherein, The step of obtaining the context embedding of the polyphonic character to be tested based on the initial word segmentation sequence includes: Each word in the initial word segmentation sequence is encoded to obtain an initial embedding vector for each word, including a word embedding vector and a position embedding vector; the word embedding vector is used to represent the semantic information of the word itself; the position embedding vector is used to represent the position information of the word itself. Based on the initial embedding vector of each word segment, the context information is fused to the initial embedding vector of the target word segment through an attention mechanism to generate the context embedding of the polyphonic character to be tested.

4. The method of claim 1, wherein, The extraction of the cross features between the character encoding vector and the part-of-speech encoding vector includes at least one of the following: Obtain the tensor product of the character encoding vector and the part-of-speech encoding vector as the cross feature; Alternatively, the element-wise sum of the character encoding vector and the part-of-speech encoding vector can be obtained as the cross-feature.

5. The method of claim 1, wherein, The step of generating the conditional weight matrix based on the first projection result, the second projection result, and the third projection result includes: Based on the three set hyperparameters, the first projection result, the second projection result, and the third projection result are weighted respectively to obtain the corresponding three weighted projection results; The three weighted projection results are summed element by element to generate an initial weight matrix; Each element in the initial weight matrix is ​​converted into a probability value to obtain the conditional weight matrix; the probability values ​​range from 0 to 1.

6. The method of claim 1, wherein, After generating the conditional weight matrix, the method further includes: Determine candidate pronunciation tags for the polyphonic character to be tested; the candidate pronunciation tags represent all pronunciations of the polyphonic character to be tested; For the set of pronunciation tags, set a mask 0 for all pronunciation tags except the candidate pronunciation tags, and set a mask 1 for the candidate pronunciation tags to obtain a mask matrix; The mask matrix is ​​multiplied element by element by the conditional weight matrix to obtain the updated conditional weight matrix.

7. The method of claim 1, wherein, The pronunciation prediction network includes a part-of-speech prediction subnetwork and is equipped with a set of part-of-speech tags; the method further includes: The context embedding of the polyphonic character to be tested is mapped into a part-of-speech score vector through the first mapping parameter in the part-of-speech prediction subnetwork; the part-of-speech score vector includes the unnormalized score of each part-of-speech tag in the part-of-speech tag set; By using an activation function for part-of-speech classification, a predicted probability distribution for the set of part-of-speech tags is generated based on the part-of-speech score vector, and the predicted part-of-speech of the target word is determined as the part-of-speech of the target word.

8. The method of claim 1, wherein, The step of generating a predicted probability distribution for the pronunciation tag set based on the context embedding by applying the output activation function of the conditional weight matrix includes: Based on the second mapping parameters learned during the training phase, the context embedding is mapped into a pronunciation score vector; the pronunciation score vector includes the unnormalized score of each pronunciation tag in the pronunciation tag set; For each pronunciation tag, obtain the conditional weight in the conditional weight matrix that is in the same position as the unnormalized score of the pronunciation tag; Based on the unnormalized score of the pronunciation tag and the conditional weight, the output activation function is used to perform normalization processing to obtain the probability value of the pronunciation tag.

9. The method of claim 8, wherein, The normalization process, based on the unnormalized score of the pronunciation tag and the conditional weights, and performed through the output activation function, includes: Determine the exponential result of the unnormalized score of the pronunciation tag; The product of the exponential calculation result and the conditional weight of the pronunciation tag is obtained as the target calculation result; The target operation result of each pronunciation tag is normalized based on the sum of the target operation results of all pronunciation tags.

10. The method according to claim 1, characterized in that, The method also includes a training step for a pronunciation prediction network: For each sample sentence, a first segmentation sequence obtained by segmenting individual characters is obtained, and a second segmentation sequence obtained by segmenting words is obtained; the sample sentence includes a polyphonic character and a pronunciation tag for the polyphonic character; the second segmentation sequence also includes a part-of-speech tag for the segmentation character containing the polyphonic character; The first word segmentation sequence and the second word segmentation sequence are input together into the pronunciation prediction network to obtain the first predicted pronunciation corresponding to the first word segmentation sequence and the second predicted pronunciation corresponding to the second word segmentation sequence; the conditional weight matrix is ​​determined based on the word segmentation of the second word segmentation sequence containing polyphonic characters and the part-of-speech tag; Based on the first predicted pronunciation and the second predicted pronunciation, a target loss function is obtained, and the model parameters of the pronunciation prediction network are adjusted by minimizing the target loss function; the target loss includes a first loss between the first predicted pronunciation and the pronunciation marker, and a second loss between the second predicted pronunciation and the pronunciation marker.

11. The method according to claim 10, characterized in that, The pronunciation prediction network includes a part-of-speech prediction subnetwork, and the method further includes: The context embedding of the word segment containing polyphonic characters is determined by the pronunciation prediction network. The context embedding is input into the part-of-speech prediction subnetwork to obtain the predicted part-of-speech output by the part-of-speech prediction subnetwork; The conditional weight matrix is ​​determined based on the word segmentation of characters containing polyphonic characters and the predicted part-of-speech tags; The target loss function also includes a third loss based on the predicted part-of-speech tag and the part-of-speech tag.

12. A polyphonic character prediction device, characterized in that, A pre-trained pronunciation prediction network is applied, wherein the pronunciation prediction network has a set of pronunciation labels corresponding to all pronunciations of each polyphonic character; the pronunciation prediction network is trained by acquiring the predicted pronunciations of polyphonic characters in the same sample sentence under different word segmentation methods, and minimizing the weighted result of the loss function corresponding to each predicted pronunciation; the device includes: The word segmentation module is used to obtain the initial word segmentation sequence by word segmentation for the processed text containing polyphonic characters to be tested. The context acquisition module is used to acquire the context embedding of the polyphonic character to be tested based on the initial word segmentation sequence; The conditional weight matrix generation module is used to perform feature linear projection on at least one of the following in the initial word segmentation sequence: the target word containing the polyphonic character to be tested, the part-of-speech tag of the target word, and the cross-feature of the target word and the part-of-speech tag; the length of the conditional weight matrix is ​​the same as the length of the pronunciation tag set. The prediction probability and pronunciation determination module is used to generate a prediction probability distribution for the pronunciation tag set based on the context embedding by applying the output activation function of the conditional weight matrix, and to determine the pronunciation of the polyphonic character to be tested. The generated conditional weight matrix includes: Obtain the character encoding vector of the target word segment and the part-of-speech encoding vector of the part-of-speech tag; Extract the cross-feature vector of the character encoding vector and the part-of-speech encoding vector, and use the first projection matrix in the linear projection matrix to obtain the first projection result of the cross-feature; The second projection result of the character encoding vector is obtained by using the second projection matrix in the linear projection matrix; The third projection result of the part-of-speech encoding vector is obtained by using the second projection matrix in the linear projection matrix; The conditional weight matrix is ​​generated based on the first projection result, the second projection result, and the third projection result.

13. An electronic device, characterized in that, include: Memory, processor; The memory is used to store computer programs; The processor is configured to invoke the computer program to implement the method as described in any one of claims 1-11.

14. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-11.

Images