Small-parameter Chinese polyphone disambiguation method capable of saving computing power

Abstract

The invention relates to the field of natural language processing, and discloses a small-parameter Chinese polyphone disambiguation method capable of saving computing power. The method is characterized in that a small Chinese pre-training model is adopted, and the problems that when a large-scale pre-training model carries out a polyphone disambiguation task, deployment on a low-resource computing platform is difficult, and a large number of computing power resources are occupied are solved. Then, text data are input into the pre-training model for training, a corresponding semantic feature sequence is obtained, meanwhile, Chinese word segmentation and part-of-speech tagging feature sequences are added, and a classifier composed of a bidirectional LSTM and a full-connection layer is used for classification, so that the accuracy of polyphone disambiguation is further improved, and the accuracy of polyphone disambiguation is improved. And verifying the effectiveness of word segmentation and part-of-speech tagging features on polyphone disambiguation and the effectiveness of bidirectional LSTM on polyphone classification. Meanwhile, through a contrast experiment, the effectiveness of the method in the aspect of polyphone disambiguation and the superiority of the method compared with a large-scale pre-training model are verified.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for disambiguating Chinese polyphones using a small number of parameters and saving computing power. Background Art

[0002] Speech and text are two of the primary ways humans receive and express information, playing a vital role in our lives. Text-to-speech (TTS) technology converts text into speech signals. In recent years, with the advancement of computer computing power and the development of deep learning, speech synthesis technology has found applications in our daily lives, most commonly in mobile phone voice assistants and voice navigation. As speech synthesis technology advances, its application scenarios are also expanding.

[0003] Grapheme-to-phoneme conversion (G2P) is a crucial component of Chinese speech synthesis, as Chinese pronunciation is independent of its glyph form. The accuracy of Chinese G2P determines whether Chinese speech synthesis can produce accurately pronounced speech. Polyphones are characters that have different pronunciations depending on their semantic and syntactic usage in context. Therefore, a major challenge in Chinese G2P is polyphone disambiguation—determining the pronunciation of characters with multiple pronunciations. In recent years, pre-trained language models based on the BERT architecture have significantly improved the accuracy of polyphone classification by learning contextual representations. However, the number of parameters in these models generally exceeds 110M, posing significant deployment challenges in low-resource computing environments such as embedded devices or mobile terminals. Summary of the Invention

[0004] In order to solve the problem that large-scale pre-training models are difficult to deploy on low-resource computing platforms and have too many parameters, the present invention uses a small Chinese pre-training model MiniRBT. Text data is input into the pre-training model for training to obtain the corresponding semantic feature sequence, and Chinese word segmentation and part-of-speech tagging feature sequences are added at the same time. A classifier composed of a bidirectional LSTM and a fully connected layer is used to classify them, further improving the accuracy of polyphone disambiguation, verifying the effectiveness of word segmentation and part-of-speech tagging features for polyphone disambiguation, and verifying the effectiveness of bidirectional LSTM for polyphone classification. At the same time, through comparative experiments, the effectiveness of the present invention in polyphone disambiguation and its superiority over the use of large-scale pre-training models are verified.

[0005] To achieve the above objectives, the present invention proposes a method for disambiguating Chinese polyphones with a small number of parameters and low computing power, which comprises the following specific steps:

[0006] S1: Preprocess the dataset;

[0007] S2: Perform word segmentation and词性 tagging on Chinese sentences in the dataset;

[0008] S3: Input Chinese sentences in the dataset into the pre-trained model MiniRBT to obtain semantic feature sequences;

[0009] S4: Add the word segmentation embedding feature sequence and词性 embedding feature sequence corresponding to the text to the semantic features output by MiniRBT, which can be regarded as the context feature vectors corresponding to each Chinese character in the text;

[0010] S5: Classify the obtained feature vectors through a classifier composed of BLSTM and fully connected layers, and output the final pronunciation of polyphonic characters.

[0011] In step S1, there are several problems with the CPP dataset, which impose certain limitations on the polyphonic character disambiguation model. First, the pronunciations of some polyphonic characters in the CPP dataset are incorrect. For example, for the character "会" in "会稽", its correct pronunciation should be "kuai4", while the pronunciation marked in CPP is "hui4". Second, the pronunciations of some polyphonic characters in CPP are outdated. For example, for the character "骑" in "坐骑", the pronunciation marked in CPP is "ji4", but according to the latest edition of the Modern Chinese Dictionary, the correct pronunciation should be "qi2". Finally, some polyphonic characters in the CPP dataset only have one pronunciation collected. Take the Chinese character "识" as an example, it has two pronunciations "shi2" and "zhi4". However, there are only 200 sentences with the pronunciation "shi2" in CPP, and no sentences with the pronunciation "zhi4", which will lead to an unbalanced dataset.

[0012] To address the above problems, the following processing was performed on the CPP dataset. According to the latest edition of the Modern Chinese Dictionary, the incorrect pronunciations in the dataset were corrected, and at the same time, the polyphonic characters with changed pronunciations were changed to the latest corrected pronunciations. Finally, the polyphonic characters with only one pronunciation in the dataset were deleted. After a series of processing, the final polyphonic character disambiguation dataset was obtained, containing 69094, 8640, and 8935 sentences, which are used for the training set, validation set, and test set respectively.

[0013] Furthermore, in step S2:

[0014] Chinese word segmentation and词性 tagging can be used as an aid to improve the accuracy of polyphonic character disambiguation. Therefore, in this invention, the features of Chinese word segmentation and词性 tagging are also added to the input of the polyphonic character disambiguation classifier to provide more abundant feature information to improve the performance of the polyphonic character disambiguation model.

[0015] Research findings show that Chinese word segmentation can affect the semantic or syntactic usage of a character or word, thereby influencing the pronunciation of polyphonic characters. For example, in the sentence "The school / is / holding / a sports meeting", the pronunciation of the character "校" is "xiao4", while in the sentence "How to / correct / the pronunciation of polyphonic characters", the pronunciation of the character "校" is "jiao4". Therefore, different word segmentation methods can lead to different pronunciations of polyphonic characters.

[0016] Chinese word segmentation aims to determine and divide the boundaries between words in a sentence, and at the same time assign word segmentation tags to each Chinese character in the sentence. These tags are divided into four categories: B (Begin), E (End), M (Middle), and S (Single), representing the four situations where the Chinese character is at the starting position, ending position, non-starting and ending position of a certain word, and the Chinese character alone constitutes a word respectively. After performing word segmentation preprocessing on the sentence, it can be transformed into word segmentation embedding features according to the word segmentation tags.

[0017] Chinese word tagging is based on the result of Chinese word segmentation, and marks the correct word types for each word, such as nouns, verbs, adjectives, etc. After obtaining the Chinese word tagging result through text preprocessing, we tag the word type label of the word where each Chinese character in the sentence is located, and then transform these labels into word tagging embedding features. Finally, add the word segmentation embedding feature sequence, word tagging embedding feature sequence corresponding to the text and the semantic features output by MiniRBT, and then input them into the classifier of the polyphonic character disambiguation model.

[0018] Furthermore, in step S3:

[0019] The pre-trained model MiniRBT takes the original Chinese character sequence as input and outputs a series of semantic features. Before reaching the embedding sequence, the character embedding layer and the position embedding layer process the input character sequence respectively. The following Transformer block converts the embedding sequence into a semantic feature sequence.

[0020] The MiniRBT model is pre-trained on a large amount of unlabeled data, divided into two tasks: predicting the masked input characters and predicting the next sentence, and learning semantic representations from the original character sequence.

[0021] Furthermore, in step S4:

[0022] Adding the word segmentation embedding feature sequence and word tagging embedding feature sequence corresponding to the text to the semantic features output by MiniRBT can be regarded as the context feature vector corresponding to each Chinese character in the text.

[0023] The semantic features output by MiniRBT are denoted as x = x mini. Chinese word segmentation tags can also be converted into vectors through embedding. For each character, we represent its tokenized information vector as x token We then concatenate it with the character vector before the final prediction, i.e. x = [x mini ;x token ].

[0024] After word segmentation, part-of-speech tagging assigns a tag to each Chinese word. For each character, we assume that its tag is inherited from the word it belongs to. Tags can also be converted into vectors through embedding. For each character, we represent the vector of its POS tag information as x pos We then concatenate the character vector x = [x mini ;x token ;x pos ].

[0025] Furthermore, in step S5, the feature vector obtained in step S4 is classified by a classifier composed of a BLSTM and a fully connected layer, and the final pronunciation of the polyphonetic word is output.

[0026] The method uses a two-layer fully connected network to predict the pronunciation of polyphonetic characters based on MiniRBT output sequences. The first fully connected layer is shared by all polyphonetic characters. The second fully connected output layer is not shared. Each polyphonetic character has a separate output layer with a number of units equal to the number of possible pronunciations. The classifier is trained using a softmax cross-entropy loss. The model predicts the label distribution using p.

[0027] p=softmax(Wx+b)

[0028] Where W,b are the weights and biases of the fully connected layer.

[0029] During training, we use cross entropy to measure the loss between the predicted label distribution p and the one-hot encoding of the label y:

[0030]

[0031] where C represents the total number of all pronunciations collected from all polyphones.

[0032] During inference, to restrict the selection of polyphonetic characters from candidate pronunciations, we use a hard mask m = [m1, ···, m C ], except for the candidate pronunciation which is 1, all other pronunciations are 0. Then the final pronunciation is determined according to the maximum value of p⊙m, where ⊙ represents element-by-element multiplication.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] The polyphone disambiguation method based on the pre-training model can better learn text data and understand the semantics of the context in the text, so that the classification of polyphones can be more accurate. At the same time, the present invention also adds Chinese word segmentation and part-of-speech tagging to the semantic features, which also strengthens the model's understanding of the text semantics and promotes the accuracy of polyphone disambiguation. Finally, in order to solve the problem that the current pre-training model data is too large and difficult to deploy on a low-resource computing platform, the present invention uses a small Chinese pre-training model with smaller parameters, which can be deployed on a low-resource computing platform and improves the accuracy of polyphone disambiguation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a method for disambiguating Chinese polyphones with a small number of parameters that saves computing power.

[0036] Figure 2 This is a schematic diagram showing the necessity of adding Chinese word segmentation and part-of-speech tagging in a specific implementation example of the present invention.

[0037] Figure 3 This is a structural diagram of the pre-training model in the method of the present invention.

[0038] Figure 4 This is an overall structural diagram of a small-parameter Chinese polyphone disambiguation method that saves computing power as described in the present invention.

[0039] Figure 5 This is a schematic diagram of the result of why a single-layer BLSTM is used in a classifier of an example of a specific implementation of the present invention.

[0040] Figure 6 This is a schematic diagram of the results of the polyphone disambiguation method of a specific embodiment of the present invention and other commonly used methods. DETAILED DESCRIPTION

[0041] like Figure 1 As shown, the example of the present invention provides a method for disambiguating Chinese polyphones with a small number of parameters and saving computing power, comprising the following steps:

[0042] In step S1, the following processing was performed on the CPP dataset to address the issues it encountered. According to the latest version of the Modern Chinese Dictionary, incorrect pronunciations in the dataset were corrected. Polyphonetic characters with altered pronunciations were also corrected to the newly corrected pronunciations. Finally, polyphonetic characters with a single pronunciation were deleted from the dataset. After this series of processing, the final polyphonetic character disambiguation dataset was obtained, containing 69,094, 8,640, and 8,935 sentences for the training, validation, and test sets, respectively.

[0043] In step S2, Chinese word segmentation aims to determine the boundaries between words in a sentence and divide them, while assigning a word segmentation label to each Chinese character in the sentence. These labels are divided into four categories: B (Begin), E (End), M (Middle) and S (Single), which respectively represent the four situations in which the Chinese character is at the starting position, the ending position, the non-starting and ending position of a word, and the Chinese character constitutes a word alone. After the sentence is pre-processed with word segmentation, it can be converted into word segmentation embedding features based on the word segmentation labels. Chinese part-of-speech tagging is to mark the correct word type for each word based on the Chinese word segmentation results, such as noun, verb, adjective, etc. After obtaining the Chinese part-of-speech tagging results through text pre-processing, we label each Chinese character in the sentence with the part-of-speech label of the word it belongs to, and then convert these labels into part-of-speech embedding features. Finally, the word segmentation embedding feature sequence and part-of-speech embedding feature sequence corresponding to the text are added to the semantic features output by MiniRBT, and then input into the classifier of the polyphone disambiguation model.

[0044] like Figure 2 As shown, the accuracy of the method of the present invention is 93.40% when the features of Chinese word segmentation and part-of-speech tagging are not added. When only the word segmentation feature is added, the accuracy is 93.36%, and the two results are not much different. In fact, the result of adding only the word segmentation feature is slightly lower than the accuracy of not adding either of them, indicating that adding only the word segmentation feature does not improve the performance of the model. However, when only the part-of-speech tagging is added, the accuracy of the model reaches 93.82%, and the accuracy is improved by 0.42%, indicating that the addition of part-of-speech tagging improves the accuracy of the model for polyphone classification. When both word segmentation and part-of-speech tagging features are added to the model, the accuracy of the model is improved again, reaching 94.14%, which is 0.32% higher than adding only part-of-speech tagging and 0.74% higher than not adding either of them. It can be seen that the addition of Chinese word segmentation and part-of-speech tagging improves the accuracy of polyphone classification, verifying its effectiveness in improving model performance.

[0045] Step S3 is as follows Figure 3 As shown in Figure 2, the Chinese sentences in the dataset are input into the pre-trained model MiniRBT to obtain semantic feature sequences.

[0046] The pre-trained model MiniRBT accepts a raw Chinese character sequence as input and outputs a series of semantic features. The character embedding layer and position embedding layer process the input character sequence before converting it into an embedding sequence. The following Transformer block converts the embedding sequence into a semantic feature sequence.

[0047] The MiniRBT model is pre-trained on a large amount of unlabeled data and is divided into two tasks: predicting masked input characters and predicting the next sentence, and learning semantic representations from the original character sequence.

[0048] The MiniRBT network consists of two modules: an input representation module that converts Chinese characters into feature vectors and a main network module that performs feature operations.

[0049] The [CLS] tag is a special tag set by the MiniRBT network to identify a functional position and is always placed at the beginning of a sentence. Similarly, the [SEP] tag is also a special tag set by MiniRBT to indicate a separator between two sentences. Text input to MiniRBT is first converted into character embeddings corresponding to individual Chinese characters. The network then incorporates sentence information, using labels 0 or 1 to indicate whether the current text belongs to the first or second sentence. These features are then converted into sentence embeddings based on the labels. Finally, the position embeddings corresponding to each Chinese character are calculated. MiniRBT pre-sets a maximum of 512 characters per input. Therefore, the model pre-initializes 512 embedding vectors to represent positions. Each character only needs to select the corresponding position vector as its positional embedding based on its position in the sentence. Finally, each Chinese character in the sentence is represented as the sum of the character embedding, the sentence embedding, and the position embedding, resulting in a feature sequence for the Chinese sentence.

[0050] The main network module is composed of multiple stacked network models similar to the encoders in the Transformer, including multiple multi-head self-attention modules, layer normalization, fully connected layers, and residual connections. By varying parameters such as the number of stacked Transformer encoders in the MiniRBT network, the number of heads in the Transformer encoder's multi-head attention mechanism, and the dimension of the network's word vectors, MiniRBT networks with various parameter sizes can be constructed. Generally, a larger parameter size indicates a stronger model.

[0051] In order to be deployed on a low-resource computing platform, this paper uses a small Chinese pre-training model, sets the number of heads of the multi-head attention mechanism in the Transformer encoder to 8, and the dimension of the word vector in the network to 512.

[0052] Step S4 is as follows Figure 3 As shown in Figure 3, the word segmentation embedding feature sequence and part-of-speech embedding feature sequence corresponding to the text are added to the semantic features output by MiniRBT, which can be regarded as the context feature vector corresponding to each Chinese character in the text.

[0053] The semantic features output by MiniRBT are denoted as x = x mini . Chinese word segmentation tags can also be converted into vectors through embedding. For each character, we represent its tokenized information vector as x tokenWe then concatenate it with the character vector before the final prediction, i.e. x = [x mini ;x token ].

[0054] After word segmentation, part-of-speech tagging assigns a tag to each Chinese word. For each character, we assume that its tag is inherited from the word it belongs to. Tags can also be converted into vectors through embedding. For each character, we represent the vector of its POS tag information as x pos We then concatenate the character vector x = [x mini ;x token ;x pos ].

[0055] Step S5 is as follows Figure 4 As shown in FIG, the obtained feature vector is classified by a classifier composed of two fully connected layers, and the final pronunciation of the polyphonetic word is output.

[0056] A two-layer fully connected network is used to predict the pronunciation of polyphonetic characters based on MiniRBT output sequences. The first fully connected layer is shared by all polyphonetic characters. The second fully connected output layer is not shared. Each polyphonetic character has a separate output layer with a number of units equal to the number of possible pronunciations. The classifier is trained using a softmax cross-entropy loss. The model predicts the label distribution using p.

[0057] p=softmax(Wx+b)

[0058] Where W,b are the weights and biases of the fully connected layer.

[0059] During training, we use cross entropy to measure the loss between the predicted label distribution p and the one-hot encoding of the label y:

[0060]

[0061] where C represents the total number of all pronunciations collected from all polyphones.

[0062] For the training process, we first need to construct polyphone disambiguation training samples for the model. First, we process the polyphone text data corresponding to each polyphone in the training dataset separately.

[0063] Divide the polyphone text into several sets S1,…,S according to the pronunciation of the polyphone m , where m is the number of pronunciations of the polyphone. Each set should contain at least two polyphone text data points. If a polyphone text point cannot contain two data points for a particular pronunciation, all data points for that polyphone text point are discarded and not included in the training data.

[0064] A piece of data is randomly extracted from each text set of the polyphone without replacement, so a total of m polyphone texts are obtained, and the pronunciations of the polyphones in these texts are different. Use symbols t1,…,t m express.

[0065] A text set of the polyphone is randomly selected, and a polyphone text t is extracted from it. t is used as the target polyphone text for the polyphone pronunciation to be confirmed, and the actual pronunciation of the polyphone is used as the output target p of the model.

[0066] Then t1,…,t m ,t and p form a training sample of the model in this chapter, and the combination t1,…,t m ,t is the input of the model, and p is the target output of the model.

[0067] During inference, to restrict the selection of polyphonetic characters from candidate pronunciations, we use a hard mask m = [m1, ···, m C ], except for the candidate pronunciation which is 1, all other pronunciations are 0. Then the final pronunciation is determined according to the maximum value of p⊙m, where ⊙ represents element-by-element multiplication.

[0068] like Figure 5 As shown in the figure, without BLSTM, the accuracy of polyphone disambiguation was 93.53%, 0.61% lower than the 94.14% accuracy achieved with a single BLSTM layer. This demonstrates that BLSTM, with its powerful memory advantage, can better understand contextual information, thereby improving the accuracy of polyphone classification. However, when the number of BLSM layers increased to two or three, the model's polyphone disambiguation accuracy decreased, indicating that the increased number of layers made the model structure overly complex, reaching a performance bottleneck and preventing further improvement in classification accuracy.

[0069] Figure 6The accuracy comparison results of the method of the present invention and other commonly used polyphone disambiguation methods are shown. xpinyin achieved an accuracy of 67.36% on the CPP dataset, which is a relatively reasonable result for dictionary-based tools. The accuracy of pypinyin's polyphone disambiguation is much better than xpinyin, and is also relatively reasonable for the results based on dictionaries and word segmentation tools, with an accuracy 19.34% higher than xpinyin. The latter three models are polyphone disambiguation models based on deep learning, among which g2pM is a BLSTM-based model with an accuracy of 92.10%, which is 2.04% lower than the method of the present invention. The g2pW model, benefiting from the power of the Chinese BERT pre-training model, achieved the best accuracy of 97.46%, which is 3.32% higher than the method of the present invention. However, the MiniRBT model has only 10% of the parameters of the Chinese BERT model, but its accuracy reaches 97% of the accuracy of the g2pW model. Therefore, in order to enable the polyphone disambiguation model to be deployed on a low-resource computing platform, the method of the present invention uses fewer parameters and achieves better results. It can save more computing resources while achieving a good level of accuracy in polyphone disambiguation.

Claims

1. A method for disambiguating Chinese polyphones with a small number of parameters and saving computing power mainly includes the following steps: S1: Preprocess the dataset; S2: Segment and tag the Chinese sentences in the dataset; S3: Input the Chinese sentences in the dataset into the pre-trained model MiniRBT to obtain semantic feature sequences; S4: Add the word segmentation embedding feature sequence and part-of-speech embedding feature sequence corresponding to the text to the semantic features output by MiniRBT, which can be regarded as the context feature vector corresponding to each Chinese character in the text; S5: The obtained feature vector is classified by a classifier composed of BLSTM and a fully connected layer, and the final pronunciation of the polyphonetic word is output.

2. A method for disambiguating Chinese polyphones with a small number of parameters and saving computing power according to claim 1, characterized in that In step S1, the only open-source public dataset in the field of Chinese polyphone disambiguation is the CPP dataset. This dataset extracts sentences from Wikipedia Chinese text, filters out sentences containing traditional Chinese characters and sentences of unusual length, retains sentences containing polyphones, marks target polyphones, and balances the sample size. Two native Chinese speakers annotate the sentences, providing candidate pinyins, and discarding sentences with inconsistent annotations. Ultimately, 99,264 correctly annotated sentences are obtained, which are divided into a training set, a validation set, and a test set in an 8:1:1 ratio. The present invention addresses the issues existing in the CPP dataset, deletes some problematic sentences, and obtains a refined CPP dataset.

3. A method for disambiguating Chinese polyphones with a small number of parameters and saving computing power according to claim 1, characterized in that In step S2, the Chinese sentences in the data set are segmented and POS tagged, as follows: The present invention performs word segmentation and part-of-speech tagging on the Chinese sentences in the processed CPP dataset. Chinese word segmentation refers to finding the boundaries between words in a sentence and dividing them, and then assigning a word segmentation label to each Chinese character in the sentence. The types of labels include B (Begin), E (End), M (Middle), and S (Single), which respectively indicate that the Chinese character is at the beginning position of a word, the end position of a word, the non-beginning and end position of a word, and the Chinese character constitutes a word alone. After the sentence is pre-processed for word segmentation, the label can be converted into a word segmentation embedding feature based on the word segmentation label. Chinese part-of-speech refers to labeling each word with its correct word type, such as noun, verb, or adjective, based on the Chinese word segmentation results. After obtaining the Chinese part-of-speech tagging results through text preprocessing, we tag each Chinese character in the sentence with the part-of-speech label of the word it belongs to, and then convert this label into a part-of-speech embedding feature.

4. A method for disambiguating Chinese polyphones with a small number of parameters and saving computing power according to claim 1, characterized in that In step S3, the Chinese sentences in the dataset are input into the pre-trained model MiniRBT to obtain semantic feature sequences, as follows: The pre-trained model MiniRBT accepts a raw Chinese character sequence as input and outputs a series of semantic features. The character embedding layer and position embedding layer process the input character sequence before converting it into an embedding sequence. The following Transformer block converts the embedding sequence into a semantic feature sequence. The MiniRBT model is pre-trained on a large amount of unlabeled data and is divided into two tasks: predicting masked input characters and predicting the next sentence, and learning semantic representations from the original character sequence.

5. A method for disambiguating Chinese polyphones with a small number of parameters and saving computing power according to claim 1, characterized in that : In step S4, the word segmentation embedding feature sequence and part-of-speech embedding feature sequence corresponding to the text are added to the semantic features output by MiniRBT, which can be regarded as the context feature vector corresponding to each Chinese character in the text. The semantic features output by MiniRBT are denoted as x = x mini . Chinese word segmentation tags can also be converted into vectors through embedding. For each character, we represent its tokenized information vector as x token We then concatenate it with the character vector before the final prediction, i.e. x = [x mini ;x token ]. After word segmentation, part-of-speech tagging assigns a tag to each Chinese word. For each character, we assume that its tag is inherited from the word it belongs to. Tags can also be converted into vectors through embedding. For each character, we represent the vector of its POS tag information as x pos We then concatenate the character vector x = [x mini ;x token ;x pos ].

6. A method for disambiguating Chinese polyphones with a small number of parameters and saving computing power according to claim 1, characterized in that : In step S5, the feature vector obtained in step S4 is classified by a classifier composed of BLSTM and a fully connected layer, and the final polyphonetic pronunciation is output. The concatenated feature vector x is classified by a classifier consisting of a bidirectional LSTM and a fully connected layer. Leveraging the bidirectional LSTM's ability to memorize time series data, the model network performs further feature filtering, amplification, and enhancement on each feature vector from a global perspective. Finally, it is normalized through a Softmax layer to obtain the distribution probability of polyphones. The calculation process is as follows: x′=BLSTM(x) p=softmax(Wx′+b) Where W,b are the weights and biases of the fully connected layer. During training, we use cross entropy to measure the loss between the predicted label distribution p and the one-hot encoding of the label y: where C represents the total number of all pronunciations collected from all polyphonetic characters. During inference, to restrict the selection of polyphonetic characters from candidate pronunciations, we use a hard mask m = [m1, ···, m C ], except for the candidate pronunciation which is 1, all other pronunciations are 0. Then the final pronunciation is determined according to the maximum value of p⊙m, where ⊙ represents element-by-element multiplication.

Images