Model training method, polyphone disambiguation method and device and computer storage medium

Abstract

The invention provides a polyphone disambiguation model training method, a polyphone disambiguation method, a polyphone disambiguation device and a computer storage medium. The model training method comprises the following steps: extracting context features of target polyphones in a plurality of sentences; clustering all context features of the target polyphone to obtain a plurality of types of context feature sets; and training the corresponding polyphone disambiguation sub-network by using each type of context feature set to obtain a trained polyphone disambiguation model. Through the polyphone disambiguation method, the context features of the target polyphone in each sentence are clustered, and one polyphone disambiguation sub-network is independently trained for each category, so that the local plasticity of the polyphone disambiguation model can be improved, and the model training effect is improved.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a model training method for polyphone disambiguation, a polyphone disambiguation method, a polyphone disambiguation device, and a computer storage medium. Background Art

[0002] In the TTS (Text To Speech) system, G2P (Grapheme To Phoneme) is a key module that determines pronunciation accuracy. The purpose of G2P is to annotate text with pronunciation. In languages ​​such as Chinese and Japanese, polyphone disambiguation is the core algorithm of G2P, which directly determines the performance of the G2P module. Among them, the polyphone disambiguation algorithm determines the pronunciation of polyphones based on the context.

[0003] The existing mainstream algorithm solutions for polyphone disambiguation are all offline training on pre-labeled data sets, and the training is used for a long time. The existing training methods for polyphone disambiguation all adopt global training, which leads to poor local expressiveness of the model. Summary of the invention

[0004] In order to solve the above technical problems, the present application proposes a model training method for polyphone disambiguation, a polyphone disambiguation method, a polyphone disambiguation device and a computer storage medium.

[0005] In order to solve the above technical problems, the present application proposes a model training method for polyphone disambiguation, which includes:

[0006] Extracting contextual features of target polyphones in multiple sentences;

[0007] Clustering all context features of the target polyphone to obtain several sets of context features;

[0008] Each type of context feature set is used to train the corresponding polyphone disambiguation subnetwork to obtain a fully trained polyphone disambiguation model.

[0009] The step of extracting context features of target polyphones in multiple sentences includes:

[0010] Use the pre-trained language model to extract the vector sequence of each sentence;

[0011] The vector sequence is input into a multi-head self-attention layer to extract context features of the target polyphone in each sentence.

[0012] The method of using each type of context feature set to train a corresponding polyphone disambiguation subnetwork includes:

[0013] Initialize the network parameters of the polyphone disambiguation sub-network according to the network parameters of the polyphone disambiguation basic network;

[0014] The network parameters of the pre-trained language model and / or the network parameters of the polyphone disambiguation basic network are fixed, and the corresponding polyphone disambiguation sub-network is trained using each type of context feature set.

[0015] Wherein, the model training method further includes:

[0016] Inputting the context features of the target polyphone in each sentence into the polyphone disambiguation basic network to obtain the predicted pronunciation of the target polyphone;

[0017] The multi-head self-attention layer is trained using the predicted pronunciation.

[0018] In order to solve the above technical problems, the present application also proposes a polyphonetic character disambiguation method, which comprises:

[0019] Inputting the sentence to be disambiguated and its target polyphone into a polyphone disambiguation module, wherein the polyphone disambiguation module includes a polyphone disambiguation model trained by the above-mentioned model training method;

[0020] The pronunciation of the target polyphonetic character output by the polyphonetic character disambiguation module is obtained.

[0021] The step of inputting the sentence to be disambiguated and its target polyphonetic character into the polyphonetic character disambiguation module comprises:

[0022] Using a word segmentation module to segment the sentence to be disambiguated, and obtaining a word segmentation result of the target polyphone;

[0023] If the segmentation result of the target polyphone is a single word, the sentence to be disambiguated and the target polyphone and the target polyphone are input into the polyphone disambiguation model in the polyphone disambiguation module to predict the pronunciation of the target polyphone.

[0024] After obtaining the pronunciation of the target polyphonetic character output by the polyphonetic character disambiguation module, the polyphonetic character disambiguation method further includes:

[0025] When the predicted pronunciation of the target polyphonetic character output by the polyphonetic character disambiguation module is different from the actual pronunciation, obtaining the classification of the target polyphonetic character using the polyphonetic character disambiguation basic network in the polyphonetic character disambiguation model;

[0026] The real pronunciation is used to train the polyphone disambiguation subnetwork corresponding to the classification.

[0027] The step of training the polyphone disambiguation subnetwork corresponding to the classification by using the real pronunciation includes:

[0028] Obtaining the number of data samples of the classification;

[0029] When the number of the data samples exceeds a preset threshold, reclassifying all the data samples of the classification using the context features of the target polyphone of the sentence to be disambiguated as cluster centers to obtain two new classifications;

[0030] The real pronunciation is used to train the newly classified polyphone disambiguation subnetwork where the context features of the target polyphone of the sentence to be disambiguated are located.

[0031] Wherein, after the word segmentation module is used to segment the sentence to be disambiguated to obtain the word segmentation result of the target polyphone, the polyphone disambiguation method further includes:

[0032] If the segmentation result of the target polyphonetic character is a multi-character word, searching the pronunciation dictionary for the pre-stored pronunciation of the phrase where the target polyphonetic character is located;

[0033] If the pronunciation dictionary contains a pre-stored pronunciation of the phrase in which the target polyphone is located, the pre-stored pronunciation is used as the output of the polyphone disambiguation module;

[0034] If the pre-stored pronunciation of the phrase in which the target polyphone is located does not exist in the pronunciation dictionary, the sentence to be disambiguated and its target polyphone are input into the polyphone disambiguation model in the polyphone disambiguation module to predict the pronunciation of the target polyphone, and the output of the polyphone disambiguation module is used to update the pronunciation dictionary.

[0035] In order to solve the above-mentioned technical problems, the present application also proposes a polyphonetic character disambiguation device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the model training method and / or the polyphonetic character disambiguation method as described above.

[0036] In order to solve the above technical problems, the present application also proposes a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above-mentioned model training method and / or polyphone disambiguation method.

[0037] Compared with the prior art, the beneficial effects of the present application are as follows: the polyphone disambiguation device extracts the context features of the target polyphones in multiple sentences; clusters all the context features of the target polyphones to obtain several categories of context feature sets; uses each category of context feature sets to train the corresponding polyphone disambiguation subnetwork to obtain a fully trained polyphone disambiguation model. Through the above-mentioned polyphone disambiguation method, the context features of the target polyphones in each sentence are clustered, and a polyphone disambiguation subnetwork is trained separately for each category, which can improve the local plasticity of the polyphone disambiguation model and improve the model training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] in:

[0040] Figure 1 It is a schematic diagram of the model structure of the polyphone disambiguation task solution based on the pre-trained language model provided in this application;

[0041] Figure 2 It is a flow chart of an embodiment of a model training method for polyphone disambiguation provided by the present application;

[0042] Figure 3 It is a schematic diagram of the structure of the polyphone disambiguation model provided in this application;

[0043] Figure 4 It is a flowchart of an embodiment of a polyphone disambiguation method provided by the present application;

[0044] Figure 5 It is a flowchart of realizing online learning by updating the pronunciation dictionary provided by the present application;

[0045] Figure 6 yes Figure 4 The specific flow chart of step S21 of the polyphone disambiguation method shown in FIG.

[0046] Figure 7 It is a flowchart of determining online learning approaches provided by this application;

[0047] Figure 8 It is a structural schematic diagram of an embodiment of a polyphonetic character disambiguation device provided by the present application;

[0048] Fig. 9 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0051] There are many algorithms for disambiguating polyphones. The current mainstream algorithms include sequence tagging based on the BLSTM model and taking on downstream tasks based on pre-trained language models. These algorithms all belong to the classic offline supervised learning paradigm. Figure 1 The model structure of the polyphone disambiguation task solution based on a pre-trained language model is demonstrated.

[0052] The mainstream algorithm solutions for polyphone disambiguation are all offline training on pre-labeled data sets, and are used for a long time after training once. Although this classic offline supervised learning can achieve good model performance, it will encounter difficult application problems in practice:

[0053] (1) Errors are not easy to fix. No matter how well a model is trained, it is impossible to be 100% accurate. If an error is found, it is basically impossible to accurately fix the error in a black box model of deep learning.

[0054] (2) The quality of labeled data affects the quality of the algorithm. If the labeled data used to train the model is incorrect, the better the model indicators, the worse the quality of the algorithm. If you want to change this situation, you can only correct the labeled data and retrain the model, which takes a long time and is costly.

[0055] (3) The algorithm does not support online learning. That is, once the model is trained, all parameters are fixed, and new knowledge cannot be obtained from user feedback to update the model in real time. The user experience is that the algorithm has no active learning ability and is not that smart.

[0056] In order to solve the application difficulties encountered by mainstream algorithm solutions in TTS systems in practice, this application proposes an online learning method for polyphone disambiguation. The overall idea of ​​this method is: (1) offline training of a polyphone disambiguation basic model to provide services; (2) collecting user feedback during the service process and using the feedback information to update the model in real time; (3) when updating the model, try not to change the model's inference results on historical data; (4) introducing a pronunciation dictionary that can be updated online in real time to improve efficiency and accuracy.

[0057] The following introduces the online learning method for polyphone disambiguation provided by this application in two stages: model training stage and model usage stage.

[0058] Please refer to Figure 2 , Figure 2 It is a flow chart of an embodiment of a model training method for polyphone disambiguation provided in the present application.

[0059] In the model training phase, the polyphone disambiguation device trains a polyphone disambiguation basic model. Based on the solution of pre-trained language model to accept polyphone disambiguation task, offline training is performed on pre-labeled data to train a basic model. This approach is similar to the existing mainstream algorithm solution, with the following differences: (1) The model structure is specially designed to make the model have good local plasticity, see Figure 3 , Figure 3 This is a schematic diagram of the structure of the polyphone disambiguation model provided in this application; (2) the model training is divided into three stages, rather than the two stages of the existing mainstream algorithm; (3) the sub-networks used in the finished model are different.

[0060] The model training method of the present application is applied to a polyphone disambiguation device, wherein the polyphone disambiguation device of the present application can be a server, a terminal device, or a system in which a server and a terminal device cooperate with each other. Accordingly, the various parts included in the polyphone disambiguation device, such as various units, subunits, modules, and submodules, can all be set in the server, can all be set in the terminal device, or can be set in the server and the terminal device respectively.

[0061] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules used to provide distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here.

[0062] like Figure 2 As shown, the specific steps are as follows:

[0063] Step S11: extracting context features of target polyphones in multiple sentences.

[0064] In the embodiments of the present application, Figure 3 As shown, the polyphone disambiguation device trains the pre-trained language model in the pre-training stage.

[0065] Specifically, like language models such as BERT and GPT, Figure 3The pre-trained language model shown in Figure 1 learns the representation of text through self-supervision and encodes the text sequence into an implicit vector sequence α 0 ,α 1 ,...α 12 This step only requires training Figure 3 For the pre-trained language model part, you can also directly use the ready-made model trained by a third party.

[0066] The polyphone disambiguation device uses the pre-trained language model trained in the pre-training phase to extract the vector sequence α of each sentence 0 ,a 1 ,...a 12 Then, the vector sequence is input into Figure 3 The multi-head self-attention layer in the middle is the Multi-head Self-attention layer.

[0067] Specifically, in Figure 3 The first stage of training in the polyphone disambiguation model: In order to enable the model to learn the ability to disambiguate polyphones, it is necessary to add a polyphone disambiguation basic network based on the pre-trained language model. This basic network is generally simple, such as two layers of BLSTM plus one layer of Softmax. The first layer in the polyphone disambiguation model of this application must be a Multi-head Self-attention layer. This step only requires training Figure 3 Multi-head Self-attention layer and polyphone disambiguation basic network.

[0068] Furthermore, the Multi-head Self-attention layer can be trained through the polyphone disambiguation basic network, that is, the context features of the target polyphones in the training set extracted by the Multi-head Self-attention layer are input into the polyphone disambiguation basic network, and the Multi-head Self-attention layer is trained using the difference between the predicted output and the actual output of the polyphone disambiguation basic network. After the training is completed, the network parameters of the Multi-head Self-attention layer and the polyphone disambiguation basic network are fixed. In the process of training the Multi-head Self-attention layer, the network parameters of the polyphone disambiguation basic network can also be directly fixed.

[0069] Step S12: clustering all context features of the target polyphone to obtain several sets of context features.

[0070] In the embodiment of the present application, the polyphone disambiguation device uses the Multi-head Self-attention layer to extract the vector sequence a 0 ,a1 ,...α 12 The context feature, that is, the attention output of the Multi-head Self-attention layer is used as the context feature of the vector sequence.

[0071] Specifically, as Figure 3 in the input character sequence, the character "倒" is the target polyphonic character, and its position in the sequence is the 3rd (the position starts from 0). α 3 is the hidden vector representing the character after encoding, and b 3 is the self-attention output of the character. a 3 , b 3 are both vectors rather than scalars. To reduce the computational complexity of the clustering algorithm, the dimension of b 3 is set to a relatively small value, such as 64.

[0072] It should be noted that the position of the target polyphonic character needs to be specified in advance or provided synchronously.

[0073] The polyphonic character disambiguation device calculates the attention output for all target polyphonic characters in the training dataset, and then clusters these output vectors and divides them into C categories. For example, the polyphonic character disambiguation device obtains b 3 for each sentence, and then clusters all b 3 . Since the self-attention output of the target polyphonic character contains the context information of the target polyphonic character in the original sentence, through clustering, the context features with the same or similar context information can be assigned to the same category for training, which can accelerate the training efficiency of each category of polyphonic character disambiguation sub-networks.

[0074] Among them, the number of categories of C can be specified manually. The polyphonic character disambiguation device sets a polyphonic character disambiguation sub-network with the same network structure as the polyphonic character disambiguation basic network trained in the first stage of training for each category, and initializes the network parameters of the polyphonic character disambiguation sub-networks of all categories using the network parameters of the polyphonic character disambiguation basic network.

[0075] Therefore, in Figure 3 the second stage of training, the polyphonic character disambiguation device freezes or fixes the network parameters that have been trained in the pre-training stage and the first stage of training, that is, only the network parameters of the polyphonic character disambiguation sub-networks corresponding to all categories are iteratively updated in the second stage of training. Moreover, during the training of the polyphonic character disambiguation sub-networks, all training samples, that is, all sentences to be trained, are classified and only participate in the training of the polyphonic character disambiguation sub-network corresponding to their respective categories.

[0076] Step S13: Use each category of context feature set to train the corresponding polyphonic character disambiguation sub-network to obtain a trained polyphonic character disambiguation model.

[0077] In the embodiment of the present application, the polyphone disambiguation device completes Figure 3 After the three stages of model training, the final polyphone disambiguation model is obtained. Among them, unlike the existing mainstream algorithms, the polyphone disambiguation model derived after the model training does not include the polyphone disambiguation basic network, but is replaced by the polyphone disambiguation sub-network, that is, in the use stage, the polyphone disambiguation model hides the polyphone disambiguation basic network and uses the polyphone disambiguation sub-network to perform the polyphone disambiguation task.

[0078] In an embodiment of the present application, a polyphone disambiguation device extracts context features of target polyphones in multiple sentences; clusters all context features of the target polyphones to obtain several context feature sets; uses each context feature set to train a corresponding polyphone disambiguation subnetwork to obtain a fully trained polyphone disambiguation model. Through the above-mentioned polyphone disambiguation method, the context features of the target polyphones in each sentence are clustered, and a polyphone disambiguation subnetwork is trained for each category separately, which can improve the local plasticity of the polyphone disambiguation model and improve the model training effect.

[0079] The following is a model usage phase of the online learning method for polyphone disambiguation provided by this application. Please continue to refer to Figure 4 , Figure 4 It is a flowchart of an embodiment of a polyphone disambiguation method provided in the present application.

[0080] like Figure 4 As shown, the specific steps are as follows:

[0081] Step S21: inputting the sentence to be disambiguated and its target polyphone into a polyphone disambiguation module, wherein the polyphone disambiguation module includes a polyphone disambiguation model trained by the above-mentioned model training method.

[0082] In the embodiment of the present application, Figure 2 Based on the polyphone disambiguation module composed of the polyphone disambiguation model trained by the model training method, the polyphone disambiguation device also creates a word segmentation module and a pronunciation dictionary. Among them, the word segmentation module and the pronunciation dictionary are very common in modern TTS systems. The pronunciation dictionary provided in this application is editable online, while the pronunciation dictionary in some existing TTS systems is frozen during the service process. For details, please refer to Figure 5 , Figure 5 This is a flow chart of realizing online learning by updating the pronunciation dictionary provided by this application.

[0083] Specifically, the polyphone disambiguation device needs to determine the online learning path based on the segmentation results of the target polyphone. Figure 6 and Figure 7 , Figure 6 yes Figure 4Schematic diagram of the specific process of step S21 of the method for disambiguating polyphonic characters shown Figure 7 is a schematic diagram of the process for determining the online learning path provided by this application.

[0084] Such as Figure 6 shown, the specific steps are as follows:

[0085] Step S211: Use the word segmentation module to segment the sentence to be disambiguated to obtain the word segmentation result of the target polyphonic character.

[0086] In the embodiment of this application, after the polyphonic character disambiguation device receives the correct pronunciation of the target polyphonic character fed back by the user, it first determines the online learning path. Such as Figure 7 shown, if the target polyphonic character is segmented into a two-character word or a multi-character word by the word segmentation module, online learning is realized by updating the pronunciation dictionary; otherwise, online learning is realized by updating the polyphonic character disambiguation model.

[0087] Step S212: If the word segmentation result of the target polyphonic character is a single-character word, input the sentence to be disambiguated and its target polyphonic character into the polyphonic character disambiguation model in the polyphonic character disambiguation module to predict the pronunciation of the target polyphonic character.

[0088] In the embodiment of this application, when the target polyphonic character is segmented into a single-character word by the word segmentation module, the polyphonic character disambiguation device can directly realize online learning by using the polyphonic character disambiguation module.

[0089] Specifically, in an example like "The bottle fell over and the water flowed out.", the word segmentation result is "bottle / fell over / , / water / just / flowed out / ." The target polyphonic character "fell over" is segmented into a single-character word. The polyphonic character disambiguation device first obtains the predicted pronunciation of the polyphonic character disambiguation model, and then uses the true pronunciation fed back by the user to judge whether the prediction of the polyphonic character disambiguation model is correct.

[0090] If the pronunciation of the character "fell over" is predicted wrongly by the G2P module, the text sequence of this sentence is passed through the polyphonic character disambiguation basic network hidden in the polyphonic character disambiguation model to calculate its classification. Suppose it is the i-th class, then only the i-th class of polyphonic character disambiguation sub-network is retrained, while keeping the other C-1 polyphonic character disambiguation sub-networks unchanged.

[0091] Specifically, the steps for retraining the polyphonic character disambiguation sub-network provided by this application are as follows:

[0092] (1) Generate a group of similar sentences containing the target polyphonic character through the similar sentence generation model, and label the pronunciation of the target polyphonic character in them as the correct pronunciation fed back by the user.

[0093] (2) Retrain the polyphonic character disambiguation sub-network together with the sentence fed back by the user, the generated similar sentences, and the training samples involved in training the i-th class of polyphonic character disambiguation sub-network.

[0094] (3) Classification fission. When the following two situations occur, the i-th class will be split into two classes and retrained separately. i) The performance of the retrained sub-network is poor (such as accuracy less than 99%), or the samples fed back by users after retraining are still predicted incorrectly by the model; ii) There are too many data samples in this class (such as more than 32,768), which reduces the efficiency of online learning.

[0095] Among them, the classification fission method provided in this application is: taking the samples fed back by users as the first cluster center, selecting the second cluster center by minimizing the SSE (Sum of Squared Error) principle, and clustering all samples into 2 categories.

[0096] (4) Update the training sample set of the sub-network. Add the sentence of user feedback to the training sample set of its corresponding category so that it will also participate in the training during the next retraining.

[0097] Step S213: If the segmentation result of the target polyphone is a multi-character word, searching the pronunciation dictionary for the pre-stored pronunciation of the phrase where the target polyphone is located.

[0098] In the embodiment of the present application, when the target polyphone is classified into a polyphone by the word segmentation module, the polyphone disambiguation device needs to use the polyphone disambiguation module to query whether the pronunciation dictionary has a pre-stored pronunciation of the phrase in which the target polyphone is located. If yes, proceed to step S214; if no, proceed to step S215.

[0099] Step S214: If there is a pre-stored pronunciation of the phrase where the target polyphone is located in the pronunciation dictionary, the pre-stored pronunciation is used as the output of the polyphone disambiguation module.

[0100] Step S215: If the pre-stored pronunciation of the phrase in which the target polyphone is located does not exist in the pronunciation dictionary, the sentence to be disambiguated and its target polyphone are input into the polyphone disambiguation model in the polyphone disambiguation module to predict the pronunciation of the target polyphone, and the output of the polyphone disambiguation module is used to update the pronunciation dictionary.

[0101] In the embodiment of the present application, if the pronunciation dictionary does not contain the pre-stored pronunciation of the phrase in which the target polyphone is located, the polyphone disambiguation device needs to implement online learning by updating the pronunciation dictionary.

[0102] Specifically, considering that the pronunciation of most polyphonic characters in two-character words and multi-character words is fixed, the effect of online learning can be achieved by updating the pronunciation dictionary in real time. Figure 5The following example is used for illustration. The input text is "Liuan Guapian is tea, not melon". Before the pronunciation dictionary was updated, the G2P module mislabeled the character "六" as "liu4". Since the character "六" was segmented into the two-character word "六安" by the word segmentation module, the correct pronunciation of "六" provided by the user feedback and the pronunciation of the character "安" given by the G2P module were combined to update "六安:lu4 an1" into the pronunciation dictionary. After the update, whenever the G2P module encounters the word "六安" again, it will obtain the correct pronunciation "lu4 an1" from the pronunciation dictionary.

[0103] Step S22: Obtain the pronunciation of the target polyphonic character output by the polyphonic character disambiguation module.

[0104] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not impose any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0105] In the embodiment of the present application, in order to make the model have good local plasticity, the attention output of the target polyphonic character is clustered, and a polyphonic character disambiguation sub-network is trained separately for each category; in order to ensure the stability of the classification result, when training each polyphonic character disambiguation sub-network separately, the weights of the Multi-head Self-attention layer, which is the first layer of the polyphonic character disambiguation task model, are frozen; in order to ensure the performance of each classified polyphonic character disambiguation sub-network, a high-performance polyphonic character disambiguation sub-network is trained using all the training set data in the first training stage, and its weights are used to initialize each classified polyphonic character disambiguation sub-network; classification fission is introduced to solve the problem that the user feedback samples still cannot be correctly predicted after retraining the polyphonic character disambiguation sub-network, and the problem of reducing the online learning efficiency as the training samples are expanded; the online learning path is judged according to the word segmentation result, and by reasonably using the word segmentation module and the online editable pronunciation dictionary, the effect of efficient online learning is achieved.

[0106] To implement the above model training method and / or polyphonic character disambiguation method, the present application also proposes a polyphonic character disambiguation device. For details, please refer to Figure 8 , Figure 8 is a schematic structural diagram of an embodiment of the polyphonic character disambiguation device provided by the present application.

[0107] The polyphonic character disambiguation device 400 in this embodiment includes a processor 41, a memory 42, an input / output device 43, and a bus 44.

[0108] The processor 41, memory 42, and input / output device 43 are respectively connected to a bus 44. The memory 42 stores program data. The processor 41 is used to execute the program data to implement the model training method and / or the polyphonetic character disambiguation method described in the above embodiments.

[0109] In the embodiment of the present application, the processor 41 may also be referred to as a CPU (Central Processing Unit). The processor 41 may be an integrated circuit chip having the ability to process signals. The processor 41 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 41 may also be any conventional processor, etc.

[0110] This application also provides a computer storage medium, please continue to refer to Fig. 9 , Fig. 9 It is a structural diagram of an embodiment of a computer storage medium provided in the present application. The computer storage medium 600 stores a computer program 61. When the computer program 61 is executed by a processor, it is used to implement the model training method and / or the polyphone disambiguation method of the above-mentioned embodiment.

[0111] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0112] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A model training method for polyphone disambiguation, characterized in that: The model training method comprises: Extracting contextual features of target polyphones in multiple sentences; Clustering all context features of the target polyphone to obtain several sets of context features; Each type of context feature set is used to train the corresponding polyphone disambiguation subnetwork to obtain a fully trained polyphone disambiguation model.

2. The model training method according to claim 1, characterized in that: The extracting context features of target polyphones in multiple sentences includes: Use the pre-trained language model to extract the vector sequence of each sentence; The vector sequence is input into a multi-head self-attention layer to extract context features of the target polyphone in each sentence.

3. The model training method according to claim 2, characterized in that: The method of using each type of context feature set to train a corresponding polyphone disambiguation subnetwork includes: Initialize the network parameters of the polyphone disambiguation sub-network according to the network parameters of the polyphone disambiguation basic network; The network parameters of the pre-trained language model and / or the network parameters of the polyphone disambiguation basic network are fixed, and the corresponding polyphone disambiguation sub-network is trained using each type of context feature set.

4. The model training method according to claim 3, characterized in that: The model training method further includes: Inputting the context features of the target polyphone in each sentence into the polyphone disambiguation basic network to obtain the predicted pronunciation of the target polyphone; The multi-head self-attention layer is trained using the predicted pronunciation.

5. A method for disambiguating polyphones, characterized in that: The polyphone disambiguation method comprises: Inputting the sentence to be disambiguated and its target polyphone into a polyphone disambiguation module, wherein the polyphone disambiguation module includes a polyphone disambiguation model trained by the model training method according to any one of claims 1 to 4; The pronunciation of the target polyphonetic character output by the polyphonetic character disambiguation module is obtained.

6. The polyphone disambiguation method according to claim 5, characterized in that: The step of inputting the sentence to be disambiguated and the target polyphonetic character thereof into the polyphonetic character disambiguation module comprises: Using a word segmentation module to segment the sentence to be disambiguated, and obtaining a word segmentation result of the target polyphone; If the segmentation result of the target polyphone is a single word, the sentence to be disambiguated and the target polyphone and the target polyphone are input into the polyphone disambiguation model in the polyphone disambiguation module to predict the pronunciation of the target polyphone.

7. The polyphone disambiguation method according to claim 6, characterized in that: After obtaining the pronunciation of the target polyphonetic character output by the polyphonetic character disambiguation module, the polyphonetic character disambiguation method further includes: When the predicted pronunciation of the target polyphonetic character output by the polyphonetic character disambiguation module is different from the actual pronunciation, obtaining the classification of the target polyphonetic character using the polyphonetic character disambiguation basic network in the polyphonetic character disambiguation model; The real pronunciation is used to train the polyphone disambiguation subnetwork corresponding to the classification.

8. The polyphone disambiguation method according to claim 7, characterized in that: The using the real pronunciation to train the polyphone disambiguation subnetwork corresponding to the classification includes: Obtaining the number of data samples of the classification; When the number of the data samples exceeds a preset threshold, reclassifying all the data samples of the classification using the context features of the target polyphone of the sentence to be disambiguated as cluster centers to obtain two new classifications; The real pronunciation is used to train the newly classified polyphone disambiguation subnetwork where the context features of the target polyphone of the sentence to be disambiguated are located.

9. The polyphone disambiguation method according to claim 6, characterized in that: After the word segmentation module is used to segment the sentence to be disambiguated to obtain the word segmentation result of the target polyphone, the polyphone disambiguation method further includes: If the segmentation result of the target polyphonetic character is a multi-character word, searching the pronunciation dictionary for the pre-stored pronunciation of the phrase where the target polyphonetic character is located; If the pronunciation dictionary contains a pre-stored pronunciation of the phrase in which the target polyphone is located, the pre-stored pronunciation is used as the output of the polyphone disambiguation module; If the pre-stored pronunciation of the phrase in which the target polyphone is located does not exist in the pronunciation dictionary, the sentence to be disambiguated and its target polyphone are input into the polyphone disambiguation model in the polyphone disambiguation module to predict the pronunciation of the target polyphone, and the output of the polyphone disambiguation module is used to update the pronunciation dictionary.

10. A polyphonetic character disambiguation device, characterized in that: The polyphone disambiguation device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the model training method as described in any one of claims 1 to 4, and / or the polyphone disambiguation method as described in any one of claims 5 to 9.

11. A computer storage medium, characterized in that: The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the model training method as described in any one of claims 1 to 4, and / or the polyphone disambiguation method as described in any one of claims 5 to 9.

Images