A multi-syllable word disambiguation method and device, medium

Abstract

The application discloses a multi-sound character disambiguation method and device, and a medium. The method comprises the following steps: predicting pronunciation of each multi-sound character in an initial text sequence by using a first pronunciation prediction model to obtain a first pronunciation result of each multi-sound character under any pronunciation; the initial text sequence has at least one multi-sound character with different pronunciations; performing pronunciation statistics on each multi-sound character in the initial text sequence by using a second pronunciation prediction model and / or a third pronunciation prediction model to obtain a second pronunciation result and / or a third pronunciation result of each multi-sound character under any pronunciation; the second pronunciation prediction model is used for predicting pronunciation of the multi-sound character, and the third pronunciation prediction model is used for correcting pronunciation of the multi-sound character; and fusing the second pronunciation result and the third pronunciation result with the first pronunciation result to obtain a final pronunciation of each multi-sound character, and determining a predicted pronunciation sequence corresponding to the initial text sequence. In this way, the accuracy of multi-sound character pronunciation prediction can be improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing, and in particular to a method, apparatus, and medium for disambiguating polyphonic characters. Background Technology

[0002] Currently, character-to-speech conversion is an essential component of Chinese speech synthesis systems, and its accuracy directly affects the intelligibility of the synthesized speech. Most Chinese characters have definite pronunciations, which can be obtained by searching a dictionary. However, many Chinese characters have two or more pronunciations. The key and challenge of character-to-speech conversion lies in the identification and disambiguation of these polyphonic characters.

[0003] It is generally believed that the pronunciation of polyphonic characters is closely related to specific contextual information, semantics, and language habits. Existing technologies usually use pre-trained models to predict the pronunciation of polyphonic characters, but this method has low prediction accuracy. Summary of the Invention

[0004] This application provides a method, apparatus, and medium for disambiguating polyphonic characters.

[0005] This application is achieved through the following technical solution.

[0006] The first aspect of this application provides a method for disambiguating polyphonic characters, the method comprising: using a first pronunciation prediction model to predict the pronunciation of each polyphonic character in an initial text sequence, obtaining a first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence contains at least one polyphonic character with different pronunciations; using a second pronunciation prediction model and / or a third pronunciation prediction model to perform pronunciation statistics on each polyphonic character in the initial text sequence, obtaining a second pronunciation result and / or a third pronunciation result of each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of the polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of the polyphonic characters; at least one of the second pronunciation result and the third pronunciation result is fused with the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining a predicted pronunciation sequence corresponding to the initial text sequence.

[0007] Based on this, the technical solution of this application uses at least two different models to predict the pronunciation of polyphonic characters in the initial text sequence, obtains the pronunciation results under at least two prediction methods, and then combines the pronunciation results under at least two prediction methods to determine the final pronunciation of each polyphonic character. In this way, this application can determine a more accurate pronunciation of polyphonic characters based on at least two different model prediction methods, thereby improving the effect of polyphonic character pronunciation prediction.

[0008] In some embodiments of this application, at least one of the second and third pronunciation results is fused with the first pronunciation result to obtain the final pronunciation of each polyphonic character. This includes: fusing at least one of the second and third pronunciation results with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result of any pronunciation of each polyphonic character, and determining the pronunciation corresponding to the highest fusion result in each polyphonic character as the final pronunciation of that polyphonic character.

[0009] Based on this, this application can determine the pronunciation corresponding to the highest fusion result among the first pronunciation result and at least one of the second and third pronunciation results as the final pronunciation, thus predicting the most accurate pronunciation for polyphonic characters.

[0010] In some embodiments of this application, at least one of the second pronunciation results and the third pronunciation results is fused with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result of any pronunciation of each polyphonic character, including: determining a first weight assigned to each of the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model; and based on the respective first weights, fusing at least one of the second pronunciation results and the third pronunciation results with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result of any pronunciation of each polyphonic character.

[0011] Based on this, in this application, a corresponding weight can be set for each model based on the prediction performance of each model. This weight can be understood as the proportion of the pronunciation result predicted by the corresponding model in the final pronunciation result. Based on this, the final pronunciation result can comprehensively consider the prediction performance of each model, and the weight of each model can be dynamically adjusted according to the prediction accuracy of the model, thereby improving the prediction effect of the model.

[0012] In some embodiments of this application, the method further includes: determining non-polyphonic characters in the initial text sequence, excluding at least one polyphonic character; searching for the corresponding pronunciation of the non-polyphonic character from a preset non-polyphonic character mapping dictionary; and determining a predicted pronunciation sequence corresponding to the initial text sequence, including: generating a predicted pronunciation sequence corresponding to the initial text according to the text order of the initial text sequence based on the final pronunciation of each polyphonic character and the pronunciation of the non-polyphonic characters.

[0013] Based on this, this application, by setting a preset non-polyphonic character mapping dictionary, can determine the pronunciation of each non-polyphonic character in the initial text while predicting polyphonic characters, and then generate a predicted pronunciation sequence of the initial text based on the pronunciation of polyphonic characters and the pronunciation of non-polyphonic characters.

[0014] In some embodiments of this application, the method further includes: when deployed on a first device, determining the idle resources allocated in the first device for deploying the model; when the idle resources are greater than a first threshold, simultaneously deploying the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model to the first device; when the idle resources are not greater than the first threshold, deploying the first pronunciation prediction model and the third pronunciation prediction model to the first device; wherein the memory size required by the second pronunciation prediction model is greater than the memory size required by the third pronunciation prediction model.

[0015] Based on this, this application can take into account the idle resources of the first device when deploying the model to the first device, and dynamically adjust the deployment method of the model according to the idle resources.

[0016] In some embodiments of this application, the method further includes: receiving an initial text sequence; determining the semantics corresponding to the initial text sequence based on the predicted pronunciation sequence if a predicted pronunciation sequence corresponding to the initial text sequence is determined; and performing a first operation corresponding to the initial text sequence based on the semantics.

[0017] In some embodiments of this application, the method further includes: obtaining polyphonic character cases that were predicted incorrectly in the history of the first pronunciation prediction model; generating an error case set based on the polyphonic character cases and the first correct pronunciation of the polyphonic character cases; and training the third initial pronunciation prediction model using the error case set to obtain the third pronunciation prediction model.

[0018] In some embodiments of this application, the method further includes: in the case of a pronunciation prediction error for at least one polyphonic character in the predicted pronunciation sequence and / or the first prediction sequence predicted by the first pronunciation prediction model, obtaining a second correct pronunciation corresponding to the first polyphonic character; updating the predicted pronunciation sequence or the first prediction sequence based on the second correct pronunciation to obtain a latest prediction sequence; generating new error cases based on the latest prediction sequence and the initial text sequence; adding the new error cases to the error case set to obtain an updated error case set; and iteratively updating the third pronunciation prediction model using the updated error case set.

[0019] Based on this, this application can record the erroneous cases when pronunciation prediction is wrong, and update the error case set based on the recorded error cases. This allows for continuous training of the third pronunciation prediction model based on the updated error case set, which can gradually improve the prediction performance of the third pronunciation prediction model.

[0020] In some embodiments of this application, the method further includes: when the error case set is continuously updated and the number of error cases in the updated error case set reaches a second threshold, retraining the first pronunciation prediction model and the second pronunciation prediction model using the updated error case set and the training sample set for the next pronunciation prediction of a new text sequence; the training sample set is the sample set used when training the first pronunciation prediction model.

[0021] Based on this, this application can retrain all the above models when there are many incorrect prediction cases, thereby further improving the accuracy of model prediction.

[0022] A second aspect of this application provides a disambiguation device for polyphonic characters. The device includes: a prediction unit, configured to predict the pronunciation of each polyphonic character in an initial text sequence using a first pronunciation prediction model to obtain a first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence contains at least one polyphonic character with different pronunciations; the prediction unit is further configured to perform pronunciation statistics on each polyphonic character in the initial text sequence using a second pronunciation prediction model and / or a third pronunciation prediction model to obtain a second pronunciation result and / or a third pronunciation result of each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of the polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of the polyphonic characters; and a fusion unit, configured to fuse at least one of the second and third pronunciation results with the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining a predicted pronunciation sequence corresponding to the initial text sequence.

[0023] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above-described methods for disambiguating polyphonic characters.

[0024] The technical solution proposed in this application uses at least two different models to predict the pronunciation of polyphonic characters in the initial text sequence, obtaining pronunciation results under at least two prediction methods. Then, the pronunciation results under at least two prediction methods are combined to determine the final pronunciation of each polyphonic character. In this way, this application can determine a more accurate pronunciation of polyphonic characters based on at least two different model prediction methods, thereby improving the effect of polyphonic character pronunciation prediction. Attached Figure Description

[0025] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0026] Figure 1 A flowchart illustrating a method for disambiguating polyphonic characters provided in an embodiment of this application;

[0027] Figure 2 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 1 ;

[0028] Figure 3 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 2 ;

[0029] Figure 4 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 3 ;

[0030] Figure 5 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 4 ;

[0031] Figure 6 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 5 ;

[0032] Figure 7 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 6 ;

[0033] Figure 8 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 7 ;

[0034] Figure 9 A flowchart illustrating an exemplary method for disambiguating polyphonic characters provided in this application embodiment. Figure 8 ;

[0035] Figure 10 A schematic diagram of the overall process of an exemplary method for disambiguating polyphonic characters provided in this application embodiment;

[0036] Figure 11 A schematic diagram illustrating the calculation of the final probability of a polyphonic character under any pronunciation, provided for an embodiment of this application;

[0037] Figure 12 An exemplary decoding flowchart for disambiguation of polyphonic characters is provided in this application embodiment;

[0038] Figure 13 A schematic diagram of the composition structure of a disambiguation device for polyphonic characters provided in an embodiment of this application;

[0039] Figure 14 This is a schematic diagram of the hardware entity of a first device provided in an embodiment of this application. Detailed Implementation

[0040] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application; the terms “comprising” and “having”, and any variations thereof, in the specification and the foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0042] In the description of the embodiments of this application, technical terms such as "first," "second," and "third" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0043] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0044] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship.

[0045] In the description of the embodiments of this application, the technical terms "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed, operated or used in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0046] In the description of the embodiments of the present application, unless otherwise clearly specified or limited, technical terms such as "installation", "connection", "connection", "fixation", etc. shall be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can also be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and can be the connection inside two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific situations.

[0047] In the description of the embodiments of the present application, unless otherwise clearly specified or limited, the technical term "contact" shall be understood in a broad sense. It can be direct contact, or contact隔着中间媒介层的接触 (should be 'contact through an intermediate medium layer' here, seems there is a typo in the original), it can be contact where there is basically no mutual force between the two in contact, or contact where there is a mutual force between the two in contact.

[0048] Next, the present application will be described in detail.

[0049] The present application proposes a method for disambiguating polyphonic characters, which is applied to a device for disambiguating polyphonic characters. Refer to Figure 1 , and the specific method includes the following steps S101 to step S103:

[0050] Step S101: Use the first pronunciation prediction model to predict the pronunciation of each polyphonic character in the initial text sequence, and obtain the first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence includes at least one polyphonic character with different pronunciations.

[0051] In the embodiments of the present application, the device for disambiguating polyphonic characters uses the first pronunciation prediction model to predict the pronunciation of each polyphonic character in the initial text sequence, and obtains the first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence includes at least one polyphonic character with different pronunciations.

[0052] In some embodiments, the initial text sequence can be a sentence, a paragraph of text, or an article. At the same time, a sentence, a paragraph of text, or an article includes at least one polyphonic character with different pronunciations; the specific initial text sequence can be determined according to the actual situation, and the embodiments of the present application do not make specific limitations here.

[0053] Exemplarily, the initial text sequence can be "Xiaoming is reading a book in the classroom. He likes to read while gently pointing at the words with his finger and counting the number of pages". It can be seen that this sentence includes the polyphonic characters "教 (should be 'jiao' or 'teach' in English here)" and "数 (should be'shu' or 'count' in English here)".

[0054] It should be noted that there seems to be some inaccuracies or typos in the original Chinese text, especially in the part of "隔着中间媒介层的接触", which is translated as best as possible based on the context. Also, the handling of some Chinese characters representing polyphonic characters might need to be adjusted according to more accurate English translations of the relevant words.In some embodiments, assuming the current situation is a smart interaction scenario, the smart device determines the corresponding answer based on certain instructions given by the user and needs to transmit the answer to the user via voice. In this case, the answer is the initial text sequence. Alternatively, the smart device needs to perform corresponding operations based on instructions input by the user, where the instructions are the initial text sequence. Or, assuming the current situation is a recitation scenario, the user clicks the recitation button to expect certain text to be played out by voice, where the text is the initial text sequence. The specific application scenario and the initial text sequence can be determined according to the actual situation, and this application embodiment does not impose specific limitations here.

[0055] In some embodiments, the disambiguation device for polyphonic characters can pre-count all polyphonic characters in Mandarin Chinese and their corresponding at least two pronunciations to obtain a preset polyphonic character mapping dictionary. That is, the preset polyphonic character mapping dictionary contains mapping relationships between polyphonic characters and at least two pronunciations. Similarly, it can also count all non-polyphonic characters in Mandarin Chinese based on their corresponding at least two pronunciations to obtain a preset non-polyphonic character mapping dictionary. That is, the preset non-polyphonic character mapping dictionary contains a one-to-one mapping relationship between non-polyphonic characters and a pronunciation. The specific preset polyphonic character mapping dictionary and preset non-polyphonic character mapping dictionary can be generated according to the actual situation, and this application embodiment does not make specific limitations here.

[0056] In some embodiments, when filtering polyphonic characters in an initial text sequence, the polyphonic character disambiguation device can use the aforementioned preset polyphonic character mapping dictionary to determine all characters appearing in the preset polyphonic character mapping dictionary in the initial text sequence as polyphonic characters, thereby obtaining at least one polyphonic character. Similarly, it can also use the aforementioned preset non-polyphonic character mapping dictionary to determine the characters appearing in the preset non-polyphonic character mapping dictionary in the initial text sequence as non-polyphonic characters, and then determine the characters in the initial text sequence other than the non-polyphonic characters as polyphonic characters, thereby obtaining at least one polyphonic character. The specific method for filtering polyphonic characters can be selected according to the actual situation, and this application embodiment does not make specific limitations here.

[0057] In some embodiments, since the preset polyphonic character mapping dictionary stores all polyphonic characters and all the pronunciations corresponding to each polyphonic character, after determining the polyphonic characters in the initial text sequence, at least two pronunciations of each polyphonic character can be found from the preset polyphonic character mapping dictionary.

[0058] For example, suppose the initial text is "Xiaoming is reading a book in the classroom. He likes to lightly tap the words with his finger and count the page numbers while reading." In this sentence, the "shu" in "jiao" and "shuzhe" are both polyphonic characters. The pronunciation of each polyphonic character can be determined. For example, the pronunciation of "jiao" can be "jiao1" and "jiao4", and the pronunciation of "shu" can be "shu3" and "shu4".

[0059] In some embodiments, when the disambiguation device for polyphonic characters uses a first pronunciation prediction model to predict the pronunciation of each polyphonic character in the initial text sequence, it first needs to map each Chinese character in the initial text sequence into a character embedding vector, and then input the multiple character embedding vectors corresponding to the initial text sequence into a neural network model. The neural network model is then used to map these multiple character embedding vectors into a high-dimensional vector. It should be noted that after mapping to obtain the high-dimensional vector, it is necessary to select the high-dimensional vectors of polyphonic characters in the initial text. For example, the neural network model can be a pre-trained deep learning model (Bidirectional Encoder Representations from Transformers, BERT). The specific method for extracting the high-dimensional vectors of polyphonic characters can be selected according to the actual situation, and this application embodiment does not make specific limitations here.

[0060] In some embodiments, after obtaining at least two pronunciations of each polyphonic character and a high-dimensional vector of each polyphonic character, the disambiguation device for polyphonic characters can input the high-dimensional vector of each polyphonic character and the corresponding at least two pronunciations into a first pronunciation prediction model, and use the first pronunciation prediction model to predict the pronunciation result of each polyphonic character under each pronunciation in the context of the initial text, that is, to obtain the first pronunciation result of each polyphonic character under any pronunciation.

[0061] For example, the pronunciation result can be understood as the pronunciation probability or the pronunciation score; the first pronunciation prediction model can be a linear model LINEAR, which can predict the pronunciation probability of each polyphonic character under any pronunciation when considering the context; the specific pronunciation result and the first pronunciation prediction model can be selected according to the actual situation, and the embodiments of this application do not make specific limitations here.

[0062] Step S102: Using the second pronunciation prediction model and / or the third pronunciation prediction model, perform pronunciation statistics on each polyphonic character in the initial text sequence to obtain the second pronunciation result and / or the third pronunciation result of each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of polyphonic characters.

[0063] In an embodiment of the present application, the disambiguation device for polyphonic characters uses the second pronunciation prediction model and / or the third pronunciation prediction model to perform pronunciation statistics on each polyphonic character in the initial text sequence, and obtains the second pronunciation result and / or the third pronunciation result of each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of polyphonic characters.

[0064] In some embodiments, the disambiguation device for polyphonic characters may only use the second pronunciation prediction model to perform pronunciation statistics on each polyphonic character in the initial text sequence, and obtain the second pronunciation result of each polyphonic character under any pronunciation; that is, the initial text sequence is input into the second pronunciation prediction model. The second pronunciation prediction model first screens out the polyphonic characters in the initial text sequence, and then matches the training sample set for each polyphonic character based on the context of the polyphonic character in the initial text sequence. Since the training sample set includes dozens of gigabytes of training samples, it is possible to match each polyphonic character in the initial text scenario based on the sample set. At this time, in a text sequence similar to the initial text sequence, the pronunciation corresponding to the polyphonic character can be counted. Thus, the matching number of each polyphonic character under any pronunciation is obtained, and finally, the pronunciation result of each polyphonic character under any pronunciation, that is, the second pronunciation result, is obtained based on the matching number.

[0065] Exemplarily, assume that the initial text sequence is "Xiaoming is reading a book in the classroom. He likes to read while gently pointing at the words with his finger and counting the number of pages". At this time, the initial text sequence is input into the second pronunciation prediction model. The second pronunciation prediction model matches the words "jiao" and "shu" in this sentence from the training set based on the context of the initial text sequence. Assume that in the current context, the number of hits for the pronunciation "jiao1" of the word "jiao" is 20, and the number of hits for the pronunciation "jiao4" is 80. At this time, the second pronunciation result of the polyphonic character "jiao" under the pronunciation "jiao1" is 20%, and the second pronunciation result under the pronunciation "jiao4" is 80%. Similarly, assume that in the current context, the number of hits for the pronunciation "shu3" of the word "shu" is 60, and the number of hits for the pronunciation "shu4" is 40. At this time, the second pronunciation result of the polyphonic character "shu" under the pronunciation "shu3" is 60%, and the second pronunciation result under the pronunciation "shu4" is 40%. The specific prediction method of the second pronunciation prediction model can be determined according to the actual situation, and the embodiments of the present application do not make specific limitations here.

[0066] In other embodiments, the disambiguation device for polyphonic characters may also utilize only the third pronunciation prediction model to perform pronunciation statistics on each polyphonic character in the initial text sequence, obtaining the third pronunciation result of each polyphonic character under any pronunciation. That is, the initial text sequence is input into the third pronunciation prediction model, which first filters out the polyphonic characters in the initial text sequence, and then performs error case matching on each polyphonic character based on the context of the polyphonic character in the initial text sequence. Since the error case set includes the error case set generated during the deployment of the model, each polyphonic character in the initial text sequence scenario can be matched based on the error case set. At this time, the pronunciation corresponding to the polyphonic character in texts similar to the initial text sequence in the error case set can be counted. Thus, the number of matches for each polyphonic character under any pronunciation is obtained, and finally, the pronunciation result of each polyphonic character under any pronunciation is obtained based on the number of matches, i.e., the third pronunciation result.

[0067] In other embodiments, the disambiguation device for polyphonic characters can also simultaneously use the second pronunciation prediction model and the third pronunciation prediction model to perform pronunciation statistics on each polyphonic character in the initial text sequence, and obtain the second pronunciation result and the third pronunciation result of each polyphonic character under any pronunciation, which is the same as the above steps and will not be repeated.

[0068] Step S103: At least one of the second and third pronunciation results is fused with the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining the predicted pronunciation sequence corresponding to the initial text sequence.

[0069] In the embodiments of this application, after obtaining the first pronunciation result, the second pronunciation result, and / or the third pronunciation result of each polyphonic character under any pronunciation, the disambiguation device for polyphonic characters performs pronunciation fusion with at least one of the second and third pronunciation results and the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining the predicted pronunciation sequence corresponding to the initial text sequence.

[0070] In some embodiments, the disambiguation device for polyphonic characters can fuse the second pronunciation result with the first pronunciation result to obtain the final pronunciation of each polyphonic character, fuse the third pronunciation result with the first pronunciation result to obtain the final pronunciation of each polyphonic character, or fuse the second pronunciation result, the third pronunciation result, and the first pronunciation result to obtain the final pronunciation of each polyphonic character. The specific pronunciation fusion method can be determined according to the actual situation, and the embodiments of this application do not make specific limitations here.

[0071] In some embodiments, after obtaining the final pronunciation of each polyphonic character, the disambiguation device for polyphonic characters also needs to obtain the pronunciation of each non-polyphonic character in the initial text sequence, and determine the predicted pronunciation sequence corresponding to the initial text sequence based on the final pronunciation of each polyphonic character and the pronunciation of each non-polyphonic character.

[0072] In some embodiments of this application, the disambiguation device for polyphonic characters performs phonetic fusion between at least one of the second and third pronunciation results and the first pronunciation result to obtain the final pronunciation of each polyphonic character. (Refer to...) Figure 2 This includes the following step S201:

[0073] Step S201: At least one of the second pronunciation result and the third pronunciation result is fused with the same pronunciation in each polyphonic character in the first pronunciation result to obtain the fusion result of any pronunciation of each polyphonic character, and the pronunciation corresponding to the highest fusion result in each polyphonic character is determined as the final pronunciation of the polyphonic character.

[0074] In some embodiments, the disambiguation device for polyphonic characters fuses the second pronunciation result with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a first fusion result for any pronunciation of each polyphonic character, and determines the pronunciation corresponding to the highest first fusion result in each polyphonic character as the final pronunciation.

[0075] In other embodiments, the disambiguation device for polyphonic characters merges the third pronunciation result with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a second fusion result for any pronunciation of each polyphonic character, and determines the pronunciation corresponding to the highest second fusion result in each polyphonic character as the final pronunciation.

[0076] In other embodiments, the disambiguation device for polyphonic characters fuses the second pronunciation result, the third pronunciation result, and the same pronunciation in each polyphonic character in the first pronunciation result to obtain a third fusion result for any pronunciation of each polyphonic character, and determines the pronunciation corresponding to the highest third fusion result in each polyphonic character as the final pronunciation.

[0077] In some embodiments of this application, the disambiguation device for polyphonic characters fuses at least one of the second and third pronunciation results with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result for any pronunciation of each polyphonic character. (Refer to...) Figure 3 This includes the following steps S301 to S302:

[0078] Step S301: Determine the first weight assigned to each of the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model.

[0079] In some embodiments, the disambiguation device for polyphonic characters can select to configure weights for any of the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model. It can select to configure weights for one of the models, two of the models, or all of the models. The specific weight configuration method can be selected according to the actual situation, and this application embodiment does not make specific limitations.

[0080] Step S302: Based on their respective first weights, at least one of the second pronunciation results and the third pronunciation results is fused with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result of any pronunciation of each polyphonic character.

[0081] In some embodiments, the pronunciation result to be fused is determined, and it is determined whether to assign weights to the model corresponding to the pronunciation result. Pronunciation fusion is performed based on the pronunciation result and the corresponding weights to obtain the fusion result.

[0082] For example, suppose that the same pronunciation in each polyphonic character in the first pronunciation result, second pronunciation result and third pronunciation result are fused, and a first weight is set for the first pronunciation prediction model, a second weight is set for the second pronunciation prediction model and a third weight is set for the third pronunciation prediction model. Based on this, the first pronunciation result, the first weight, the second pronunciation result, the second weight and the third pronunciation result and the third weight can be fused to obtain the fusion result of any pronunciation of each polyphonic character. Finally, the pronunciation corresponding to the highest fusion result in each polyphonic character is determined as the final pronunciation.

[0083] In some embodiments of this application, reference is made to Figure 4 The disambiguation device for polyphonic characters can also perform the following steps S401 to S403:

[0084] Step S401: Determine the non-polyphonic characters in the initial text sequence, excluding at least one polyphonic character.

[0085] In some embodiments, the disambiguation device for polyphonic characters needs to determine the pronunciation of each polyphonic character in the initial text sequence while simultaneously determining the pronunciation of each non-polyphonic character in the initial text sequence. Based on this, the non-polyphonic characters in the initial text sequence, excluding at least one polyphonic character, are first determined.

[0086] Step S402: Find the corresponding pronunciation of the non-polyphonic character from the preset non-polyphonic character mapping dictionary.

[0087] In some embodiments, since a preset non-polyphonic character mapping dictionary is obtained by pre-statistically counting all non-polyphonic characters in Mandarin Chinese based on at least two corresponding pronunciations, the polyphonic character disambiguation device can obtain the pronunciation corresponding to the non-polyphonic character based on the one-to-one mapping of the non-polyphonic characters in the preset non-polyphonic character mapping dictionary and the corresponding pronunciation.

[0088] Step S403: Based on the final pronunciation of each polyphonic character and the pronunciation of non-polyphonic characters, generate the predicted pronunciation sequence corresponding to the initial text according to the text order of the initial text sequence.

[0089] In some embodiments, the disambiguation device for polyphonic characters can generate a predicted pronunciation sequence for all characters in the initial text sequence based on the final pronunciation of each polyphonic character in the initial text sequence predicted by the first pronunciation prediction model, the second pronunciation prediction model and / or the third pronunciation prediction model, and the pronunciation of each non-polyphonic character in the initial text sequence found from a preset non-polyphonic character mapping dictionary.

[0090] In some embodiments of this application, reference is made to Figure 5 The disambiguation device for polyphonic characters can also perform the following steps S501 to S503:

[0091] Step S501: In the case of deployment on the first device, determine the idle resources allocated to the deployment model in the first device.

[0092] In some embodiments, the disambiguation device for polyphonic characters is deployed on a first device, which may be a robotic device or a cloud server, and the deployment method of the model can be selected according to the first device.

[0093] Step S502: When the idle resources are greater than the first threshold, deploy the first pronunciation prediction model, the second pronunciation prediction model and the third pronunciation prediction model to the first device simultaneously.

[0094] In some embodiments, since the second pronunciation prediction model includes a training sample set of tens of gigabytes, the second pronunciation prediction model has a large memory requirement. Assuming the first device is a cloud server, since cloud servers usually allocate a large amount of idle resources for deploying models, the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model can be deployed on the first device simultaneously.

[0095] Step S503: If the idle resources are not greater than the first threshold, deploy the first pronunciation prediction model and the third pronunciation prediction model to the first device.

[0096] In this embodiment of the application, the memory required by the second pronunciation prediction model is greater than the memory required by the third pronunciation prediction model.

[0097] In some embodiments, assuming the first device is a robotic device, since the robotic device has limited memory, the available resources allocated for deploying the model are usually small, and it is possible to deploy only the first pronunciation prediction model and the third pronunciation prediction model to the first device.

[0098] In some embodiments of this application, reference is made to Figure 6 The disambiguation device for polyphonic characters can also perform the following steps S601 to S603:

[0099] Step S601: Receive the initial text sequence.

[0100] In some embodiments, assuming that a disambiguation device for polyphonic characters is deployed on a robot device, the user inputs a first instruction into the robot device, which is an initial text sequence, for example, the first instruction is "play the song 'Bicycle'".

[0101] Step S602: If the predicted pronunciation sequence corresponding to the initial text sequence is determined, the semantics corresponding to the initial text sequence is determined based on the predicted pronunciation sequence.

[0102] In some embodiments, after determining the predicted pronunciation sequence corresponding to the initial text sequence, the robotic device can understand the semantics of the initial text sequence based on the predicted pronunciation sequence, thereby enabling it to perform corresponding operations based on the semantics.

[0103] Step S603: Perform the first operation corresponding to the initial text sequence based on semantics.

[0104] In some embodiments, the robot device performs a first operation corresponding to an initial text sequence based on semantics. For example, if the first instruction is "play the song 'Bicycle'", the robot device will play "Bicycle" to the user after understanding the meaning of the sentence.

[0105] In some embodiments of this application, reference is made to Figure 7 The disambiguation device for polyphonic characters can also perform the following steps S701 to S702:

[0106] Step S701: Obtain the polyphonic character cases that were predicted incorrectly in the history of the first pronunciation prediction model, and generate an error case set based on the polyphonic character cases and the first correct pronunciation of the polyphonic character cases.

[0107] In some embodiments, the disambiguation device for polyphonic characters can save the polyphonic characters that the first pronunciation prediction model predicted incorrectly during actual deployment or historical prediction. It can choose to save single polyphonic characters or the text sequence to which the polyphonic character belongs at the same time. Then, based on the first correct pronunciation of the polyphonic character and the polyphonic character / text sequence, an error case is generated, and finally an error case set is obtained.

[0108] Step S702: Train the third initial pronunciation prediction model using the error case set to obtain the third pronunciation prediction model.

[0109] In some embodiments, after obtaining the set of incorrect cases, the disambiguation device for polyphonic characters can use the set of incorrect cases to train the third initial pronunciation prediction model to obtain the third pronunciation prediction model.

[0110] In some embodiments of this application, reference is made to Figure 8 The disambiguation device for polyphonic characters can also perform the following steps S801 to S804:

[0111] Step S801: If the pronunciation prediction of the first polyphonic character in at least one polyphonic character in the predicted pronunciation sequence and / or the first pronunciation prediction model is incorrect, obtain the second correct pronunciation corresponding to the first polyphonic character.

[0112] In some embodiments, after obtaining the predicted pronunciation sequence / first predicted sequence corresponding to the initial text sequence, the polyphonic character disambiguation device may choose to play the corresponding speech according to the predicted pronunciation sequence / first predicted sequence, and receive feedback information on the speech, and determine whether the pronunciation of polyphonic characters in the generated predicted pronunciation sequence / first predicted sequence is correct based on the feedback information.

[0113] In some embodiments, when the pronunciation prediction of the first polyphonic character in the predicted pronunciation sequence obtained by the first pronunciation prediction model, the second pronunciation prediction model, and / or the third pronunciation prediction model is incorrect, the disambiguation device for polyphonic characters can obtain the second correct pronunciation of the first polyphonic character.

[0114] In other embodiments, when the disambiguation device for polyphonic characters makes a pronunciation prediction error for the first polyphonic character in the first prediction sequence obtained by the first pronunciation prediction model, it can also obtain the second correct pronunciation of the first polyphonic character. It is understood that the first prediction sequence can be determined based on the final predicted pronunciation sequence and the first pronunciation result for each pronunciation obtained by the first pronunciation prediction model.

[0115] Step S802: Update the predicted pronunciation sequence or the first predicted sequence based on the second correct pronunciation to obtain the latest predicted sequence; generate new error cases based on the latest predicted sequence and the initial text sequence.

[0116] In some embodiments, after obtaining the second correct pronunciation of the first polyphonic character with a pronunciation prediction error, the disambiguation device for polyphonic characters can update the predicted pronunciation sequence or the first predicted sequence based on the second correct pronunciation to obtain the latest predicted sequence; and generate new error cases based on the latest predicted sequence and the initial text sequence.

[0117] Step S803: Add the new error case to the error case set to obtain the updated error case set.

[0118] In some embodiments, the disambiguation device for polyphonic characters can add new error cases to the error case set after each new error case is obtained, thus obtaining an updated error case set.

[0119] Step S804: Iteratively update the third pronunciation prediction model using the updated set of error cases.

[0120] In some embodiments, after updating the error case set, the disambiguation device for polyphonic characters can use the updated error case set to train the third pronunciation prediction model. Thus, polyphonic characters that are easy to predict incorrectly can be put into the third pronunciation prediction model so that when predicting polyphonic characters in the same context again, the third pronunciation prediction model can successfully predict the polyphonic character based on the continuously updated error case set.

[0121] In some embodiments of this application, reference is made to Figure 9 The disambiguation device for polyphonic characters can also perform the following steps S901 to S902:

[0122] Step S901: While continuously updating the error case set, and when the number of error cases in the updated error case set reaches the second threshold, retrain the first pronunciation prediction model and the second pronunciation prediction model using the updated error case set and the training sample set, so as to perform pronunciation prediction on the next new text sequence; the training sample set is the sample set used when training the first pronunciation prediction model.

[0123] In some embodiments, the disambiguation device for polyphonic characters adds erroneous cases to the error case set each time a prediction error occurs. If the number of erroneous cases in the updated error case set reaches a second threshold, it indicates that the number of prediction failures is too high, and it is necessary to consider retraining the first pronunciation prediction model and the second pronunciation prediction model. That is, the first pronunciation prediction model and the second pronunciation prediction model can be retrained simultaneously using the updated error case set and the training sample set for the next pronunciation prediction of a new text sequence.

[0124] The embodiment of the present application provides a method for disambiguating polyphonic characters. The method includes: using a first pronunciation prediction model to predict the pronunciation of each polyphonic character in the initial text sequence, and obtaining a first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence includes at least one polyphonic character with different pronunciations; using a second pronunciation prediction model and / or a third pronunciation prediction model to perform pronunciation statistics on each polyphonic character in the initial text sequence, and obtaining a second pronunciation result and / or a third pronunciation result of each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of polyphonic characters; fusing the pronunciation of at least one of the second pronunciation result and the third pronunciation result with the first pronunciation result to obtain the final pronunciation of each polyphonic character, so as to determine a predicted pronunciation sequence corresponding to the initial text sequence; adopting the above implementation solution, the present application predicts the pronunciation of polyphonic characters in the initial text sequence through at least two different models, obtains pronunciation results under at least two prediction methods, and then synthesizes the pronunciation results under at least two prediction methods to determine the final pronunciation of each polyphonic character. In this way, the present application can determine a more accurate pronunciation of polyphonic characters based on at least two different model prediction methods, thereby improving the effect of polyphonic character pronunciation prediction.

[0125] The above text introduced the method for disambiguating polyphonic characters provided by the embodiment of the present application. To facilitate the understanding of the embodiment of the present application, the possible implementation solutions of the method for disambiguating polyphonic characters applicable to the embodiment of the present application will be introduced below in the specific application scenario of the model.

[0126] Suppose the text sequence to be predicted (the initial text sequence in the above embodiment) is "Teacher Shan said that Chanyu can only ride a horse, not a bicycle". It can be seen that there are the following polyphonic characters in this sequence: "Shan", "said", "only", "can", "ride", and "bike", and it is necessary to accurately predict the pronunciation of each polyphonic character. The specific steps are as follows:

[0127] Step 1: Count all polyphonic characters and their corresponding pronunciations in Chinese Mandarin, and organize them into a polyphonic character pronunciation mapping dictionary (the preset polyphonic character mapping dictionary in the above embodiment), and count all non-polyphonic characters and their corresponding pronunciations in Chinese Mandarin, and organize them into a non-polyphonic character pronunciation dictionary (the preset non-polyphonic character mapping dictionary in the above embodiment).

[0128] Step 2: Use the training sample set to train the LINEAR model (the first pronunciation prediction model in the above embodiment), and use the training sample set to train a 6th-order ngram model (the second pronunciation prediction model in the above embodiment). The output of the ngram model is the splicing probability of a character and its corresponding pronunciation. For example: jiao_jiao4 50%; Finally, it is also necessary to count the error cases that occur during the model prediction process, organize the error cases into an error case set, and train a small 6th-order ngram sub-model (the third pronunciation prediction model in the above embodiment), and the output is also the splicing probability of a character and its corresponding pronunciation.

[0129] Step 3: Input the text sequence to be predicted, "Teacher Shan said that Chanyu can only ride a horse, not a bicycle", into the embedding layer of the Chinese Mandarin pronunciation conversion neural network model. The embedding layer maps each Chinese character in the text sequence to be predicted, "Teacher Shan said that Chanyu can only ride a horse, not a bicycle", into a character embedding vector.

[0130] Step 4: Input the character embedding vectors corresponding to the text sequence to be predicted, "Teacher Shan said that Chanyu can only ride a horse, not a bicycle", into the BERT model. Refer to Figure 10 , the BERT model maps the character embedding vectors to high-dimensional vectors (the vector encoding in the above embodiment).

[0131] Step 5: After obtaining the high-dimensional vectors corresponding to each Chinese character in the text sequence to be predicted, "Teacher Shan said that Chanyu can only ride a horse, not a bicycle", first screen out the polyphonic characters from "Teacher Shan said that Chanyu can only ride a horse, not a bicycle" according to the polyphonic character pronunciation mapping dictionary, and at least two pronunciations corresponding to each polyphonic character. Input the high-dimensional vectors corresponding to each polyphonic character and at least two pronunciations corresponding to each polyphonic character into the trained LINEAR model, and output the posterior probability of each polyphonic character under any pronunciation (the first pronunciation result in the above embodiment).

[0132] Refer to Figure 10 , input the high-dimensional vectors corresponding to each polyphonic character in "Teacher Shan said that Chanyu can only ride a horse, not a bicycle" and at least two pronunciations corresponding to each polyphonic character into the trained LINEAR model in turn. That is, input the high-dimensional vector corresponding to "Dan" and the pronunciations "shan4", "chan2", and "dan1" corresponding to "Dan" into the trained LINEAR model respectively, and obtain that the posterior probability of "shan4" is 0.9, the posterior probability of "chan2" is 0.07, and the posterior probability of "dan1" is 0.03. Similarly, obtain the posterior probabilities of each polyphonic character in "Teacher Shan said that Chanyu can only ride a horse, not a bicycle" in turn, as shown in Figure 10 shown.

[0133] Step 6: Input the text sequence to be predicted, "Teacher Shan said that Chanyu can only ride a horse and not a bicycle", and the posterior probability of each polyphonic character under any pronunciation into the decoder (which can be a beam search decoder). The decoder inputs the text sequence to be predicted, "Teacher Shan said that Chanyu can only ride a horse and not a bicycle", into the ngram model and the ngram sub-model respectively, and obtains the first splicing probability (the second pronunciation result in the above embodiment) and the second splicing probability (the third pronunciation result in the above embodiment) of each polyphonic character and the possible pronunciations in the text sequence to be predicted, "Teacher Shan said that Chanyu can only ride a horse and not a bicycle".

[0134] Step 7: The encoder determines the final probability of each polyphonic character under any pronunciation according to the posterior probability, the first splicing probability, and the second splicing probability of each polyphonic character under any pronunciation.

[0135] Exemplarily, when fusing the probabilities obtained by different models, the following formula can be used:

[0136]

[0137] In formula (1), p is the final probability, and p i is the probability output by different models.

[0138] Specifically, when the encoder determines the final probability of each polyphonic character under any pronunciation according to the posterior probability, the first splicing probability, and the second splicing probability of each polyphonic character under any pronunciation, it also needs to determine the weights assigned to each model, that is, the first weight assigned to the LINEAR model, the second weight assigned to the ngram model, and the third weight assigned to the ngram sub-model, and determine the final probability according to the probabilities output by different models and the corresponding assigned weights.

[0139] Refer to Figure 11 , taking the "Dan" in "Teacher Dan said" as an example, calculate the final probabilities of "Dan" under "shan4", "chan2", and "dan1" respectively. That is, the final probability p(shan4) of "Dan" under "shan4" is the posterior probability p L (shan4) output by the LINEAR model and the first weight w1, the first splicing probability p n1 (shan4) output by the ngram model and the second weight w2, and the second splicing probability p n2 (shan4) output by the ngram sub-model and the third weight w3. Similarly, the final probability p(chan2) of "Dan" under "chan2" is the posterior probability p L(chan2) and the first weight w1, the first concatenation probability p output by the ngram model. n1 (chan2) and the second weight w2, the second concatenation probability p output by the ngram sub-model. n2 (chan2) and the third weight w3 are used to calculate; the final probability p(dan1) of "single" under "dan1" is the posterior probability p output by the LINEAR model. L (dan1) and the first weight w1, the first concatenation probability p output by the ngram model. n1 (dan1) and the second weight w2, the second concatenation probability p output by the ngram sub-model. n2 (dan1) and the third weight w3 are calculated.

[0140] Step 8: After obtaining the final probability of each polyphonic character under any pronunciation, the encoder determines the pronunciation with the highest final probability among all pronunciations corresponding to each polyphonic character as the final pronunciation of each polyphonic character; and determines the pronunciations of non-polyphonic characters in the text sequence to be predicted based on the pronunciation dictionary of non-polyphonic characters. Based on the final pronunciations of polyphonic characters and the pronunciations of non-polyphonic characters, a predicted pronunciation sequence of the text sequence to be predicted is determined. (Refer to...) Figure 10 The result (a predicted pronunciation sequence) only includes the final pronunciation of polyphonic characters. However, in different implementations, the final pronunciation of non-polyphonic characters can also be marked in the result.

[0141] Step 9: In the process of using the above model to predict polyphonic characters, the weights of different models can be adjusted appropriately according to the prediction results. Usually, the second weight of the ngram model can be set to 0.4, and the third weight of the ngram sub-model can be set between 0.4 and 1.0. Since the ngram sub-model is generated from erroneous cases, when the third weight of the ngram sub-model is adjusted to 1.0, the errors can be corrected by 99%.

[0142] Step 10: Continuously update the ngram sub-model by collecting error cases during the deployment process.

[0143] Step 11: When the number of erroneous cases in the ngram sub-model reaches n (n≥1000), add the erroneous cases to the training sample set and retrain the LINEAR model and the ngram model.

[0144] Understandably, when deploying the above model, the deployment plan can be determined based on the deployment location, referring to... Figure 12The above scheme mainly involves inputting the posterior probability output by the LINEAR model, the first concatenation probability output by the ngram model, the second concatenation probability output by the ngram sub-model, the polyphonic character pronunciation mapping dictionary, and the non-polyphonic character pronunciation dictionary into the decoder, thereby determining a final pronunciation for each text sequence to be predicted that includes polyphonic characters. It can be seen that local deployment or cloud deployment can be selected according to different deployment methods. When deployed in the cloud, the ngram model is usually 2G-6G in size. The larger the model, the better the discrimination. After exceeding 6G, further increasing the model size does not yield significant benefits. When deployed on robotic devices (such as Nvidia AGX Orin devices), although the robotic devices have 12 cores and 64G of memory, due to the resource constraints allocated to the deployed model, the ngram model will be removed, and only the ngram sub-model will be retained.

[0145] Based on the above embodiments, in another embodiment of this application, a disambiguation device for polyphonic characters is provided. Figure 13 This is a schematic diagram of the composition structure of a disambiguation device for polyphonic characters provided in an embodiment of this application, as shown below. Figure 13 As shown, the disambiguation device 1300 for polyphonic characters includes:

[0146] The prediction unit 1301 is used to predict the pronunciation of each polyphonic character in the initial text sequence using a first pronunciation prediction model, and to obtain the first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence contains at least one polyphonic character with different pronunciations;

[0147] The prediction unit 1301 is also used to perform pronunciation statistics on each polyphonic character in the initial text sequence using a second pronunciation prediction model and / or a third pronunciation prediction model, to obtain the second pronunciation result and / or the third pronunciation result of each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of polyphonic characters;

[0148] The fusion unit 1302 is used to perform pronunciation fusion of at least one of the second pronunciation result and the third pronunciation result with the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining the predicted pronunciation sequence corresponding to the initial text sequence.

[0149] In some embodiments, the fusion unit 1302 is further configured to fuse at least one of the second pronunciation result and the third pronunciation result with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result of any pronunciation of each polyphonic character, and determine the pronunciation corresponding to the highest fusion result in each polyphonic character as the final pronunciation of the polyphonic character.

[0150] In some embodiments, the disambiguation device 1300 for polyphonic characters further includes: a determining unit;

[0151] A determining unit is used to determine a first weight assigned to each of at least one of the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model;

[0152] The fusion unit 1302 is further configured to, based on their respective first weights, fuse at least one of the second and third pronunciation results with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result for any pronunciation of each polyphonic character.

[0153] In some embodiments, the disambiguation device 1300 for polyphonic characters further includes: a lookup unit and a generation unit;

[0154] The determining unit is also used to determine non-polyphonic characters in the initial text sequence, excluding at least one polyphonic character;

[0155] The lookup unit is used to find the corresponding pronunciation of a non-polyphonic character from a preset non-polyphonic character mapping dictionary;

[0156] The generation unit is used to generate a predicted pronunciation sequence corresponding to the initial text, based on the final pronunciation of each polyphonic character and the pronunciation of non-polyphonic characters, in accordance with the text order of the initial text sequence.

[0157] In some embodiments, the disambiguation device 1300 for polyphonic characters further includes: a deployment unit;

[0158] The determining unit is also used to determine, in the case of deployment on the first device, the idle resources allocated in the first device for the deployment model;

[0159] The deployment unit is used to simultaneously deploy the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model to the first device when the idle resources are greater than the first threshold value.

[0160] The deployment unit is also used to deploy the first pronunciation prediction model and the third pronunciation prediction model to the first device when the idle resources are not greater than the first threshold value; wherein the memory size required by the second pronunciation prediction model is greater than the memory size required by the third pronunciation prediction model.

[0161] In some embodiments, the polyphonic character disambiguation device 1300 further includes: a receiving unit and an execution unit;

[0162] The receiving unit is used to receive the initial text sequence;

[0163] The determining unit is also used to determine the semantics corresponding to the initial text sequence based on the predicted pronunciation sequence when a predicted pronunciation sequence corresponding to the initial text sequence is determined.

[0164] An execution unit is used to perform the first operation corresponding to the initial text sequence based on semantics.

[0165] In some embodiments, the polyphonic character disambiguation device 1300 further includes: an acquisition unit and a model training unit;

[0166] The acquisition unit is used to acquire cases of polyphonic characters that were predicted incorrectly in the history of the first pronunciation prediction model, and to generate a set of incorrect cases based on the cases of polyphonic characters and the first correct pronunciation of the cases of polyphonic characters.

[0167] The model training unit is used to train the third initial pronunciation prediction model using the set of error cases to obtain the third pronunciation prediction model.

[0168] In some embodiments, the disambiguation device 1300 for polyphonic characters further includes: an updating unit;

[0169] The acquisition unit is further configured to acquire the second correct pronunciation corresponding to the first polyphonic character in the first prediction sequence predicted by the first prediction model and / or the first prediction sequence predicted by the first pronunciation prediction model if the pronunciation prediction of the first polyphonic character is incorrect.

[0170] The update unit is used to update the predicted pronunciation sequence or the first predicted sequence based on the second correct pronunciation to obtain the latest predicted sequence; generate new error cases based on the latest predicted sequence and the initial text sequence; and add the new error cases to the error case set to obtain the updated error case set.

[0171] The model training unit is also used to iteratively update the third pronunciation prediction model using the updated set of error cases.

[0172] In some implementations, the model training unit is further configured to retrain the first pronunciation prediction model and the second pronunciation prediction model using the updated error case set and the training sample set when the error case set is continuously updated and the number of error cases in the updated error case set reaches a second threshold, so as to perform pronunciation prediction on the next new text sequence; the training sample set is the sample set used when training the first pronunciation prediction model.

[0173] This application provides a disambiguation device for polyphonic characters. The device includes: a prediction unit, configured to predict the pronunciation of each polyphonic character in an initial text sequence using a first pronunciation prediction model, to obtain a first pronunciation result for each polyphonic character under any pronunciation; the initial text sequence contains at least one polyphonic character with different pronunciations; the prediction unit is further configured to perform pronunciation statistics on each polyphonic character in the initial text sequence using a second pronunciation prediction model and / or a third pronunciation prediction model, to obtain a second pronunciation result and / or a third pronunciation result for each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of the polyphonic characters, and the third pronunciation prediction model is used to perform pronunciation statistics on the polyphonic characters. The device includes a pronunciation correction unit and a fusion unit, which fuses at least one of the second and third pronunciation results with the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining the predicted pronunciation sequence corresponding to the initial text sequence. Using the above implementation scheme, this application uses at least two different models to predict the pronunciation of polyphonic characters in the initial text sequence, obtaining pronunciation results under at least two prediction methods. Then, by combining the pronunciation results under at least two prediction methods, the final pronunciation of each polyphonic character is determined. Thus, this application can determine a more accurate pronunciation of polyphonic characters based on at least two different model prediction methods, thereby improving the effect of polyphonic character pronunciation prediction.

[0174] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this application can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0175] It should be noted that, in the embodiments of this application, if the above-mentioned method for disambiguating polyphonic characters is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a first device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0176] Figure 14This is a schematic diagram of the hardware entity of a first device provided in an embodiment of this application, such as... Figure 14 As shown, the hardware entity of the first device 1400 includes a processor 1401 and a memory 1402, wherein the memory 1402 stores a computer program that can run on the processor 1401, and the processor 1401 executes the program to implement the steps in the method of any of the above embodiments.

[0177] The memory 1402 stores computer programs that can run on the processor. The memory 1402 is configured to store instructions and applications that can be executed by the processor 1401. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) of the processor 1401 and the modules in the first device 1400. It can be implemented by flash memory or random access memory (RAM).

[0178] When processor 1401 executes a program, it implements the steps of the disambiguation method for polyphonic characters mentioned above. Processor 1401 typically controls the overall operation of the first device 1400.

[0179] This application provides a computer storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the polyphonic character disambiguation method as described in any of the above embodiments.

[0180] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above-described method for disambiguating polyphonic characters.

[0181] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0182] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0183] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0184] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0185] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0187] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0188] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0189] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0190] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a first device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0191] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for disambiguating polyphonic characters, characterized in that, The method includes: Using a first pronunciation prediction model, the pronunciation of each polyphonic character in the initial text sequence is predicted to obtain the first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence contains at least one polyphonic character with different pronunciations; Using a second pronunciation prediction model and / or a third pronunciation prediction model, pronunciation statistics are performed on each polyphonic character in the initial text sequence to obtain a second pronunciation result and / or a third pronunciation result for each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of polyphonic characters; At least one of the second pronunciation result and the third pronunciation result is fused with the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining the predicted pronunciation sequence corresponding to the initial text sequence.

2. The method according to claim 1, characterized in that, The step of fusing at least one of the second and third pronunciation results with the first pronunciation result to obtain the final pronunciation of each polyphonic character includes: At least one of the second pronunciation result and the third pronunciation result is fused with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result of any pronunciation of each polyphonic character, and the pronunciation corresponding to the highest fusion result in each polyphonic character is determined as the final pronunciation of the polyphonic character.

3. The method according to claim 2, characterized in that, The step of fusing at least one of the second and third pronunciation results with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result for any pronunciation of each polyphonic character includes: A first weight is assigned to each of at least one of the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model; Based on their respective first weights, at least one of the second pronunciation result and the third pronunciation result is fused with the same pronunciation in each polyphonic character in the first pronunciation result to obtain a fusion result of any pronunciation of each polyphonic character.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Identify the non-polyphonic characters in the initial text sequence, excluding the at least one polyphonic character; Search for the corresponding pronunciation of the non-polyphonic character in the preset non-polyphonic character mapping dictionary; Determining the predicted pronunciation sequence corresponding to the initial text sequence includes: Based on the final pronunciation of each polyphonic character and the pronunciation of the non-polyphonic characters, the predicted pronunciation sequence corresponding to the initial text is generated according to the text order of the initial text sequence.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In the case of deployment on the first device, determine the idle resources allocated in the first device for the deployment model; If the idle resources are greater than the first threshold, the first pronunciation prediction model, the second pronunciation prediction model, and the third pronunciation prediction model will be deployed simultaneously to the first device. If the idle resources are not greater than the first threshold, the first pronunciation prediction model and the third pronunciation prediction model are deployed to the first device; The second pronunciation prediction model requires more memory than the third pronunciation prediction model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Receive the initial text sequence; If a predicted pronunciation sequence corresponding to the initial text sequence is determined, the semantics corresponding to the initial text sequence are determined based on the predicted pronunciation sequence. The first operation corresponding to the initial text sequence is performed based on the semantics.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain the polyphonic character cases that were incorrectly predicted in the history of the first pronunciation prediction model, and generate an error case set based on the polyphonic character cases and the first correct pronunciation of the polyphonic character cases; The third pronunciation prediction model is obtained by training the third initial pronunciation prediction model using the error case set.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: If the pronunciation prediction of the first polyphonic character in at least one polyphonic character in the predicted pronunciation sequence and / or the first prediction sequence predicted by the first pronunciation prediction model is incorrect, the second correct pronunciation corresponding to the first polyphonic character is obtained; The predicted pronunciation sequence or the first predicted sequence is updated based on the second correct pronunciation to obtain the latest predicted sequence; new error cases are generated based on the latest predicted sequence and the initial text sequence. The new error cases are added to the error case set to obtain the updated error case set; The third pronunciation prediction model is iteratively updated using the updated set of error cases.

9. The method according to claim 8, characterized in that, The method further includes: If the error case set is continuously updated and the number of error cases in the updated error case set reaches a second threshold, the first pronunciation prediction model and the second pronunciation prediction model are retrained using the updated error case set and the training sample set for the next pronunciation prediction of a new text sequence; the training sample set is the sample set used when training the first pronunciation prediction model.

10. A disambiguation device for polyphonic characters, characterized in that, The device includes: A prediction unit is used to predict the pronunciation of each polyphonic character in an initial text sequence using a first pronunciation prediction model, to obtain the first pronunciation result of each polyphonic character under any pronunciation; the initial text sequence contains at least one polyphonic character with different pronunciations; The prediction unit is further configured to use a second pronunciation prediction model and / or a third pronunciation prediction model to perform pronunciation statistics on each polyphonic character in the initial text sequence, and obtain a second pronunciation result and / or a third pronunciation result for each polyphonic character under any pronunciation; the second pronunciation prediction model is used to predict the pronunciation of polyphonic characters, and the third pronunciation prediction model is used to correct the pronunciation of polyphonic characters; A fusion unit is used to fuse at least one of the second pronunciation result and the third pronunciation result with the first pronunciation result to obtain the final pronunciation of each polyphonic character, thereby determining the predicted pronunciation sequence corresponding to the initial text sequence.

11. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of any one of the polyphonic character disambiguation methods as claimed in claims 1 to 9.

Images