Word phoneme conversion method, training method of word phoneme conversion model and electronic equipment

Abstract

The invention provides a method for converting words into phonemes, a training method of a model of the method and electronic equipment. The method comprises the following steps: acquiring training data; inputting the training word into a letter sequence blocking module to obtain a predicted letter block sequence and a feature matrix corresponding to the training word; inputting the feature matrix into a phoneme sequence generation and partitioning module to obtain a predicted phoneme block sequence corresponding to the training word; and performing iterative training on the word conversion phoneme model according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence and the real phoneme block sequence until a preset condition is met, and obtaining a trained word conversion phoneme model. In the training method, a letter block sequence and a phoneme block sequence output by a word conversion model are jointly trained by adopting triple data, so that the word conversion model obtained by subsequent training can output the letter block sequence and the phoneme block sequence, and letter blocks in the letter block sequence are in one-to-one correspondence with phoneme blocks in the phoneme block sequence.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent conversion technology, and in particular to a method for converting words into phonemes, a training method for its model, and an electronic device. Background Art

[0002] The conversion of words to phonemes is an important part of speech synthesis, speech recognition and speech evaluation. At present, in the word-to-phoneme model, only the conversion of word letter sequences into phoneme sequences is concerned, and the alignment relationship between letters and phonemes is not concerned. In this way, when this technology is applied to the field of speech evaluation to score and provide feedback on the pronunciation level of users, it is difficult to align letters and phonemes one by one, and to provide feedback to users on which letters in the word have problems with user pronunciation. For example, the word "fume" has four letters, the letter block sequence is "f", "u", "m" and "e", and the phoneme block sequence is "F", "Y", "UM" and "M", where the letter block "u" corresponds to the phoneme blocks "Y" and "UM", and the letter block "e" is silent. According to the spelling rules, it can be combined with the previous letter block "m" as a unit corresponding to the phoneme "M". If the conversion model does not correctly match the letter block with the phoneme block, it will cause incorrect feedback and affect the user experience. Summary of the invention

[0003] The present application provides a word-to-phoneme method and a training method for its model, as well as an electronic device. In the training method, the word conversion model is trained by using triple data. The word conversion model obtained by subsequent training can output a letter block sequence and a phoneme block sequence, and the letter blocks in the letter block sequence correspond one-to-one to the phoneme blocks in the phoneme block sequence.

[0004] In a first aspect, an embodiment of the present application provides a method for training a word-to-phoneme model, wherein the word-to-phoneme model includes a letter sequence chunking module and a phoneme sequence generation and chunking module, and the method includes: obtaining training data, wherein the training data includes a plurality of training words, a real letter block sequence corresponding to the training words, and a real phoneme block sequence corresponding to the training words, and each letter block in the real letter block sequence has a one-to-one correspondence with each phoneme block in the real phoneme block sequence; inputting the training word into the letter sequence chunking module to obtain a predicted letter block sequence and a feature matrix corresponding to the training word; inputting the feature matrix into the phoneme sequence generation and chunking module to obtain a predicted phoneme block sequence corresponding to the training word; and iteratively training the word-to-phoneme model according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain a trained word-to-phoneme model.

[0005] In some embodiments, the letter sequence blocking module is an encoding module, and the phoneme sequence generation and blocking module is a decoding module.

[0006] In some embodiments, the word-to-phoneme model includes one of a long short-term memory network, a convolutional neural network, an attention mechanism model, and a converter model.

[0007] In some embodiments, the letter sequence block module includes an input layer, a hidden layer and an output layer, and the feature matrix is ​​the output of the output layer or the output of the hidden layer.

[0008] In some embodiments, the obtaining of training data includes: obtaining the multiple training words and the phoneme block sequence corresponding to each of the training words; obtaining the letter block sequence corresponding to each of the training words based on the multiple training words; obtaining the spelling rules, and modifying the letter block sequence and the phoneme block sequence based on the spelling rules to obtain the real letter block sequence corresponding to each of the training words and the real phoneme block sequence corresponding to each of the training words; obtaining the training data based on the multiple training words, the real phoneme block sequence corresponding to each of the training words, and the real letter block sequence corresponding to each of the training words.

[0009] In some embodiments, the word-to-phoneme model is iteratively trained according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain a trained word-to-phoneme model, including: obtaining the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence according to a loss function, and iteratively adjusting the parameters of the word-to-phoneme model until the preset condition is met to obtain a trained word-to-phoneme model.

[0010] In some embodiments, the loss function includes an encoding loss function and a decoding loss function, and the total loss between the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence is obtained based on the loss function, including: obtaining the encoding loss according to the predicted letter block sequence, the real letter block sequence, and the encoding loss function; obtaining the decoding loss according to the predicted phoneme block sequence, the real phoneme block sequence, and the decoding loss function; obtaining the total loss according to the encoding loss and the decoding loss.

[0011] In a second aspect, an embodiment of the present application provides a word-to-phoneme conversion method, the method comprising: obtaining a word to be converted; inputting the word to be converted into a word-to-phoneme model to obtain a corresponding letter block sequence and a phoneme block sequence, wherein each letter block in the letter block sequence has a one-to-one correspondence with each phoneme block in the phoneme block sequence, and the word-to-phoneme model is trained using the training method described in any one of the embodiments of the first aspect.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of the embodiments of the first or second aspect above.

[0013] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method described in any one of the embodiments of the first aspect or the second aspect above.

[0014] In a fifth aspect, an embodiment of the present application further provides a computer program product, wherein the computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer executes the method described in any one of the embodiments of the first aspect or the second aspect above.

[0015] Compared with the prior art, the beneficial effects of the present application are as follows: Different from the prior art, the embodiment of the present application provides a word-to-phoneme method and a training method and electronic device for its model, wherein the word-to-phoneme model includes a letter sequence block module and a phoneme sequence generation and block module, and the method includes: obtaining training data, wherein the training data includes multiple training words, real letter block sequences corresponding to the training words, and real phoneme block sequences corresponding to the training words, and each letter block in the real letter block sequence has a one-to-one correspondence with each phoneme block in the real phoneme block sequence; inputting the training word into the letter sequence block module to obtain a predicted letter block sequence and a feature matrix corresponding to the training word; inputting the feature matrix into the phoneme sequence generation and block module to obtain a predicted phoneme block sequence corresponding to the training word; iteratively training the word-to-phoneme model according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain a trained word-to-phoneme model. In this training method, the letter block sequence and the phoneme block sequence output by the word conversion model are jointly trained by using triple data, so that the word conversion model obtained by subsequent training can output the letter block sequence and the phoneme block sequence, and the letter blocks in the letter block sequence correspond to the phoneme blocks in the phoneme block sequence one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements / modules and steps with the same reference numerals in the drawings are represented as similar elements / modules and steps. Unless otherwise specified, the figures in the drawings do not constitute proportional limitations.

[0017] Figure 1 This is a schematic diagram of an application scenario of a word-to-phoneme conversion system provided in an embodiment of the present application;

[0018] Figure 2 is a structural block diagram of an electronic device provided in an embodiment of the present application;

[0019] Figure 3 It is a flowchart of a method for training a word-to-phoneme model provided in an embodiment of the present application;

[0020] Figure 4 This is a schematic diagram of a conversion process of converting a word into phonemes provided by an embodiment of the present application;

[0021] Figure 5 It is a flowchart of a method for converting words into phonemes provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements can also be made without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0023] For ease of understanding of the present application, the present application is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those generally understood by those skilled in the art of the present application. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not used to limit the present application. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0024] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic, in some cases, the module division can be different from that in the device. In addition, the words "first", "second", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0025] At present, in the word-to-phoneme algorithm, the alignment method of letters to phonemes is currently mainly based on the many-to-many alignment model (m2m-aligner) proposed by Tarek Sherif (2007). This model can create an alignment of the letter sequence and phoneme sequence of words in the dictionary without the need for labeled data or language knowledge. However, this model cannot predict the alignment information of out-of-words, because the model requires paired letter sequences and phoneme sequences of words, while out-of-words lack phoneme sequences. At the same time, the alignment method of m2m-aligner does not take into account linguistic knowledge, resulting in some letter block divisions not conforming to pronunciation and spelling rules.

[0026] In order to solve the above technical problems, the embodiments of the present application provide a method for converting words into phonemes and a training method and electronic device for its model. The word-to-phoneme model obtained by the training method can realize both the conversion of letters to phonemes and the correspondence of letter blocks to phoneme blocks, and can solve the alignment problem of letters and phonemes of out-of-word words in the m2m-aligner model online.

[0027] As an example, see Figure 1 , Figure 1The electronic device 100 is connected to the server 200 via a network, wherein the network may be a wide area network or a local area network, or a combination of the two.

[0028] The electronic device 100 can be used to obtain training data and construct a word-to-phoneme model. For example, a person skilled in the art downloads the prepared training data on a terminal, and constructs a model structure of the word-to-phoneme model. The training data includes a plurality of words. It is understood that the electronic device 100 can also be used to obtain words for training or testing.

[0029] In some embodiments, the electronic device 100 locally executes the method for training the word-to-phoneme model provided in the embodiment of the present application to complete the training of the designed model using the training data, determine the final model parameters, and thus configure the final model parameters by the preset model to obtain the trained word-to-phoneme model. In some embodiments, the electronic device 100 can also send the training data and the constructed word-to-phoneme model stored on the terminal by a person skilled in the art to the server 200 through the network, the server 200 receives the training data and the word-to-phoneme model, trains the word-to-phoneme model using the training data, determines the final model parameters, and then sends the final model parameters to the electronic device 100, the electronic device 100 saves the final model parameters, so that the word-to-phoneme model configures the final model parameters, and the trained word-to-phoneme model can be obtained.

[0030] The following describes an exemplary application of an electronic device 100 for training a word-to-phoneme model or for word-to-phoneme conversion provided in an embodiment of the present application. It can be understood that the electronic device 100 can both train a word-to-phoneme model and use the word-to-phoneme model to perform word-to-phoneme conversion.

[0031] The electronic device 100 provided in some embodiments of the present application may be a server, such as a server deployed in the cloud. When the server is used to train a word-to-phoneme model, the word-to-phoneme model is iteratively trained using the word according to the word and word-to-phoneme model provided by other devices or those skilled in the art, and the final model parameters are determined, so that the classifier configures the final model parameters to obtain the trained word-to-phoneme model. When the server is used to convert word-to-phonemes, the built-in trained word-to-phoneme model is called, the word to be tested is provided to the word-to-phoneme device, and corresponding calculations are performed to obtain the corresponding letter block sequence and phoneme block sequence, and the letter blocks in the letter block sequence correspond one-to-one to the phoneme blocks in the phoneme block sequence.

[0032] The electronic device 100 provided in other embodiments of the present application can also be various types of terminals such as a laptop computer, a desktop computer or a mobile device. When the terminal is used to train the word-to-phoneme model, a person skilled in the art inputs the prepared word into the terminal, and designs the word-to-phoneme model on the terminal. The terminal uses the word to iteratively train the word-to-phoneme model, and determines the final model parameters, so that the word-to-phoneme model configures the final model parameters, and the trained word-to-phoneme model can be obtained. When the terminal is used for word-to-phoneme conversion, the test word is processed accordingly to obtain a corresponding feature map of a fixed step length, and then the built-in trained word-to-phoneme model is called to perform corresponding calculation processing on the feature map of the fixed step length to obtain a corresponding letter block sequence and a phoneme block sequence, and the letter blocks in the letter block sequence correspond to the phoneme blocks in the phoneme block sequence.

[0033] For details, see Figure 2 , which shows the hardware structure of an electronic device 100 capable of executing the method for training a word-to-phoneme model and the word-to-phoneme method described in this application. The electronic device 100 may be Figure 1 An electronic device 100 is shown.

[0034] See also Figure 2 , the electronic device 100 includes a processor 101 and a memory 102 connected in communication. Here, the communication connection can be connected via a bus, Figure 2 The bus connection is used as an example. It can be understood that Figure 2 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0035] The processor 101 is configured to support the electronic device 100 to execute the method for training a word-to-phoneme model or a corresponding function in the word-to-phoneme method. The processor 101 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above-mentioned hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0036] The memory 102, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the method for training a word-to-phoneme model or the program instructions / modules corresponding to the word-to-phoneme method in the embodiments of the present application. The processor 101 can implement the method for training a word-to-phoneme model or the word-to-phoneme method in any of the following method embodiments by running the non-transitory software programs, instructions and modules stored in the memory 102.

[0037] The memory 102 may include a volatile memory (VM), such as a random access memory (RAM); the memory 1002 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 102 may also include a combination of the above types of memory.

[0038] It is understandable that the electronic device also includes other hardware and software that support operation. The hardware may include antennas, various sensors, microphones, etc. The software may include an operating system, etc. The operating system is a program that manages and controls the hardware and software resources of the electronic device. The software may also include various application programs (applications, apps). Here, other parts of the electronic device that are not related to the improvement of the embodiment of the present application will not be described.

[0039] In some embodiments, the electronic device may also be a terminal or a server, etc. In the embodiments of the present application, there is no restriction on the form of the electronic device, as long as it can acquire words and has computing and processing capabilities.

[0040] It can be understood from the above that the word-to-phoneme conversion method provided in the embodiment of the present application can be implemented by various types of electronic devices with processing capabilities, such as by a processor of an electronic device or by other devices with computing and processing capabilities. Other devices with computing and processing capabilities can be intelligent terminals or servers that are connected to the electronic device for communication.

[0041] The following describes the method for training a word-to-phoneme model provided by the embodiment of the present application in combination with the exemplary application and implementation of the electronic device provided by the embodiment of the present application. Figure 3 , Figure 3 1 is a flow chart of a method for training a word-to-phoneme model provided in an embodiment of the present application. It is understandable that the execution subject of the training method may be one or more processors of an electronic device. The word-to-phoneme model includes a letter sequence block module and a phoneme sequence generation and block module, and the method includes:

[0042] Step S10: Acquire training data, wherein the training data includes multiple training words, real letter block sequences corresponding to the training words, and real phoneme block sequences corresponding to the training words, and each letter block in the real letter block sequence has a one-to-one correspondence with each phoneme block in the real phoneme block sequence.

[0043] The electronic device can download the training data through the network. It can be understood that the training data is a triple data in the form of (x, y, z), where x can be a training word, y is a real letter block sequence corresponding to the training word, and z is a real phoneme block sequence corresponding to the training word. The real letter block sequence contains letter blocks arranged in sequence, and the real phoneme block sequence contains phoneme blocks arranged in sequence. For a training word, the number of letter blocks in the corresponding real letter block sequence and the number of phoneme blocks in the corresponding real phoneme block sequence are equal, and the corresponding letter blocks and phoneme blocks are arranged in the same order in the sequence. For example, if the symbol "- " represents a separator and the symbol ":" represents a merge symbol, then for the training data (good, go:od, G-UH-D), the training word is "good", and the corresponding letter blocks in the letter block sequence are "g", "o:o", and "d", and the corresponding phoneme block sequence is "G", "UH", and "D", and the letter block "g" corresponds to the phoneme block "G", the letter block "o:o" corresponds to the phoneme block "UH", and the letter block "d" corresponds to the phoneme block "D".

[0044] Step S20: input the training word into the letter sequence block module 10 to obtain the predicted letter block sequence and feature matrix corresponding to the training word.

[0045] The letter sequence block module 10 is a classification model, which can be an encoding module, and is used to output letter sequences, separators between letters, and merging symbols. Figure 4 As shown, after obtaining the training data, the electronic device can insert a separator (such as "-") into each letter in the word and input it into the letter sequence block module 10, so that the letter sequence block module 10 outputs a predicted letter block sequence. For example, for the training word "fume", the electronic device inserts the separator "-" to obtain the letter block sequence "fume", and inputs the letter block sequence "fume" into the letter sequence block module 10 in sequence. It can be understood that the letter block sequence expected to be output is "fum:e". In practical applications, the separator and the merging symbol can be set according to actual needs, and there is no need to stick to the limitations in this article.

[0046] Step S30: input the feature matrix into the phoneme sequence generation and block segmentation module 20 to obtain a predicted phoneme block sequence corresponding to the training word.

[0047] The phoneme sequence generation and block module 20 is a generation model, which can be a decoding module, and is used to output a phoneme sequence, a separation symbol and a merging symbol between phonemes. Specifically, the letter sequence block module 10 includes an input layer, a hidden layer, and an output layer, and the feature matrix can be the output of the output layer or the output of the hidden layer. After obtaining the feature matrix, the electronic device can input the feature matrix into the phoneme sequence generation and block module 20, so that the phoneme sequence generation and block module 20 outputs a predicted phoneme block sequence.

[0048] Specifically, in some embodiments, the step includes: simultaneously inputting the feature matrix and the output of the phoneme sequence generation and block module 20 at the previous moment into the phoneme sequence generation and block module 20, obtaining the output at the next moment, and looping in sequence until the end symbol (such as "") is encountered or the maximum length set for the generated sequence is reached, wherein, at the initial moment, the output of the phoneme sequence generation and block module 20 at the previous moment is the initial symbol, such as " <os>”.

[0049] For example, see Figure 4 , the letter sequence block module 10 outputs "fum:e" in sequence, then the phoneme sequence generation and block module 20 will combine the initial symbol <os>The attention mechanism is used to output the phoneme "F" with the feature matrix output by the letter sequence block module 10; then, the phoneme sequence is generated and the " <os>" and "F" are self-attentioned and then the attention mechanism outputs "-" with the feature matrix output by the letter sequence block module 10; then, the phoneme sequence is generated and the " <os> ”, “F” and “-” are self-attentioned and then the feature matrix output by the letter sequence block module 10 is used as the attention mechanism to output “Y”, and so on, until the phoneme sequence generation and block module 20 outputs the end symbol< / os> Or the maximum length of the generated sequence is reached.

[0050] Step S40: Iteratively train the word-to-phoneme model according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until the preset conditions are met to obtain a trained word-to-phoneme model.

[0051] Finally, the word-to-phoneme model is trained using the predicted letter block sequence and the real letter block sequence, the predicted phoneme block sequence and the real phoneme block sequence, and the parameters of the word-to-phoneme model are continuously adjusted until the preset conditions are finally met, such as the word-to-phoneme model converges or the predicted letter block sequence output by the word-to-phoneme model is getting closer and closer to the real letter block sequence, and the predicted phoneme block sequence is getting closer and closer to the real phoneme block sequence. At this time, the parameter is used as the word-to-phoneme model parameter to obtain the trained word-to-phoneme model.

[0052] In the training method provided in the embodiment of the present application, the letter block sequence and the phoneme block sequence output by the word conversion model are jointly trained by using triple data, so that the word conversion model obtained by subsequent training can output the letter block sequence and the phoneme block sequence, and the letter blocks in the letter block sequence correspond to the phoneme blocks in the phoneme block sequence one by one. In addition, the letter sequence block module 10 can be of certain help to the task of converting letters to phonemes, and can further improve the accuracy of the phoneme sequence generation and the phoneme block sequence generated by the block module 20. It can be subsequently applied to speech synthesis, speech recognition and speech evaluation systems, and can improve the accuracy of subsequent speech synthesis, speech recognition and speech evaluation.

[0053] In some of the embodiments, the word-to-phoneme model includes one of a long short-term memory network (Long Short-Term Memory, LSTM), a convolutional neural network (Convolutional Neural Networks, CNN), an attention mechanism model, and a converter model. The specific structure of the above-mentioned model can be referred to the prior art and is not limited here. In the present application, the specific structure of the word-to-phoneme model can be selected according to actual needs, thereby improving the flexibility and freedom of model construction.

[0054] In some embodiments, obtaining training data includes:

[0055] Step S11: Acquire multiple training words and a phoneme block sequence corresponding to each training word.

[0056] Step S12: According to a plurality of training words, a letter block sequence corresponding to each training word is obtained.

[0057] Step S13: obtaining spelling rules, and modifying the letter block sequence and the phoneme block sequence based on the spelling rules to obtain a real letter block sequence and a real phoneme block sequence corresponding to each training word;

[0058] Step S14: Obtain training data according to a plurality of training words, a real phoneme block sequence corresponding to each training word, and a real letter block sequence corresponding to each training word.

[0059] Among them, the electronic device can obtain multiple training words and phoneme block sequences corresponding to each training word from the dictionary. It can be understood that the above information is binary data. Then, the electronic device can call the open source tool m2m-aligner to obtain triple data, which includes multiple training words, letter block sequences corresponding to each training word, and phoneme block sequences corresponding to each training word. However, in order to further improve the accuracy of the triple data, in this embodiment, the electronic device will also obtain spelling rules to correct the triple data obtained by the open source tool to obtain the corrected triple data, that is, to obtain multiple training words, real letter block sequences corresponding to each training word, and real phoneme block sequences corresponding to each training word. For example, for the word "night", after passing through m2m-aligner, the obtained triple data is: (night, ni:gh:t, N-AY-T), among which the letter block sequence corresponding to the word "night" is "ni:gh:t", and the phoneme block sequence corresponding to the word "night" is "N-AY-T", and according to the spelling rule, the letter "igh" will be pronounced as a whole. Therefore, after correction using the spelling rule, the corrected triple data is (night, ni:g:ht, N-AY-T), among which the real letter block sequence corresponding to the word "night" is "ni:gh:t", and the real phoneme block sequence corresponding to the word "night" is "N-AY-T".

[0060] It can be seen that in this embodiment, the letter block sequence and phoneme block sequence output by the m2m-aligner model that do not conform to the spelling habits can be adjusted to improve the accuracy of the training data, so that the correspondence between the letter blocks in the generated letter block sequence and the phoneme blocks in the phoneme block sequence of the word-to-phoneme model obtained by subsequent training will be more in line with the linguistic spelling habits, and the readability and understandability of the alignment of letter blocks and phoneme blocks will be improved. Compared with converting bigram data into trigram data by manual annotation, the annotation speed and efficiency can be improved.

[0061] In some embodiments, the word-to-phoneme model is iteratively trained according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain a trained word-to-phoneme model, including:

[0062] Step S41: Obtain the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence according to the loss function, and iteratively adjust the parameters of the word-to-phoneme model until the preset conditions are met to obtain the trained word-to-phoneme model.

[0063] The algorithm can be used to iteratively optimize the parameters of the word-to-phoneme model, and the number of iterations can be set to 100,000. In this embodiment, the parameters in the word-to-phoneme model can be obtained by training the model through an iterative algorithm, thereby obtaining a trained word-to-phoneme model.

[0064] Specifically, the loss function includes an encoding loss function, a decoding loss function and a total loss function. The total loss between the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence and the real phoneme block sequence is obtained according to the loss function, including: obtaining the encoding loss according to the predicted letter block sequence, the real letter block sequence and the encoding loss function; obtaining the decoding loss according to the predicted phoneme block sequence, the real phoneme block sequence and the decoding loss function; obtaining the total loss according to the encoding loss, the decoding loss and the total loss function.

[0065] Among them, the encoding loss function and the decoding loss function can use the cross entropy loss function, and the total loss = encoding loss + decoding loss, that is, when calculating the total loss, the weight ratio of the encoding loss and the decoding loss is 1:1.

[0066] It can be seen that in this embodiment, the use of the cross entropy loss function as the loss function can speed up the model convergence speed and reduce the convergence difficulty. The letter block sequence and the phoneme block sequence are equally important training reference factors, which can improve the accuracy of the trained word-to-phoneme model detection.

[0067] The present application also provides a method for converting words into phonemes. Figure 5 , the word conversion phoneme method includes:

[0068] Step S100: obtaining a word to be converted;

[0069] Step S200: Input the word to be converted into the word-to-phoneme model to obtain a corresponding letter block sequence and a phoneme block sequence, wherein each letter block in the letter block sequence has a one-to-one correspondence with each phoneme block in the phoneme block sequence, and the word-to-phoneme model is trained using the training method described in any one of the embodiments of the first aspect.

[0070] The electronic device is built with a word-to-phoneme application, and the word-to-phoneme model is encapsulated in the word-to-phoneme application. The word-to-phoneme model is called to convert the word to be converted into phonemes with a separator inserted therein, and after a series of calculations and processing, a predicted letter block sequence and a phoneme block sequence are output, and each letter block in the letter block sequence has a one-to-one correspondence with each phoneme block in the phoneme block sequence. It can be understood that the word-to-phoneme model is trained by the method of training the word-to-phoneme detection model in the above embodiment, and has the same structure and function as the word-to-phoneme model in the above embodiment, which will not be described one by one here.

[0071] The present application embodiment also provides a non-volatile computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, for example, to execute the method for training a word-to-phoneme model and the word-to-phoneme method described in the above embodiment. The computer-readable storage medium can be a storage medium such as FRAM, ROM, PROM, EPROM, EE PROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; it can also be a variety of devices including one or any combination of the above memories. The executable instructions can be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted language, or declarative or procedural language), and it can be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine or other unit suitable for use in a computing environment.

[0072] An embodiment of the present application also provides a computer program product, including a computer program stored on a non-volatile computer-readable storage medium, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method for training a word-to-phoneme model and the word-to-phoneme method described in the above embodiments.

[0073] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions to use at least one computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.< / os> < / os> < / os>

Claims

1. A method for training a word-to-phoneme model, characterized in that: The word-to-phoneme model includes a letter sequence block module and a phoneme sequence generation and block module, and the method includes: Acquire training data, wherein the training data includes a plurality of training words, a real letter block sequence corresponding to the training words, and a real phoneme block sequence corresponding to the training words, and each letter block in the real letter block sequence has a one-to-one correspondence with each phoneme block in the real phoneme block sequence; Input the training word into the letter sequence block module to obtain a predicted letter block sequence and a feature matrix corresponding to the training word; Inputting the feature matrix into the phoneme sequence generation and block segmentation module to obtain a predicted phoneme block sequence corresponding to the training word; The word-to-phoneme model is iteratively trained according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain a trained word-to-phoneme model.

2. The training method according to claim 1, characterized in that: The letter sequence block module is an encoding module, and the phoneme sequence generation and block module is a decoding module.

3. The training method according to claim 2, characterized in that: The word-to-phoneme model includes one of a long short-term memory network, a convolutional neural network, an attention mechanism model and a converter model.

4. The training method according to claim 3, characterized in that: The letter sequence block module includes an input layer, a hidden layer and an output layer, and the feature matrix is ​​the output of the output layer or the output of the hidden layer.

5. The training method according to any one of claims 1 to 4, characterized in that: The obtaining of training data comprises: Acquire the plurality of training words and a phoneme block sequence corresponding to each of the training words; According to the plurality of training words, obtaining a letter block sequence corresponding to each of the training words; Acquire spelling rules, and modify the letter block sequence and the phoneme block sequence based on the spelling rules to obtain a real letter block sequence corresponding to each of the training words and a real phoneme block sequence corresponding to each of the training words; The training data is obtained according to the plurality of training words, the real phoneme block sequence corresponding to each of the training words, and the real letter block sequence corresponding to each of the training words.

6. The training method according to any one of claims 1 to 4, characterized in that: The iterative training of the word-to-phoneme model according to the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain a trained word-to-phoneme model includes: The total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence and the real phoneme block sequence is obtained according to the loss function, and the parameters of the word-to-phoneme model are iteratively adjusted until the preset conditions are met to obtain the trained word-to-phoneme model.

7. The training method according to claim 6, characterized in that: The loss function includes an encoding loss function and a decoding loss function, and the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence is obtained according to the loss function, including: Obtaining a coding loss according to the predicted letter block sequence, the real letter block sequence, and the coding loss function; Obtaining a decoding loss according to the predicted phoneme block sequence, the real phoneme block sequence, and the decoding loss function; The total loss is obtained according to the encoding loss and the decoding loss.

8. A method for converting words into phonemes, characterized in that: include: Get the word to be converted; The word to be converted is input into a word-to-phoneme model to obtain a corresponding letter block sequence and a phoneme block sequence, wherein each letter block in the letter block sequence has a one-to-one correspondence with each phoneme block in the phoneme block sequence, and the word-to-phoneme model is trained using the training method described in any one of claims 1-7.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 8.

Images