A word-to-phoneme conversion method and its model training method, electronic equipment

Abstract

This application provides a word-to-phoneme conversion method and its model training method and electronic device. The method includes: acquiring training data; inputting training words into a letter sequence segmentation module to obtain a predicted letter block sequence and feature matrix corresponding to the training words; inputting the feature matrix into a phoneme sequence generation and segmentation module to obtain a predicted phoneme block sequence corresponding to the training words; iteratively training the word-to-phoneme conversion model based on the predicted letter block sequence, predicted phoneme block sequence, real letter block sequence, and real phoneme block sequence until a preset condition is met, thus obtaining the trained word-to-phoneme conversion model. In this training method, the letter block sequence and phoneme block sequence output by the word conversion model are jointly trained using triplet data, so that the subsequently trained word conversion model can output letter block sequences and phoneme block sequences, and the letter blocks in the letter block sequence correspond one-to-one with the phoneme blocks in the phoneme block sequence.

Description

Technical Field

[0001] This application relates to the field of intelligent conversion technology, and in particular to a word-to-phoneme conversion method and a training method and electronic device for its model. Background Technology

[0002] Word-to-phoneme conversion is a crucial component of speech synthesis, speech recognition, and speech evaluation. Currently, word-to-phoneme conversion models only focus on converting word letter sequences into phoneme sequences, neglecting the alignment between letters and phonemes. Consequently, when this technology is applied to speech evaluation to score and provide feedback on users' pronunciation, it becomes difficult to accurately align letters and phonemes, and to clearly indicate which letters in a word the user is pronouncing incorrectly. For example, the word "fume" has four letters: the letter block sequence is "f", "u", "m", and "e", and the phoneme block sequence is "F", "Y", "UM", and "M". The letter block "u" corresponds to phoneme blocks "Y" and "UM", and the letter block "e" is silent. According to spelling rules, it can be merged with the preceding letter block "m" to form a single unit corresponding to the phoneme "M". If the conversion model fails to correctly align letter blocks with phoneme blocks, it will result in incorrect feedback, negatively impacting the user experience. Summary of the Invention

[0003] This application provides a word-to-phoneme method and a training method and electronic device for the model. In the training method, the word conversion model is trained by using triplet data. The word conversion model obtained after subsequent training can output a sequence of letter blocks and a sequence of phoneme blocks, and the letter blocks in the letter block sequence correspond one-to-one with the phoneme blocks in the phoneme block sequence.

[0004] In a first aspect, embodiments of this application provide a training method for a word-to-phoneme model. The word-to-phoneme model includes a letter sequence segmentation module and a phoneme sequence generation and segmentation module. The method includes: acquiring training data, wherein the training data includes multiple training words, a real letter block sequence corresponding to the training words, and a real phoneme block sequence corresponding to the training words, wherein each letter block in the real letter block sequence has a one-to-one correspondence with each phoneme block in the real phoneme block sequence; inputting the training words into the letter sequence segmentation module to obtain a predicted letter block sequence and a feature matrix corresponding to the training words; inputting the feature matrix into the phoneme sequence generation and segmentation module to obtain a predicted phoneme block sequence corresponding to the training words; and iteratively training the word-to-phoneme model based on the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain the trained word-to-phoneme model.

[0005] In some embodiments, the letter sequence segmentation module is an encoding module, and the phoneme sequence generation and segmentation module is a decoding module.

[0006] In some embodiments, the word-to-phoneme model includes one of a long short-term memory network, a convolutional neural network, an attention mechanism model, and a converter model.

[0007] In some embodiments, the letter sequence segmentation module includes an input layer, a hidden layer, and an output layer, and the feature matrix is ​​the output of the output layer or the output of the hidden layer.

[0008] In some embodiments, obtaining training data includes: obtaining the plurality of training words and the phoneme block sequence corresponding to each training word; obtaining the letter block sequence corresponding to each training word based on the plurality of training words; obtaining spelling rules, and correcting the letter block sequence and the phoneme block sequence based on the spelling rules to obtain the real letter block sequence and the real phoneme block sequence corresponding to each training word; and obtaining the training data based on the plurality of training words, the real phoneme block sequence corresponding to each training word, and the real letter block sequence corresponding to each training word.

[0009] In some embodiments, the step of iteratively training the word-to-phoneme model based on the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met to obtain the trained word-to-phoneme model includes: obtaining the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence according to a loss function, and iteratively adjusting the parameters of the word-to-phoneme model until the preset condition is met to obtain the trained word-to-phoneme model.

[0010] In some embodiments, the loss function includes an encoding loss function and a decoding loss function. Obtaining the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the true letter block sequence, and the true phoneme block sequence based on the loss function includes: obtaining an encoding loss based on the predicted letter block sequence, the true letter block sequence, and the encoding loss function; obtaining a decoding loss based on the predicted phoneme block sequence, the true phoneme block sequence, and the decoding loss function; and obtaining the total loss based on the encoding loss and the decoding loss.

[0011] Secondly, embodiments of this application provide a word-to-phoneme conversion method, the method comprising: obtaining a word to be converted; inputting the word to be converted into a word-to-phoneme conversion model to obtain a corresponding letter block sequence and a phoneme block sequence, wherein each letter block in the letter block sequence has a one-to-one correspondence with each phoneme block in the phoneme block sequence, and the word-to-phoneme conversion model is trained using the training method described in any embodiment of the first aspect.

[0012] Thirdly, embodiments of this application provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any of the embodiments of the first or second aspect above.

[0013] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method described in any one of the embodiments of the first or second aspect above.

[0014] Fifthly, embodiments of this application also provide a computer program product, the computer program product including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in any one of the embodiments of the first or second aspect above.

[0015] Compared with the prior art, the beneficial effects of this application are as follows: Unlike the prior art, the embodiments of this application provide a word-to-phoneme method and a training method and electronic device for its model. The word-to-phoneme model includes a letter sequence segmentation module and a phoneme sequence generation and segmentation module. The method includes: acquiring training data, wherein the training data includes multiple training words, real letter block sequences corresponding to the training words, and real phoneme block sequences corresponding to the training words, wherein each letter block in the real letter block sequence and each phoneme block in the real phoneme block sequence have a one-to-one correspondence; inputting the training words into the letter sequence segmentation module to obtain a predicted letter block sequence and a feature matrix corresponding to the training words; inputting the feature matrix into the phoneme sequence generation and segmentation module to obtain a predicted phoneme block sequence corresponding to the training words; iteratively training the word-to-phoneme model based on the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is reached to obtain the trained word-to-phoneme model. In this training method, the word conversion model outputs a sequence of letter blocks and a sequence of phoneme blocks by using triplet data for joint training. This enables the word conversion model to output a sequence of letter blocks and a sequence of phoneme blocks, with a one-to-one correspondence between the letter blocks in the letter block sequence and the phoneme blocks in the phoneme block sequence. Attached Figure Description

[0016] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements / modules and steps with the same reference numerals in the drawings are represented as similar elements / modules and steps. Unless otherwise stated, the figures in the drawings do not constitute a limitation on scale.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of a word-to-phoneme conversion system provided in an embodiment of this application;

[0018] Figure 2 This is a structural block diagram of an electronic device provided in an embodiment of this application;

[0019] Figure 3 This is a flowchart illustrating a training method for a word-to-phoneme model provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram illustrating a word-to-phoneme conversion process provided in an embodiment of this application;

[0021] Figure 5 This is a flowchart illustrating a word-to-phoneme conversion method provided in an embodiment of this application. Detailed Implementation

[0022] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0023] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0024] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram, in some cases, they can be divided differently from those in the device. In addition, the terms "first" and "second" used herein do not limit the data or execution order, but only distinguish between identical or similar items with essentially the same function and effect.

[0025] Currently, in word-to-phoneme conversion algorithms, the letter-to-phoneme alignment method mainly adopts the many-to-many alignment model (m2m-aligner) proposed by Tarek-Sherif (2007). This model can create alignments between letter sequences and phoneme sequences of words in a dictionary without requiring labeled data or linguistic knowledge. However, this model cannot predict the alignment information of out-of-vocabulary words because it requires paired letter sequences and phoneme sequences for each word, while out-of-vocabulary words lack phoneme sequences. Furthermore, the m2m-aligner alignment method does not consider linguistic knowledge, resulting in some letter block divisions that do not conform to pronunciation rules.

[0026] To address the aforementioned technical problems, embodiments of this application provide a method for converting words to phonemes, a training method for the model, and an electronic device. The word-to-phoneme conversion model obtained by this training method can achieve both letter-to-phoneme conversion and letter block-to-phoneme block correspondence, and can solve the alignment problem of out-of-set word letters and phonemes in the m2m-aligner model online.

[0027] As an example, see Figure 1 , Figure 1This is a schematic diagram illustrating an application scenario of a word-to-phoneme conversion system provided in an embodiment of this application. The electronic device 100 is connected to the server 200 via a network, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0028] The electronic device 100 can be used to acquire training data and build a word-to-phoneme model. For example, someone skilled in the art can download prepared training data to a terminal and build the model structure for the word-to-phoneme model. The training data includes several words. It is understood that the electronic device 100 can also be used to acquire words for training or testing.

[0029] In some embodiments, the electronic device 100 locally executes the method for training a word-to-phoneme model provided in this application embodiment to train the designed model using training data, determine the final model parameters, and thus preset the model configuration to obtain the trained word-to-phoneme model. In some embodiments, the electronic device 100 may also send training data and a constructed word-to-phoneme model stored on a terminal by those skilled in the art to the server 200 via a network. The server 200 receives the training data and the word-to-phoneme model, trains the word-to-phoneme model using the training data, determines the final model parameters, and then sends the final model parameters to the electronic device 100. The electronic device 100 saves the final model parameters, enabling the word-to-phoneme model to be configured with the final model parameters, thereby obtaining the trained word-to-phoneme model.

[0030] The following describes an exemplary application of the electronic device 100 provided in the embodiments of this application for training a word-to-phoneme model or for word-to-phoneme conversion. It is understood that the electronic device 100 can both train the word-to-phoneme model and use the word-to-phoneme model to perform word-to-phoneme conversion.

[0031] The electronic device 100 provided in some embodiments of this application can be a server, such as a server deployed in the cloud. When the server is used to train a word-to-phoneme model, it iteratively trains the word-to-phoneme model using the word provided by other devices or those skilled in the art, based on the word and the word-to-phoneme model provided by the device, to determine the final model parameters. The classifier is then configured with these final model parameters to obtain the trained word-to-phoneme model. When the server is used for word-to-phoneme conversion, it calls the built-in trained word-to-phoneme model, provides the word to be tested to the word-to-phoneme device, performs corresponding calculations, and obtains the corresponding letter block sequence and phoneme block sequence, with a one-to-one correspondence between the letter blocks in the letter block sequence and the phoneme blocks in the phoneme block sequence.

[0032] In other embodiments of this application, the electronic device 100 can also be various types of terminals such as laptops, desktop computers, or mobile devices. When the terminal is used to train a word-to-phoneme model, those skilled in the art input prepared words into the terminal and design a word-to-phoneme model on the terminal. The terminal uses the word to iteratively train the word-to-phoneme model, determines the final model parameters, and then configures the word-to-phoneme model with the final model parameters to obtain the trained word-to-phoneme model. When the terminal is used for word-to-phoneme conversion, the test word is processed accordingly to obtain a feature map with a fixed step size. Then, the built-in trained word-to-phoneme model is called to perform corresponding calculations on the feature map with the fixed step size to obtain the corresponding letter block sequence and phoneme block sequence, and the letter blocks in the letter block sequence correspond one-to-one with the phoneme blocks in the phoneme block sequence.

[0033] For details, please see Figure 2 It illustrates the hardware structure of an electronic device 100 capable of executing the method for training a word-to-phoneme model and the word-to-phoneme method described in this application. The electronic device 100 may be... Figure 1 The electronic device 100 shown.

[0034] Please see Figure 2 The electronic device 100 includes a processor 101 and a memory 102 connected via a communication link. Here, the communication link can be established via a bus. Figure 2 The bus connection between China and Israel is illustrated by example. It is understood that... Figure 2 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0035] The processor 101 is configured to support the electronic device 100 in executing a method for training a word-to-phoneme model or a corresponding function within the word-to-phoneme method. The processor 101 can be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0036] The memory 102, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for training a word-to-phoneme model or the word-to-phoneme method in the embodiments of this application. The processor 101, by running the non-transitory software programs, instructions, and modules stored in the memory 102, can implement the method for training a word-to-phoneme model or the word-to-phoneme method in any of the following method embodiments.

[0037] Memory 102 may include volatile memory (VM), such as random access memory (RAM); memory 1002 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 102 may also include combinations of the above types of memory.

[0038] It is understood that electronic devices also include other supporting hardware and software. Hardware may include antennas, various sensors, microphones, etc. Software may include operating systems, which are programs that manage and control the hardware and software resources of electronic devices. Software may also include various applications (apps). Other parts of the improved electronic devices not involved in the embodiments of this application will not be described here.

[0039] In some embodiments, the electronic device may also be a terminal or a server, etc. In this application embodiment, no restrictions are placed on the form of the electronic device; it only needs to be able to acquire words and have computational processing capabilities.

[0040] As can be understood from the above, the word-to-phoneme conversion method provided in this application embodiment can be implemented by various types of electronic devices with processing capabilities, such as being implemented by the processor of an electronic device or by other devices with computing capabilities. Other devices with computing capabilities may be smart terminals or servers that are communicatively connected to the electronic device.

[0041] The following describes the method for training a word-to-phoneme model provided in this application, using exemplary applications and implementations of the electronic device provided in the embodiments of this application. Please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the method for training a word-to-phoneme conversion model provided in this application embodiment. It is understood that the execution entity of this training method can be one or more processors of an electronic device. The word-to-phoneme conversion model includes a letter sequence segmentation module and a phoneme sequence generation and segmentation module, and the method includes:

[0042] Step S10: Obtain training data, which includes multiple training words, real letter block sequences corresponding to the training words, and real phoneme block sequences corresponding to the training words. Each letter block in the real letter block sequence and each phoneme block in the real phoneme block sequence have a one-to-one correspondence.

[0043] The electronic device can download the training data over a network. It is understood that the training data is triplet data, in the form (x, y, z), where x can be a training word, y is the corresponding real letter block sequence, and z is the corresponding real phoneme block sequence. The real letter block sequence contains sequentially arranged letter blocks, and the real phoneme block sequence contains sequentially arranged phoneme blocks. For a training word, the number of letter blocks in its corresponding real letter block sequence and the number of phoneme blocks in its corresponding real phoneme block sequence are equal, and the corresponding letter blocks and phoneme blocks are in the same order in the sequence. For example, if the symbol "-" represents a separator and the symbol ":" represents a merge symbol, then for the training data (good, go:od, G-UH-D), the training word is "good". The letter blocks in its corresponding letter block sequence are "g", "o:o", and "d", and the phoneme block sequences are "G", "UH", and "D". The letter block "g" corresponds to the phoneme block "G", the letter block "o:o" corresponds to the phoneme block "UH", and the letter block "d" corresponds to the phoneme block "D".

[0044] Step S20: Input the training word into the letter sequence segmentation module 10 to obtain the predicted letter block sequence and feature matrix corresponding to the training word.

[0045] The letter sequence segmentation module 10 is a classification model, which can be an encoding module used to output letter sequences, letter separators, and merging symbols. For example... Figure 4 As shown, after obtaining training data, the electronic device can insert delimiters (such as "-") into each letter of a word and input them into the letter sequence segmentation module 10, causing the letter sequence segmentation module 10 to output a predicted letter block sequence. For example, for the training word "fume", the electronic device inserts the delimiter "-" to obtain the letter block sequence "fume", and inputs this letter block sequence "fume" into the letter sequence segmentation module 10 in sequence. It can be understood that the expected output letter block sequence is "fum:e". In practical applications, the delimiters and merge symbols can be set according to actual needs, and there is no need to be bound by the limitations in this example.

[0046] Step S30: Input the feature matrix into the phoneme sequence generation and segmentation module 20 to obtain the predicted phoneme block sequence corresponding to the training word.

[0047] The phoneme sequence generation and segmentation module 20 is a generation model, which can also be a decoding module, used to output phoneme sequences, phoneme separators, and merging symbols. Specifically, the letter sequence segmentation module 10 includes an input layer, a hidden layer, and an output layer. The feature matrix can be the output of the output layer or the output of the hidden layer. After obtaining the feature matrix, the electronic device can input the feature matrix into the phoneme sequence generation and segmentation module 20, causing the phoneme sequence generation and segmentation module 20 to output a predicted phoneme block sequence.

[0048] Specifically, in some embodiments, this step includes: simultaneously inputting the feature matrix and the output of the phoneme sequence generation and segmentation module 20 from the previous time step into the phoneme sequence generation and segmentation module 20 to obtain the output of the next time step, and repeating this process until an end symbol (such as "") is encountered or the maximum length set for the generated sequence is reached. In the initial time step, the output of the phoneme sequence generation and segmentation module 20 from the previous time step is the initial symbol, such as "". <os>".

[0049] For example, please see Figure 4 The letter sequence segmentation module 10 outputs "fum:e" in sequence. Therefore, the phoneme sequence generation and segmentation module 20 will combine the initial symbols at the initial moment. <os>The feature matrix output by the letter sequence segmentation module 10 is used to perform an attention mechanism to output the phoneme "F"; then, the phoneme sequence is generated and compared with the "" output by the segmentation module 20. <os>After performing a self-attention mechanism on " and "F", it is then combined with the feature matrix output by the letter sequence segmentation module 10 to output "-"; then, the phoneme sequence generation is combined with the "" output by the segmentation module 20. <os> After the letters "", "F", and "-" undergo a self-attention mechanism, they are then combined with the feature matrix output by the letter sequence segmentation module 10 to output "Y". This process continues until the phoneme sequence is generated and the segmentation module 20 outputs the end symbol.< / os> Or it will end when the maximum length set for generating the sequence is reached.

[0050] Step S40: Based on the predicted letter block sequence, predicted phoneme block sequence, real letter block sequence, and real phoneme block sequence, iteratively train the word-to-phoneme model until the preset conditions are met, and obtain the trained word-to-phoneme model.

[0051] Finally, the word-to-phoneme model is trained using the predicted letter block sequence and the real letter block sequence, as well as the predicted phoneme block sequence and the real phoneme block sequence. The parameters of the word-to-phoneme model are continuously adjusted until the preset conditions are met, such as the word-to-phoneme model converging or the predicted letter block sequence and the predicted phoneme block sequence output by the word-to-phoneme model becoming closer and closer to the real letter block sequence. At this point, the parameters are used as the parameters of the word-to-phoneme model, thus obtaining the trained word-to-phoneme model.

[0052] In the training method provided in this application embodiment, the letter block sequence and phoneme block sequence output by the word conversion model are jointly trained using triplet data. This enables the subsequently trained word conversion model to output both letter block sequences and phoneme block sequences, with a one-to-one correspondence between the letter blocks in the letter block sequence and the phoneme blocks in the phoneme block sequence. Furthermore, the letter sequence segmentation module 10 can assist in the letter-to-phoneme conversion task, further improving the accuracy of the phoneme sequence generation and segmentation module 20 in generating phoneme block sequences. Subsequent applications in speech synthesis, speech recognition, and speech evaluation systems can enhance the accuracy of these processes.

[0053] In some embodiments, the word-to-phoneme conversion model includes one of the following: Long Short-Term Memory (LSTM) network, Convolutional Neural Networks (CNN), attention mechanism model, and converter model. The specific structure of the above models can be found in existing technologies and is not limited thereto. In this application, the specific structure of the word-to-phoneme conversion model can be selected according to actual needs, thereby improving the flexibility and freedom of model construction.

[0054] In some embodiments, acquiring training data includes:

[0055] Step S11: Obtain multiple training words and the phoneme block sequence corresponding to each training word.

[0056] Step S12: Based on multiple training words, obtain the letter block sequence corresponding to each training word.

[0057] Step S13: Obtain the spelling rules, and modify the letter block sequence and phoneme block sequence based on the spelling rules to obtain the real letter block sequence and real phoneme block sequence corresponding to each training word;

[0058] Step S14: Obtain training data based on multiple training words, the real phoneme block sequence corresponding to each training word, and the real letter block sequence corresponding to each training word.

[0059] The electronic device can obtain multiple training words and their corresponding phoneme block sequences from a dictionary; this information is essentially binary data. Next, the electronic device can call the open-source tool m2m-aligner to obtain triplet data, which includes multiple training words, their corresponding letter block sequences, and their corresponding phoneme block sequences. However, to further improve the accuracy of the triplet data, in this embodiment, the electronic device also obtains phonics rules to correct the triplet data obtained from the open-source tool, resulting in corrected triplet data, which includes multiple training words, their corresponding actual letter block sequences, and their corresponding actual phoneme block sequences. For example, the triplet data obtained after the m2m-aligner for the word "night" is: (night, ni:gh:t, N-AY-T), where the letter block sequence corresponding to the word "night" is "ni:gh:t" and the phoneme block sequence corresponding to the word "night" is "N-AY-T". However, in the spelling rules, the letter "igh" is pronounced as a whole. Therefore, after correction using the spelling rules, the corrected triplet data is (night, ni:g:ht, N-AY-T), where the actual letter block sequence corresponding to the word "night" is "ni:gh:t" and the actual phoneme block sequence corresponding to the word "night" is "N-AY-T".

[0060] As can be seen, in this embodiment, adjustments can be made to the letter block sequences and phoneme block sequences output by the m2m-aligner model that do not conform to spelling habits, thereby improving the accuracy of the training data. This ensures that the correspondence between letter blocks in the generated letter block sequences and phoneme blocks in the generated phoneme block sequences of the subsequently trained word-to-phoneme model is more in line with linguistic spelling habits, improving the readability and intelligibility of the letter block and phoneme block alignment. Furthermore, compared to manually annotating binary data into triple data, this method improves annotation speed and efficiency.

[0061] In some embodiments, the word-to-phoneme model is iteratively trained based on the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met, resulting in a trained word-to-phoneme model, including:

[0062] Step S41: Based on the loss function, obtain the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence, and iteratively adjust the parameters of the word-to-phoneme model until the preset conditions are met to obtain the trained word-to-phoneme model.

[0063] The parameters of the word-to-phoneme model can be iteratively optimized using an algorithm, with the number of iterations potentially set to 100,000. In this embodiment, training the model using an iterative algorithm yields the various parameters of the word-to-phoneme model, resulting in the trained word-to-phoneme model.

[0064] Specifically, the loss function includes an encoding loss function, a decoding loss function, and a total loss function. Based on the loss function, the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence is obtained. This includes: obtaining the encoding loss based on the predicted letter block sequence, the real letter block sequence, and the encoding loss function; obtaining the decoding loss based on the predicted phoneme block sequence, the real phoneme block sequence, and the decoding loss function; and obtaining the total loss based on the encoding loss, the decoding loss, and the total loss function.

[0065] The encoding loss function and decoding loss function can use the cross-entropy loss function. The total loss = encoding loss + decoding loss, that is, when calculating the total loss, the weight ratio of encoding loss and decoding loss is 1:1.

[0066] As can be seen, in this embodiment, using the cross-entropy loss function as the loss function can accelerate the model convergence speed and reduce the convergence difficulty. Furthermore, using letter block sequences and phoneme block sequences as equally important training reference factors can improve the accuracy of word-to-phoneme detection after training.

[0067] This application also provides a method for converting words into phonemes. Please refer to [link to relevant documentation]. Figure 5 The phoneme conversion method for this word includes:

[0068] Step S100: Obtain the word to be converted;

[0069] Step S200: Input the word to be converted into the word conversion phoneme model to obtain the corresponding letter block sequence and phoneme block sequence. Each letter block in the letter block sequence has a one-to-one correspondence with each phoneme block in the phoneme block sequence. The word conversion phoneme model is trained using the training method described in any embodiment of the first aspect.

[0070] The electronic device has a built-in word-to-phoneme conversion application. The word-to-phoneme conversion model is encapsulated within this application. The application calls the word-to-phoneme conversion model to convert the word to be converted (with inserted separators). After a series of calculations, it outputs a predicted sequence of letter blocks and a sequence of phoneme blocks, with a one-to-one correspondence between each letter block in the letter block sequence and each phoneme block in the phoneme block sequence. It is understood that this word-to-phoneme conversion model is trained using the method described in the above embodiment for training the word-to-phoneme detection model, and has the same structure and function as the word-to-phoneme conversion model described in the above embodiment; therefore, it will not be described in detail here.

[0071] This application also provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, executing the method for training a word-to-phoneme model and the word-to-phoneme method described in the above embodiments. The computer-readable storage medium can be a storage medium such as FRAM, ROM, PROM, EPROM, EEE PROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; it can also be various devices including one or any combination of the above-mentioned memories. The executable instructions can take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0072] This application also provides a computer program product, including a computing program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the method for training a word-to-phoneme model and the word-to-phoneme method described in the above embodiments.

[0073] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for at least one computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.< / os> < / os> < / os>

Claims

1. A training method for a word-to-phoneme model, characterized in that, The word-to-phoneme model includes a letter sequence segmentation module and a phoneme sequence generation and segmentation module, and the method includes: Obtain multiple training words and the phoneme block sequences corresponding to each training word; Based on the multiple training words, a sequence of letter blocks corresponding to each training word is obtained; Obtain the spelling rules, and modify the letter block sequence and the phoneme block sequence based on the spelling rules to obtain the real letter block sequence and the real phoneme block sequence corresponding to each training word. Each letter block in the real letter block sequence and each phoneme block in the real phoneme block sequence have a one-to-one correspondence. The training word is input into the letter sequence segmentation module to obtain the predicted letter block sequence and feature matrix corresponding to the training word; The feature matrix is ​​input into the phoneme sequence generation and segmentation module to obtain the predicted phoneme block sequence corresponding to the training word; The word-to-phoneme model is iteratively trained based on the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met, resulting in the trained word-to-phoneme model.

2. The training method according to claim 1, characterized in that, The letter sequence segmentation module is an encoding module, and the phoneme sequence generation and segmentation module is a decoding module.

3. The training method according to claim 2, characterized in that, The word-to-phoneme model includes one of the following: Long Short-Term Memory Network, Convolutional Neural Network, Attention Mechanism Model, and Converter Model.

4. The training method according to claim 3, characterized in that, The letter sequence segmentation module includes an input layer, a hidden layer, and an output layer, and the feature matrix is ​​the output of the output layer or the output of the hidden layer.

5. The training method according to any one of claims 1-4, characterized in that, The step of iteratively training the word-to-phoneme model based on the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence until a preset condition is met, to obtain the trained word-to-phoneme model, includes: The total loss among the predicted letter block sequence, the predicted phoneme block sequence, the real letter block sequence, and the real phoneme block sequence is obtained based on the loss function. The parameters of the word-to-phoneme model are iteratively adjusted until the preset condition is met, thus obtaining the trained word-to-phoneme model.

6. The training method according to claim 5, characterized in that, The loss function includes an encoding loss function and a decoding loss function. The step of obtaining the total loss among the predicted letter block sequence, the predicted phoneme block sequence, the true letter block sequence, and the true phoneme block sequence based on the loss function includes: The coding loss is obtained based on the predicted letter block sequence, the actual letter block sequence, and the coding loss function; The decoding loss is obtained based on the predicted phoneme block sequence, the actual phoneme block sequence, and the decoding loss function; The total loss is obtained based on the encoding loss and the decoding loss.

7. A method for converting words into phonemes, characterized in that, include: Get the words to be converted; The word to be converted is input into the word conversion phoneme model to obtain the corresponding letter block sequence and phoneme block sequence. Each letter block in the letter block sequence has a one-to-one correspondence with each phoneme block in the phoneme block sequence. The word conversion phoneme model is trained using the training method described in any one of claims 1-6.

8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 7.

Images