A method for constructing a vocabulary splitting model, a method for constructing a speech recognition model

Abstract

The present application discloses a method for constructing a vocabulary splitting model and a method for constructing a speech recognition model. First, the present application uses a large number of sample vocabularies, pronunciation description information of these sample vocabularies, and splitting annotation information of these sample vocabularies to construct a vocabulary splitting model, so that the constructed vocabulary splitting model can learn the ability to perform vocabulary splitting processing based on vocabulary pronunciation; then, using the constructed vocabulary splitting model and a large number of minimum semantic units in the target language, a sub-word dictionary in the target language is constructed; then, with the help of this sub-word dictionary, a speech recognition model in the target language is constructed; finally, using this speech recognition model to perform speech recognition processing on any speech data in the target language, which is beneficial to improving the speech recognition effect in the target language.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to a method for constructing a vocabulary splitting model, a vocabulary splitting method, a sub-word dictionary construction method, a speech recognition model construction method, and a speech recognition method. Background Art

[0002] For a speech recognition model in a target language (e.g., English), the speech recognition model can be used to determine a one-hot encoded vector corresponding to a certain speech data in the target language, so that the one-hot encoded vector can represent the speech information carried by the speech data (e.g., comfortable, etc.).

[0003] In fact, the dimension of the one-hot encoded vector usually equals the total number of the smallest semantic units (e.g., English words) in the target language. However, when the total number of the smallest semantic units in the target language is too large, it is likely to affect the speech recognition ability of the speech recognition model in the target language. For the sake of easy understanding, the following will be described in conjunction with English.

[0004] As an example, for English, the smallest semantic unit of English is an English word, so that the dimension of the one-hot encoded vector in English usually equals the number of English words. However, due to the large amount of data of English words, the dimension of the one-hot encoded vector in English is relatively high, which causes some difficult-to-overcome challenges in the construction process and the use process of the speech recognition model in English (e.g., the number of parameters of the speech recognition model is too large, it is difficult for the speech recognition model to converge, etc.). Thus, it is likely to result in a relatively poor speech recognition effect in the target language. Summary of the Invention

[0005] The main purpose of the embodiments of the present application is to provide a method for constructing a vocabulary splitting model, a vocabulary splitting method, a sub-word dictionary construction method, a speech recognition model construction method, and a speech recognition method, which can improve the speech recognition effect in the target language.

[0006] The embodiments of the present application provide a method for constructing a vocabulary splitting model, and the method includes:

[0007] Obtain sample vocabulary in the target language, pronunciation description information of the sample vocabulary, and splitting annotation information of the sample vocabulary;

[0008] Use the sample vocabulary, the pronunciation description information of the sample vocabulary, and the splitting annotation information of the sample vocabulary to construct a vocabulary splitting model; wherein, the vocabulary splitting model is used to perform vocabulary splitting processing on the sample vocabulary with reference to the pronunciation description information of the sample vocabulary.

[0009] In a possible implementation, the pronunciation description information includes at least one of pronunciation annotation information and pronunciation audio data.

[0010] In a possible implementation, constructing a vocabulary splitting model by using the sample vocabulary, the pronunciation description information of the sample vocabulary, and the splitting annotation information of the sample vocabulary includes:

[0011] Determine the vocabulary description data of the sample vocabulary according to the sample vocabulary and the pronunciation description information of the sample vocabulary;

[0012] Input the vocabulary description data of the sample vocabulary into the model to be trained, and obtain the predicted splitting result of the sample vocabulary output by the model to be trained;

[0013] Update the model to be trained according to the predicted splitting result of the sample vocabulary and the splitting annotation information of the sample vocabulary, and continue to execute the step of inputting the vocabulary description data of the sample vocabulary into the model to be trained to obtain the predicted splitting result of the sample vocabulary output by the model to be trained, until after reaching a preset stop condition, determine the vocabulary splitting model according to the model to be trained.

[0014] In a possible implementation, the determining the vocabulary description data of the sample vocabulary according to the sample vocabulary and the pronunciation description information of the sample vocabulary includes:

[0015] Split the sample vocabulary according to the smallest writing unit in the target language to obtain the vocabulary writing sequence of the sample vocabulary;

[0016] Extract the vocabulary phoneme features of the sample vocabulary from the pronunciation description information of the sample vocabulary;

[0017] Perform set processing on the vocabulary writing sequence of the sample vocabulary and the vocabulary phoneme features of the sample vocabulary to obtain the vocabulary description data of the sample vocabulary.

[0018] In a possible implementation, the pronunciation description information includes pronunciation annotation information and pronunciation audio data;

[0019] The extracting the vocabulary phoneme features of the sample vocabulary from the pronunciation description information of the sample vocabulary includes:

[0020] Perform character extraction processing on the pronunciation annotation information of the sample vocabulary to obtain the vocabulary phonetic symbol sequence of the sample vocabulary;

[0021] Perform acoustic feature extraction processing on the pronunciation audio data of the sample vocabulary to obtain the vocabulary audio features of the sample vocabulary;

[0022] Perform a set process on the phonetic symbol sequence of the sample vocabulary and the vocabulary audio features of the sample vocabulary to obtain the vocabulary phoneme features of the sample vocabulary.

[0023] In a possible implementation manner, the model to be trained includes a feature extraction layer, an encoding layer, a decoding layer, and a decision-making layer;

[0024] The process of determining the predicted splitting result of the sample vocabulary includes:

[0025] Input the vocabulary description data of the sample vocabulary into the feature extraction layer to obtain the feature extraction result output by the feature extraction layer;

[0026] Input the feature extraction result into the encoding layer to obtain the feature encoding result output by the encoding layer;

[0027] Input the feature encoding result into the decoding layer to obtain the feature decoding result output by the decoding layer;

[0028] Input the feature decoding result into the decision-making layer to obtain the predicted splitting result of the sample vocabulary output by the decision-making layer.

[0029] In a possible implementation manner, the feature extraction layer includes a writing feature extraction module, a phoneme feature extraction module, and a feature processing module;

[0030] The process of determining the feature extraction result includes:

[0031] Determine the vocabulary writing feature and the vocabulary pronunciation feature according to the vocabulary description data of the sample vocabulary, the writing feature extraction module, and the phoneme feature extraction module;

[0032] Input the vocabulary writing feature and the vocabulary pronunciation feature into the feature processing module to obtain the feature extraction result output by the feature processing module.

[0033] In a possible implementation manner, the vocabulary description data includes a vocabulary writing sequence and vocabulary phoneme features; wherein, the vocabulary writing sequence is used to describe the writing characteristics of the sample vocabulary; the vocabulary phoneme features are used to describe the pronunciation characteristics of the sample vocabulary;

[0034] The determining of the vocabulary writing feature and the vocabulary pronunciation feature according to the vocabulary description data of the sample vocabulary, the writing feature extraction module, and the phoneme feature extraction module includes:

[0035] Input the vocabulary writing sequence into the writing feature extraction module to obtain the vocabulary writing feature output by the writing feature extraction module;

[0036] Input the lexical phoneme features into the phoneme feature extraction module to obtain the lexical pronunciation features output by the phoneme feature extraction module.

[0037] In a possible implementation manner, the process of determining the feature extraction result includes:

[0038] Concatenate the lexical writing features and the lexical pronunciation features to obtain the concatenated features to be used;

[0039] Perform position encoding processing on the concatenated features to be used to obtain the position encoding features corresponding to the concatenated features to be used;

[0040] Fuse the concatenated features to be used and the position encoding features corresponding to the concatenated features to be used to obtain the feature extraction result.

[0041] In a possible implementation manner, the method further includes:

[0042] Determine the size description information to be used and the feature concatenation direction to be used according to the size information of the lexical writing features and the size information of the lexical pronunciation features;

[0043] Determine the writing features to be used and the pronunciation features to be used according to the size description information to be used, the lexical writing features, and the lexical pronunciation features, so that the size information of the writing features to be used is consistent with the size information of the pronunciation features to be used;

[0044] The step of concatenating the lexical writing features and the lexical pronunciation features to obtain the concatenated features to be used includes:

[0045] Concatenate the writing features to be used and the pronunciation features to be used according to the feature concatenation direction to be used to obtain the concatenated features to be used.

[0046] The embodiments of the present application further provide a method for splitting a word, and the method includes:

[0047] Obtain the word to be split in the target language and the pronunciation description information of the word to be split;

[0048] Determine the splitting result of the word to be split according to the word to be split, the pronunciation description information of the word to be split, and a pre-constructed word splitting model; wherein, the word splitting model is constructed by using any implementation manner of the word splitting model construction method provided by the embodiments of the present application.

[0049] The embodiments of the present application further provide a method for constructing a sub-word dictionary, and the method includes:

[0050] Obtain at least one candidate word in the target language, and pronunciation description information of each of the candidate words;

[0051] Determine the splitting result of each candidate word according to each candidate word, the pronunciation description information of each candidate word, and a pre-constructed word splitting model; wherein, the word splitting model is constructed by any implementation manner of the word splitting model construction method provided in the embodiments of the present application;

[0052] Determine at least one sub-word corresponding to each candidate word according to the splitting result of each candidate word;

[0053] Perform statistical analysis processing on the at least one sub-word corresponding to the at least one candidate word to obtain a sub-word dictionary in the target language.

[0054] The embodiments of the present application also provide a method for constructing a speech recognition model, and the method includes:

[0055] Obtain a sample speech in the target language and the actual speech content of the sample speech;

[0056] Determine the speech content annotation of the sample speech according to the sub-word dictionary in the target language and the actual speech content of the sample speech, so that the data dimension of the speech content annotation is equal to the number of sub-words in the sub-word dictionary; wherein, the sub-word dictionary is constructed by any implementation manner of the sub-word dictionary construction method provided in the embodiments of the present application;

[0057] Construct a speech recognition model by using the sample speech and the speech content annotation of the sample speech.

[0058] The embodiments of the present application also provide a speech recognition method, and the method includes:

[0059] Obtain a speech to be recognized in the target language;

[0060] Perform speech recognition processing on the speech to be recognized by using a pre-constructed speech recognition model to obtain a speech recognition result of the speech to be recognized; wherein, the speech recognition model is constructed by any implementation manner of the speech recognition model construction method provided in the embodiments of the present application.

[0061] The embodiments of the present application also provide a device for constructing a word splitting model, including:

[0062] A first obtaining unit, configured to obtain a sample word in the target language, pronunciation description information of the sample word, and splitting annotation information of the sample word;

[0063] A first construction unit, configured to construct a vocabulary splitting model by using the sample vocabulary, the pronunciation description information of the sample vocabulary, and the splitting annotation information of the sample vocabulary; wherein, the vocabulary splitting model is used to perform vocabulary splitting processing on the sample vocabulary with reference to the pronunciation description information of the sample vocabulary.

[0064] An embodiment of the present application further provides a vocabulary splitting device, including:

[0065] A second acquisition unit, configured to acquire a vocabulary to be split in a target language, and pronunciation description information of the vocabulary to be split;

[0066] A first splitting unit, configured to determine a splitting result of the vocabulary to be split according to the vocabulary to be split, the pronunciation description information of the vocabulary to be split, and a pre-constructed vocabulary splitting model; wherein, the vocabulary splitting model is constructed by using any implementation manner of the vocabulary splitting model construction method provided by the embodiment of the present application.

[0067] An embodiment of the present application further provides a sub-word dictionary construction device, including:

[0068] A third acquisition unit, configured to acquire at least one candidate vocabulary in a target language, and pronunciation description information of each candidate vocabulary;

[0069] A second splitting unit, configured to determine a splitting result of each candidate vocabulary according to each candidate vocabulary, the pronunciation description information of each candidate vocabulary, and a pre-constructed vocabulary splitting model; wherein, the vocabulary splitting model is constructed by using any implementation manner of the vocabulary splitting model construction method provided by the embodiment of the present application;

[0070] A sub-word determination unit, configured to determine at least one sub-word corresponding to each candidate vocabulary according to the splitting results of each candidate vocabulary;

[0071] A dictionary construction unit, configured to perform statistical analysis processing on at least one sub-word corresponding to the at least one candidate vocabulary to obtain a sub-word dictionary in the target language.

[0072] An embodiment of the present application further provides a speech recognition model construction device, including:

[0073] A fourth acquisition unit, configured to acquire a sample speech in a target language and the actual speech content of the sample speech;

[0074] A speech annotation unit, configured to determine a speech content annotation of the sample speech according to a sub - word dictionary in the target language and the actual speech content of the sample speech, so that the data dimension of the speech content annotation is equal to the number of sub - words in the sub - word dictionary; wherein, the sub - word dictionary is constructed by any implementation manner of the sub - word dictionary construction method provided in the embodiments of the present application;

[0075] A second construction unit, configured to construct a speech recognition model by using the sample speech and the speech content annotation of the sample speech.

[0076] Embodiments of the present application further provide a speech recognition device, including:

[0077] A fifth acquisition unit, configured to acquire a speech to be recognized in the target language;

[0078] A speech recognition unit, configured to perform speech recognition processing on the speech to be recognized by using a pre - constructed speech recognition model, so as to obtain a speech recognition result of the speech to be recognized; wherein, the speech recognition model is constructed by any implementation manner of the speech recognition model construction method provided in the embodiments of the present application.

[0079] Embodiments of the present application further provide a device, the device includes: a processor, a memory, and a system bus;

[0080] The processor and the memory are connected through the system bus;

[0081] The memory is used to store one or more programs, the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes any implementation manner of the vocabulary splitting model construction method provided in the embodiments of the present application, or executes any implementation manner of the vocabulary splitting method provided in the embodiments of the present application, or executes any implementation manner of the sub - word dictionary construction method provided in the embodiments of the present application, or executes any implementation manner of the speech recognition model construction method provided in the embodiments of the present application, or executes any implementation manner of the speech recognition method provided in the embodiments of the present application.

[0082] Embodiments of the present application further provide a computer - readable storage medium, in which instructions are stored, and when the instructions run on a terminal device, the terminal device executes any implementation manner of the vocabulary splitting model construction method provided in the embodiments of the present application, or executes any implementation manner of the vocabulary splitting method provided in the embodiments of the present application, or executes any implementation manner of the sub - word dictionary construction method provided in the embodiments of the present application, or executes any implementation manner of the speech recognition model construction method provided in the embodiments of the present application, or executes any implementation manner of the speech recognition method provided in the embodiments of the present application.

[0083] The embodiment of the present application also provides a computer program product. When the computer program product runs on a terminal device, the terminal device is caused to execute any implementation manner of the vocabulary splitting model construction method provided by the embodiment of the present application, or execute any implementation manner of the vocabulary splitting method provided by the embodiment of the present application, or execute any implementation manner of the sub-word dictionary construction method provided by the embodiment of the present application, or execute any implementation manner of the speech recognition model construction method provided by the embodiment of the present application, or execute any implementation manner of the speech recognition method provided by the embodiment of the present application.

[0084] Based on the above technical solutions, the present application has the following beneficial effects:

[0085] In the technical solution provided by this application, for a target language (e.g., English), first, a large number of sample words in the target language, pronunciation description information of these sample words, and splitting annotation information of these sample words are used to construct a word splitting model, so that the word splitting model can learn the ability to perform word splitting processing based on the pronunciation of words, thereby enabling the word splitting model to perform word splitting processing on any word (e.g., sample words, candidate words, etc.) with reference to the pronunciation description information of the word; then, the constructed word splitting model is used to perform word splitting processing on a large number of candidate words to obtain sub-words obtained by splitting these candidate words, and these sub-words obtained by splitting the candidate words are used to construct a sub-word dictionary in the target language, so that a large number of sub-words recorded in the sub-word dictionary can cover the pronunciation units in the target language as much as possible; then, the sub-word dictionary is used to automatically annotate the actual speech content of a large number of sample speeches to obtain speech content annotations of these sample speeches, so that these speech content annotations can represent the speech information actually carried by these sample speeches with a data vector in the data representation space constructed by the sub-word dictionary; secondly, these sample speeches and their speech content annotations are used to construct a speech recognition model, so that the speech recognition model can not only learn the ability to perform speech recognition on speech data, but also learn the ability to express speech recognition results using the sub-word dictionary; finally, the constructed speech recognition model is used to perform speech recognition processing on the speech to be recognized in the target language to obtain the speech recognition result of the speech to be recognized, so that the speech recognition result can represent the speech information carried by the speech to be recognized with a data vector in the data representation space constructed by the sub-word dictionary, thereby making the data dimension of the speech recognition result equal to the number of sub-words in the sub-word dictionary, and further making the dimension of the speech recognition result much smaller than the total number of the smallest semantic units (e.g., English words) in the target language, so as to overcome the defect that the speech recognition ability is relatively poor due to the too large total number of the smallest semantic units in the target language, thereby improving the speech recognition ability of the speech recognition model and further improving the speech recognition effect in the target language. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0087] Figure 1 It is a flowchart of a method for constructing a word splitting model provided by an embodiment of this application;

[0088] Figure 2 Structural schematic diagram of a model to be trained provided by an embodiment of the present application;

[0089] Figure 3 Schematic diagram of the working principle of a model to be trained provided by an embodiment of the present application;

[0090] Figure 4 Schematic diagram of a splicing method provided by an embodiment of the present application;

[0091] Figure 5 Schematic diagram of another splicing method provided by an embodiment of the present application;

[0092] Figure 6 Flowchart of a vocabulary splitting method provided by an embodiment of the present application;

[0093] Figure 7 Flowchart of a sub-word dictionary construction method provided by an embodiment of the present application;

[0094] Figure 8 Flowchart of a speech recognition model construction method provided by an embodiment of the present application;

[0095] Figure 9 Flowchart of a speech recognition method provided by an embodiment of the present application;

[0096] Figure 10 Structural schematic diagram of a vocabulary splitting model construction device provided by an embodiment of the present application;

[0097] Figure 11 Structural schematic diagram of a vocabulary splitting device provided by an embodiment of the present application;

[0098] Figure 12 Structural schematic diagram of a sub-word dictionary construction device provided by an embodiment of the present application;

[0099] Figure 13 Structural schematic diagram of a speech recognition model construction device provided by an embodiment of the present application;

[0100] Figure 14 Structural schematic diagram of a speech recognition device provided by an embodiment of the present application. Detailed implementation manners

[0101] In the research on speech recognition models, the inventors found that some speech recognition models (e.g., end-to-end speech recognition models, etc.) can usually classify the acoustic extraction features of a speech data into its corresponding one-hot encoded vector with the help of a classification network (e.g., classification neural network, etc.). However, since the dimension of the one-hot encoded vector usually equals the total number of the smallest semantic units in the target language (e.g., English words), when the total number of the smallest semantic units in the target language is too large, it is likely to result in poor speech recognition ability of the speech recognition model in the target language. For ease of understanding, the following will be described with examples.

[0102] As an example, for English, the smallest semantic unit of English is an English word, and the number of English words is extremely large. Based on this, it can be known that if each English word is used as a classification, the dimension of the one-hot encoded vector in English will equal the number of English words, thus making the dimension of the one-hot encoded vector in English relatively high. This will not only pose a huge challenge to the classification ability of the classification network in the speech recognition model, but also greatly increase the number of network parameters in this classification network, thereby easily resulting in poor speech recognition ability of the speech recognition model in the target language. In addition, when a new word (e.g., a new portmanteau word, etc.) appears, it is likely to affect the speech recognition ability of this speech recognition model because the existing classification nodes in this classification network cannot correctly classify this new word.

[0103] The inventors also found that the number of classifications of the above classification network can be significantly reduced by adopting a splitting method for the smallest semantic units in the target language. For example, for English words, each English word (e.g., comfortable) can be split into multiple sub-words (e.g., com, for, ta, ble), and there will be a large number of repetitions among the sub-words of different English words. In this way, the number of these sub-words is much smaller than the number of these English words. Therefore, when constructing the classification nodes of the above classification network with these sub-words, the number of classification nodes of the classification network in the speech recognition model can be effectively reduced, and thus the number of network parameters in this classification network can be effectively reduced.

[0104] Based on the above findings, to solve the technical problems shown in the background art section, the embodiments of the present application provide a method for constructing a vocabulary splitting model, a vocabulary splitting method, a sub-word dictionary construction method, a speech recognition model construction method, and a speech recognition method; and the technical solutions disclosed in the present application are specifically as follows: For a target language (for example, English), first, use a large number of sample words in the target language, the pronunciation description information of these sample words, and the splitting annotation information of these sample words to construct a vocabulary splitting model, so that the vocabulary splitting model can learn the ability to perform vocabulary splitting processing based on the pronunciation of words; then, use the constructed vocabulary splitting model to perform vocabulary splitting processing on a large number of candidate words, obtain the sub-words obtained by splitting these candidate words, and use the sub-words obtained by splitting these candidate words to construct a sub-word dictionary in the target language, so that the large number of sub-words recorded in the sub-word dictionary can cover the pronunciation units in the target language as much as possible; then, use the sub-word dictionary to automatically annotate the actual speech content of a large number of sample speeches, obtain the speech content annotations of these sample speeches, so that these speech content annotations can represent the speech information actually carried by these sample speeches with a data vector in the data representation space constructed by the sub-word dictionary; secondly, use these sample speeches and their speech content annotations to construct a speech recognition model, so that the speech recognition model can not only learn the ability to perform speech recognition on speech data, but also learn the ability to express the speech recognition results using the sub-word dictionary; finally, use the constructed speech recognition model to perform speech recognition processing on the speech to be recognized in the target language, obtain the speech recognition result of the speech to be recognized, so that the speech recognition result can represent the speech information carried by the speech to be recognized with a data vector in the data representation space constructed by the sub-word dictionary, so that the data dimension of the speech recognition result is equal to the number of sub-words in the sub-word dictionary, and further make the dimension of the speech recognition result much smaller than the total number of the smallest semantic units in the target language (for example, English words), so as to overcome the defect that the speech recognition ability is relatively poor due to the too large total number of the smallest semantic units in the target language, thereby improving the speech recognition ability of the speech recognition model, and further improving the speech recognition effect in the target language.

[0105] In addition, since the sub-word dictionary in the target language can cover the pronunciation units in the target language, the sub-word dictionary can represent the pronunciation situations of various character combinations in the target language (for example, English letter combinations, etc.), so that the speech recognition model constructed based on the sub-word dictionary can not only perform speech recognition processing on the existing words in the target language, but also perform speech recognition processing on the newly added words in the target language (for example, newly added concatenated compound words, etc.), which is beneficial to improving the speech recognition effect of the speech recognition model for newly added words, and thus beneficial to improving the speech recognition effect in the target language.

[0106] In addition, the execution subject of the method for constructing a vocabulary splitting model in the embodiments of the present application is not limited. For example, the method for constructing a vocabulary splitting model provided by the embodiments of the present application can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device can be a smart phone, a computer, a personal digital assistant (PDA), or a tablet computer, etc. The server can be an independent server, a cluster server, or a cloud server.

[0107] Similarly, the execution subject of the vocabulary splitting method in the embodiments of the present application is not limited. For example, the vocabulary splitting method provided by the embodiments of the present application can be applied to data processing devices such as terminal devices or servers.

[0108] Similarly, the execution subject of the method for constructing a sub-word dictionary in the embodiments of the present application is not limited. For example, the method for constructing a sub-word dictionary provided by the embodiments of the present application can be applied to data processing devices such as terminal devices or servers.

[0109] Similarly, the execution subject of the method for constructing a speech recognition model in the embodiments of the present application is not limited. For example, the method for constructing a speech recognition model provided by the embodiments of the present application can be applied to data processing devices such as terminal devices or servers.

[0110] Similarly, the execution subject of the speech recognition method in the embodiments of the present application is not limited. For example, the speech recognition method provided by the embodiments of the present application can be applied to data processing devices such as terminal devices or servers.

[0111] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0112] To facilitate the understanding of the technical solutions of the present application, the relevant contents of the method for constructing a vocabulary splitting model, the vocabulary splitting method, the method for constructing a sub-word dictionary, the method for constructing a speech recognition model, and the speech recognition method will be introduced in sequence below.

[0113] Method Embodiment 1

[0114] See Figure 1 , which is a flowchart of a method for constructing a vocabulary splitting model provided by an embodiment of the present application.

[0115] The method for constructing a vocabulary splitting model provided by the embodiments of the present application includes S101 - S102:

[0116] S101: Obtain sample words in the target language, pronunciation description information of the sample words, and split annotation information of the sample words.

[0117] Among them, the target language refers to the language type for which speech recognition performance can be improved by means of word splitting processing; and the embodiments of the present application do not limit the target language. For example, the target language can be English.

[0118] In addition, the smallest semantic unit in the target language is represented in the form of a string; and the smallest semantic unit in the target language can be split into multiple pronunciation units (also known as sub-words) with specific pronunciations. For example, when the target language is English, the smallest semantic unit in the target language is an English word. Among them, an English word (such as, comfortable, etc.) is a string composed of multiple English letters (such as, c, o, m, f, o, r, t, a, b, and e); and the English word can be split into four pronunciation units with specific pronunciations, namely com, for, ta, and ble.

[0119] In addition, the number of the smallest semantic units in the target language is much higher than the number of pronunciation units (that is, the number of sub-words) with specific pronunciations in the target language.

[0120] The above-mentioned "sample words" refer to the smallest semantic units in the target language required for constructing a word splitting model. For example, when the target language is English, the sample words can be English words.

[0121] In addition, the embodiments of the present application do not limit the number of sample words. For example, it can be set according to the application scenario.

[0122] In addition, the embodiments of the present application do not limit the process of obtaining sample words. For example, specifically, it can be: select some words from various pronunciation word classes of the target language as sample words, so that a large number of sample words required for constructing a word splitting model can cover all pronunciation situations in the target language as much as possible. Among them, the words in different pronunciation word classes of the target language have different pronunciation situations. For the sake of easy understanding, the following takes English as an example for illustration.

[0123] As an example, when the target language is English, the above-mentioned "process of obtaining sample words" can be specifically as follows: Select some words from the basic word classes of English, the tense inflected word classes of this English, the foreign cultural word classes of this English, and the concatenated compound word classes of this English, etc., which are pronunciation word classes used to represent different pronunciation situations, as sample words, so that a large number of sample words required for constructing a word splitting model in English can cover all pronunciation word classes in this target language as much as possible, thereby enabling these sample words to cover all pronunciation situations in this target language as much as possible, which is conducive to improving the construction effect of the word splitting model.

[0124] It should be noted that the above-mentioned "various pronunciation word classes of the target language" can be statistically collected in advance from a large number of corpus texts in this target language; and the embodiments of the present application do not limit the statistical collection method of the "various pronunciation word classes of the target language".

[0125] The above-mentioned "pronunciation description information of the sample word" is used to describe the pronunciation situation of this sample word; and the embodiments of the present application do not limit the "pronunciation description information of the sample word". For example, it may include at least one of the pronunciation annotation information of this sample word and the pronunciation audio data of this sample word.

[0126] The above-mentioned "pronunciation annotation information of the sample word" is used to represent the pronunciation situation of this sample word in the form of a pronunciation annotation string; and the embodiments of the present application do not limit the "pronunciation annotation information of the sample word". For example, when the target language is English, the pronunciation annotation information of this sample word may be the phonetic symbol of an English word (for example, the phonetic symbol of the English word comfortable is ).

[0127] In addition, the embodiments of the present application do not limit the data format of the above-mentioned "pronunciation annotation information of the sample word". For example, it may be text data (for example, the phonetic symbol of an English word presented in text form, etc.), or it may be image data (for example, the phonetic symbol of an English word presented in image form, etc.).

[0128] In addition, the embodiments of the present application do not limit the acquisition method of the above-mentioned "pronunciation annotation information of the sample word". For example, the pronunciation annotation information of each sample word can be directly queried from the language database in the target language. Among them, the above-mentioned "language database in the target language" is used to record relevant information (such as pronunciation annotation information, semantic interpretation information, etc.) of each minimum semantic unit in this target language.

[0129] The above-mentioned "pronunciation audio data of the sample word" is used to represent the pronunciation situation of this sample word in the form of audio data.

[0130] In addition, the embodiments of the present application do not limit the acquisition method of the above "pronunciation audio data of sample words". For example, the pronunciation audio data of the sample words can be obtained by means of manual recording for the sample words, so that the pronunciation audio data carries the pronunciation information of the sample words. For another example, specifically, it can be: first, search for the whole sentence English audio carrying the pronunciation information of the sample words from the corpus audio data of the target language; then, cut out the pronunciation audio data of the sample words from the whole sentence English audio, so that the pronunciation audio data carries the pronunciation information of the sample words. For still another example, specifically, it can be: input the sample words into a pre-constructed speech synthesis tool (for example, a machine learning model or a hardware device with speech synthesis function, etc.), and obtain the pronunciation audio data of the sample words output by the speech synthesis tool, so that the pronunciation audio data carries the pronunciation information of the sample words.

[0131] The above "split annotation information of sample words" is used to represent the actual split result of the sample words. For example, when the sample word is "comfortable", the split annotation information of the sample word can show that "comfortable" can actually be split into four pronunciation units with specific pronunciations: "com", "for", "ta", and "ble".

[0132] In addition, the embodiments of the present application do not limit the representation method of the above "split annotation information of sample words". For example, it can be represented by using a pre-set split marking symbol (for example, the symbol "@"). It can be seen that when the sample word is "comfortable", the split annotation information of the sample word can include four strings: "com", "@for", "@ta", and "@ble".

[0133] In addition, the embodiments of the present application do not limit the acquisition method of the above "split annotation information of sample words". For example, the split annotation information of the sample words can be obtained by means of manual split annotation for the sample words.

[0134] Based on the relevant content of the above S101, for a target language with a need for word splitting (for example, English), if it is desired to build a word splitting model in the target language, a large number of sample words in the target language, the pronunciation description information of these sample words, and the split annotation information of these sample words are obtained from the corpus database of the target language, so that the split annotation information of these sample words can be used as prior information to guide the word splitting model in the target language to learn the ability to perform word splitting processing according to word pronunciation from these sample words and their pronunciation description information.

[0135] S102: Construct a vocabulary splitting model using sample vocabulary, pronunciation description information of the sample vocabulary, and splitting annotation information of the sample vocabulary.

[0136] Among them, the vocabulary splitting model is used to perform vocabulary splitting processing on the input data of the vocabulary splitting model; moreover, the embodiments of the present application do not limit the vocabulary splitting model. For example, it can be implemented using any existing or future machine learning model (such as Transformer, etc.).

[0137] In addition, the embodiments of the present application do not limit the construction process of the vocabulary splitting model. For example, it may specifically include S1021 - S1025:

[0138] S1021: Determine the vocabulary description data of the sample vocabulary according to the sample vocabulary and the pronunciation description information of the sample vocabulary.

[0139] The above "vocabulary description data of the sample vocabulary" is used to describe the relevant information of the sample vocabulary (such as writing information, pronunciation information, etc.).

[0140] In addition, the embodiments of the present application do not limit the determination process of the above "vocabulary description data of the sample vocabulary" (that is, the implementation manner of S1021). For example, it may specifically be: directly perform set processing on the sample vocabulary and the pronunciation description information of the sample vocabulary to obtain the vocabulary description data of the sample vocabulary.

[0141] In addition, in order to further improve the description effect of the above "vocabulary description data of the sample vocabulary" on the sample vocabulary, the embodiments of the present application also provide a possible implementation manner for determining the "vocabulary description data of the sample vocabulary" (that is, S1021), which may specifically include S10211 - S10213:

[0142] S10211: Split the sample vocabulary according to the smallest writing unit in the target language to obtain the vocabulary writing sequence of the sample vocabulary.

[0143] Among them, the smallest writing unit refers to the constituent symbol of the smallest semantic unit in the target language. For example, when the target language is English, the smallest writing unit may be an English letter.

[0144] The above "vocabulary writing sequence of the sample vocabulary" is used to represent the writing characteristics of the sample vocabulary. For example, when the target language is English, the smallest writing unit is an English letter, and the sample vocabulary is "comfortable", the vocabulary writing sequence of the sample vocabulary may specifically be the sequence {c, o, m, f, o, r, t, a, b, l, e}.

[0145] It should be noted that the embodiments of the present application do not limit the implementation manner of S10211. For example, it can be implemented by using any existing or future method capable of splitting sample words according to the minimum writing unit.

[0146] S10212: Extract the lexical phoneme features of the sample word from the pronunciation description information of the sample word.

[0147] The above-mentioned "lexical phoneme features of the sample word" are used to represent the pronunciation characteristics carried by the sample word.

[0148] In addition, the embodiments of the present application do not limit the determination process of the above-mentioned "lexical phoneme features of the sample word" (that is, the implementation manner of S10212). For example, when the above-mentioned "pronunciation description information" includes pronunciation annotation information and pronunciation audio data, S10212 may specifically include S102121 - S102123:

[0149] S102121: Perform character extraction processing on the pronunciation annotation information of the sample word to obtain the lexical phonetic symbol sequence of the sample word.

[0150] The above-mentioned "lexical phonetic symbol sequence of the sample word" is used to represent the pronunciation annotation description characters of the sample word; and the embodiments of the present application do not limit the "lexical phonetic symbol sequence of the sample word". For example, when the sample word is "comfortable", the lexical phonetic symbol sequence of the sample word may be This sequence.

[0151] In addition, the embodiments of the present application do not limit the implementation manner of S102121. For example, it can be implemented by using any existing or future method capable of performing character extraction processing on the pronunciation annotation information of the sample word.

[0152] S102122: Perform acoustic feature extraction processing on the pronunciation audio data of the sample word to obtain the lexical audio features of the sample word.

[0153] The above-mentioned "lexical audio features of the sample word" are used to represent the pronunciation acoustic characteristics of the sample word; and the embodiments of the present application do not limit the acquisition process of the "lexical audio features of the sample word". For example, specifically, it can be: input the pronunciation audio data of the sample word into a pre-constructed acoustic feature extraction model, so that the acoustic feature extraction model performs acoustic feature extraction processing on the pronunciation audio data, obtains and outputs the lexical audio features of the sample word, so that the lexical audio features can represent the pronunciation acoustic characteristics of the sample word.

[0154] Among them, the acoustic feature extraction model is used to perform acoustic feature extraction processing on the input data of the acoustic feature extraction model; moreover, the embodiments of the present application do not limit this acoustic feature extraction processing. For example, it can be a machine learning model or a physical device with the function of performing acoustic feature extraction processing on audio data.

[0155] It should be noted that the embodiments of the present application do not limit the construction process of the acoustic feature extraction model.

[0156] S102123: Determine the lexical phoneme features of the sample word according to the lexical phonetic symbol sequence of the sample word and the lexical audio features of the sample word.

[0157] In the embodiments of the present application, after obtaining the lexical phonetic symbol sequence of the sample word and the lexical audio features of the sample word, the lexical phonetic symbol sequence of the sample word and the lexical audio features of the sample word can be subjected to set processing to obtain the lexical phoneme features of the sample word, so that the lexical phoneme features include the lexical phonetic symbol sequence of the sample word and the lexical audio features of the sample word, so that the lexical phoneme features can represent the pronunciation characteristics of the sample word as comprehensively as possible.

[0158] Based on the relevant content of the above S10212, after obtaining the pronunciation description information of the sample word, the lexical phoneme features of the sample word can be extracted from the pronunciation description information, so that the lexical phoneme features can represent the pronunciation characteristics of the sample word as comprehensively as possible.

[0159] S10213: Determine the lexical description data of the sample word according to the lexical writing sequence of the sample word and the lexical phoneme features of the sample word.

[0160] In the embodiments of the present application, after obtaining the lexical writing sequence of the sample word and the lexical phoneme features of the sample word, the two can be subjected to set processing to obtain the lexical description data of the sample word, so that the lexical description data includes the lexical writing sequence of the sample word and the lexical phoneme features of the sample word, so that the lexical description data can better describe the relevant information of the sample word (such as writing information, pronunciation information, etc.).

[0161] Based on the relevant content of the above S1021, after obtaining the sample word and the pronunciation description information of the sample word, the lexical description data of the sample word can be generated by using the sample word and its pronunciation description information, so that the lexical description data can describe the relevant information of the sample word (such as writing information, pronunciation information, etc.), so that the lexical splitting result of the sample word can be determined based on the lexical description data subsequently.

[0162] S1022: Input the lexical description data of the sample word into the model to be trained, and obtain the predicted splitting result of the sample word output by the model to be trained.

[0163] Among them, the model to be trained is used to perform lexical splitting processing on the input data of the model to be trained; moreover, the embodiments of the present application do not limit the model to be trained. For example, it can be implemented using any existing or future machine learning model (such as Transformer, etc.).

[0164] In fact, in order to further improve the lexical splitting ability, the embodiments of the present application also provide a possible implementation manner of the model to be trained, as Figure 2 shown, the model to be trained 200 may specifically include a feature extraction layer 201, an encoding layer 202, a decoding layer 203, and a decision layer 204. Among them, the input data of the encoding layer 202 includes the output data of the feature extraction layer 201; the input data of the decoding layer 203 includes the output data of the encoding layer 202; the input data of the decision layer 204 includes the output data of the decoding layer 203.

[0165] To facilitate understanding of the working principle of the model to be trained 200, the following takes the determination process of the above "predicted splitting result of the sample word" as an example for illustration.

[0166] As an example, the process of using the model to be trained 200 to determine the above "predicted splitting result of the sample word" may specifically include steps 11 - step 14:

[0167] Step 11: Input the lexical description data of the sample word into the feature extraction layer 201, and obtain the feature extraction result output by the feature extraction layer 201.

[0168] Among them, the feature extraction layer 201 is used to perform feature extraction processing on the input data of the feature extraction layer 201.

[0169] In addition, the embodiments of the present application do not limit the implementation manner of the feature extraction layer 201. For example, it may specifically include a writing feature extraction module, a phoneme feature extraction module, and a feature processing module; and the input data of the feature processing module includes the output data of the writing feature extraction module and the output data of the phoneme feature extraction module. For the sake of understanding, the following takes the determination process of the above "feature extraction result" as an example for illustration.

[0170] As an example, when the feature extraction layer 201 includes a writing feature extraction module, a phoneme feature extraction module, and a feature processing module, the determination process of the above "feature extraction result" may specifically include steps 111 - step 112:

[0171] Step 111: Determine the lexical writing feature and the lexical pronunciation feature according to the lexical description data of the sample word, the writing feature extraction module, and the phoneme feature extraction module.

[0172] As an example, when the above-mentioned "lexical description data" includes the lexical phonetic symbol sequence and the lexical audio feature, step 111 may specifically include steps 1111 - 1112:

[0173] Step 1111: Input the lexical phonetic symbol sequence of the sample word into the writing feature extraction module, and obtain the lexical writing feature output by the writing feature extraction module, so that the lexical writing feature can represent the writing characteristics of the sample word.

[0174] Among them, the writing feature extraction module is used to perform a characterization vector determination process on the input data of the writing feature extraction module; moreover, the embodiments of the present application do not limit the implementation manner of the writing feature extraction module. For example, it can be implemented by using any existing or future method capable of performing a characterization vector determination process on the lexical phonetic symbol sequence (for example, using an Embedding module, etc.).

[0175] Step 1112: Input the lexical phoneme feature of the sample word into the phoneme feature extraction module, and obtain the lexical pronunciation feature output by the phoneme feature extraction module, so that the lexical pronunciation feature can represent the pronunciation characteristics of the sample word.

[0176] Among them, the phoneme feature extraction module is used to perform a characterization vector determination process on the input data of the phoneme feature extraction module; moreover, the embodiments of the present application do not limit the phoneme feature extraction module. For example, as Figure 3 shown, the phoneme feature extraction module may include an audio feature extraction module and a phonetic symbol feature extraction module. For the convenience of understanding, a possible implementation manner of step 1112 will be described below as an example.

[0177] As an example, when the above-mentioned "lexical phoneme feature" includes the lexical phonetic symbol sequence and the lexical audio feature, and the above-mentioned "lexical pronunciation feature" includes an audio characterization vector and a phonetic symbol characterization vector, step 1112 may specifically include steps 11121 - 11122:

[0178] Step 11121: Input the lexical audio feature of the sample word into the audio feature extraction module, and obtain the audio characterization vector output by the audio feature extraction module, so that the audio characterization vector can represent the pronunciation acoustic characteristics of the sample word.

[0179] Among them, the audio feature extraction module is used to perform a characterization vector determination process on the input data of the audio feature extraction module; moreover, the embodiments of the present application do not limit the implementation manner of the audio feature extraction module. For example, it can be implemented by any existing or future method capable of performing a characterization vector determination process on the lexical phonetic symbol sequence (for example, using an Embedding module, etc.).

[0180] Step 11122: Input the lexical phonetic symbol sequence of the sample word into the phonetic symbol feature extraction module to obtain the phonetic symbol characterization vector output by the phonetic symbol feature extraction module, so that the phonetic symbol characterization vector can represent the pronunciation annotation characteristics of the sample word.

[0181] Among them, the phonetic symbol feature extraction module is used to perform a characterization vector determination process on the input data of the phonetic symbol feature extraction module; moreover, the embodiments of the present application do not limit the implementation manner of the phonetic symbol feature extraction module. For example, it can be implemented by any existing or future method capable of performing a characterization vector determination process on the lexical phonetic symbol sequence (for example, using an Embedding module, etc.).

[0182] It should be noted that the embodiments of the present application do not limit the execution order between step 11121 and step 11122.

[0183] Based on the relevant content of the above step 1112, for the phoneme feature extraction module, after inputting the lexical phoneme feature of the sample word into the phoneme feature extraction module, different modules in the phoneme feature extraction module can be used to perform a characterization vector determination process on the lexical audio feature of the sample word and the lexical phonetic symbol sequence of the sample word respectively, to obtain the lexical pronunciation feature, so that the lexical pronunciation feature includes an audio characterization vector and a phonetic symbol characterization vector, thereby enabling the lexical pronunciation feature to represent the pronunciation characteristics of the sample word.

[0184] It should be noted that the embodiments of the present application do not limit the execution order between step 1112 and the above step 1111.

[0185] Based on the relevant content of the above step 111, for the feature extraction layer 201, after inputting the lexical description data of the sample word into the feature extraction layer 201, the writing feature extraction module and the phoneme feature extraction module in the feature extraction layer 201 can extract the lexical writing feature and the lexical pronunciation feature corresponding to the sample word from the lexical description data, so as to be able to determine the feature extraction result corresponding to the sample word based on the lexical writing feature and the lexical pronunciation feature subsequently.

[0186] Step 112: Input the lexical writing feature and the lexical pronunciation feature into the feature processing module to obtain the feature extraction result output by the feature processing module, so that the feature extraction result can represent the lexical characteristics of the sample word.

[0187] Among them, the feature processing module is used to perform some data processing on the input data of the feature processing module (for example, combination processing, positional encoding processing, fusion processing, etc.); moreover, the embodiments of the present application do not limit the implementation manner of the feature processing module. For example, as Figure 3 shown, the feature processing module may include a feature combination module, a positional encoding processing module, and a feature fusion module.

[0188] The above-mentioned "feature combination module" is used to perform feature combination processing on the input data of the feature combination module; moreover, the embodiments of the present application do not limit the implementation manner of the feature combination module. For example, it can be implemented in any implementation manner of the splicing method shown in step 1121 below.

[0189] The above-mentioned "positional encoding processing module" is used to perform positional encoding processing on the input data of the positional encoding processing module; moreover, the embodiments of the present application do not limit the implementation manner of the positional encoding processing module. For example, it can be implemented using any existing or future positional encoding network.

[0190] The above-mentioned "feature fusion module" is used to perform feature fusion processing on the input data of the feature fusion module; moreover, the embodiments of the present application do not limit the implementation manner of the feature fusion module. For example, it can be implemented using any existing or future method that can fuse a feature representation vector and its positional encoding vector (for example, Figure 3 shown ⊕).

[0191] The above "feature extraction result" is used to characterize the lexical characteristics of the sample word (for example, temporal characteristics, writing characteristics, pronunciation characteristics, etc.); moreover, the embodiments of the present application do not limit the determination process of the feature extraction result (that is, the working principle of the feature processing module). For example, it may specifically include steps 1121-step 1123:

[0192] Step 1121: Perform splicing processing on the lexical writing feature and the lexical pronunciation feature to obtain the splicing feature to be used.

[0193] Among them, the splicing feature to be used is used to represent the lexical characteristics of the sample word (for example, writing characteristics, pronunciation characteristics, etc.).

[0194] In addition, the embodiments of the present application do not limit the determination process of the splicing feature to be used. For example, when the above "lexical pronunciation feature" includes an audio representation vector and a phonetic symbol representation vector, asFigure 4 As shown, the determination process of the splicing feature to be used can be specifically as follows: According to a preset direction, the lexical writing feature, the audio representation vector, and the phonetic symbol representation vector are spliced to obtain the splicing feature to be used. Among them, the preset direction refers to a preset direction (for example, Figure 4 the horizontal direction shown).

[0195] In addition, in order to better utilize the timing information carried by the sample vocabulary, another possible implementation manner of the determination process of the splicing feature to be used is provided in the embodiments of the present application, which may specifically include steps 21 to 23:

[0196] Step 21: Determine the size description information to be used and the feature splicing direction to be used according to the size information of the lexical writing feature and the size information of the lexical pronunciation feature.

[0197] The above-mentioned "size information of the lexical writing feature" is used to describe the data size of the lexical writing feature. For example, as Figure 5 shown, the size information of the lexical writing feature can be N×D. Wherein, N is a positive integer and D is a positive integer.

[0198] The above-mentioned "size information of the lexical pronunciation feature" is used to describe the data size of the lexical pronunciation feature. For example, when the above-mentioned "lexical pronunciation feature" includes an audio representation vector and a phonetic symbol representation vector, as Figure 5 shown, the size information of the lexical pronunciation feature may include the size information of the audio representation vector and the size information of the phonetic symbol representation vector. Among them, the size information of the audio representation vector is used to describe the data size of the audio representation vector; and the size information of the audio representation vector can be T×D. The size information of the phonetic symbol representation vector is used to describe the data size of the phonetic symbol representation vector; and the size information of the phonetic symbol representation vector can be M×D. Wherein, T is a positive integer, M is a positive integer, and D is a positive integer.

[0199] The above-mentioned "size description information to be used" is used to represent the size information required for data size consistency processing for the lexical writing feature and the lexical pronunciation feature; and the embodiments of the present application do not limit the "size description information to be used". For example, as Figure 5 shown, when T > N and T > M, the "size description information to be used" can be T×D.

[0200] The above-mentioned "feature splicing direction to be used" is used to represent the feature splicing direction; and the "feature splicing direction to be used" is not the timing representation direction. For example, the feature splicing direction to be used can be Figure 5 the vertical direction shown (that is, the direction where D is located).

[0201] Step 22: Determine the to-be-used writing feature and the to-be-used pronunciation feature according to the to-be-used dimension description information, the vocabulary writing feature, and the vocabulary pronunciation feature, so that the dimension information of the to-be-used writing feature is consistent with the dimension information of the to-be-used pronunciation feature.

[0202] Among them, the to-be-used writing feature is used to describe the writing characteristics of the sample vocabulary; moreover, the embodiments of the present application do not limit the determination process of the to-be-used writing feature. For example, specifically, it may be: determine whether the dimension information of the vocabulary writing feature is equal to the to-be-used dimension description information. If so, directly determine the vocabulary writing feature as the to-be-used writing feature; if not, perform padding processing (such as zero-padding processing, etc.) on the vocabulary writing feature according to the to-be-used dimension description information to obtain the to-be-used writing feature, so that the dimension information of the to-be-used writing feature is equal to the to-be-used dimension description information.

[0203] The to-be-used pronunciation feature is used to describe the pronunciation characteristics of the sample vocabulary; moreover, the embodiments of the present application do not limit the determination process of the to-be-used pronunciation feature. For example, when the above "vocabulary pronunciation feature" includes an audio representation vector and a phonetic symbol representation vector, the determination process of the to-be-used pronunciation feature may specifically include the following steps one to three:

[0204] Step one: Determine whether the dimension information of the audio representation vector is equal to the to-be-used dimension description information. If so, directly determine the audio representation vector as the to-be-used audio feature; if not, perform padding processing (such as zero-padding processing, etc.) on the audio representation vector according to the to-be-used dimension description information to obtain the to-be-used audio feature, so that the dimension information of the to-be-used audio feature is equal to the to-be-used dimension description information.

[0205] Step two: Determine whether the dimension information of the phonetic symbol representation vector is equal to the to-be-used dimension description information. If so, directly determine the phonetic symbol representation vector as the to-be-used phonetic symbol feature; if not, perform padding processing (such as zero-padding processing, etc.) on the phonetic symbol representation vector according to the to-be-used dimension description information to obtain the to-be-used phonetic symbol feature, so that the dimension information of the to-be-used phonetic symbol feature is equal to the to-be-used dimension description information.

[0206] Step three: Determine the to-be-used pronunciation feature according to the to-be-used audio feature and the to-be-used phonetic symbol feature, so that the to-be-used pronunciation feature includes the to-be-used audio feature and the to-be-used phonetic symbol feature, so that the dimension information of the to-be-used pronunciation feature conforms to the to-be-used dimension description information, and further so that the dimension information of the to-be-used pronunciation feature is consistent with the dimension information of the to-be-used writing feature.

[0207] Step 23: Perform splicing processing on the to-be-used writing feature and the to-be-used pronunciation feature according to the to-be-used feature splicing direction to obtain the to-be-used splicing feature.

[0208] In the embodiments of the present application, after obtaining the to-be-used writing feature and the to-be-used pronunciation feature, the to-be-used writing feature and the to-be-used pronunciation feature can be directly spliced according to the splicing direction of the to-be-used features (such as the vertical direction shown in Figure 5 ), to obtain the to-be-used splicing feature, so that the to-be-used splicing feature can represent the writing characteristics and pronunciation characteristics carried by the sample word, and can also represent the timing characteristics carried by the sample word, so that the to-be-used splicing feature can more accurately represent the word characteristics of the sample word.

[0209] Based on the relevant content of step 1121 above, for the feature processing module, after inputting the word writing feature and the word pronunciation feature into the feature processing module, the feature processing module can perform a combination process on the word writing feature and the word pronunciation feature to obtain the to-be-used splicing feature, so that the to-be-used splicing feature can represent the word characteristics of the sample word.

[0210] Step 1122: Perform position encoding processing on the to-be-used splicing feature to obtain the position encoding feature corresponding to the to-be-used splicing feature.

[0211] It should be noted that the embodiments of the present application do not limit the implementation manner of "position encoding processing" in step 1122. For example, it can be implemented using any existing or future position encoding network (such as the position encoding module used in the transform model, etc.).

[0212] Step 1123: Perform a fusion process on the to-be-used splicing feature and the position encoding feature corresponding to the to-be-used splicing feature to obtain a feature extraction result.

[0213] In the embodiments of the present application, after obtaining the to-be-used splicing feature and the position encoding feature corresponding to the to-be-used splicing feature, the two can be fused (for example, the ⊕ shown in Figure 3 ) to obtain a feature extraction result, so that the feature extraction result can better represent the word characteristics of the sample word.

[0214] Based on the relevant content of the above steps 1121 to 1123, for the feature processing module, after inputting the lexical writing feature and the lexical pronunciation feature into the feature processing module, the feature processing module can first combine and process the lexical writing feature and the lexical pronunciation feature to obtain the splicing feature to be used; then, the feature processing module performs position encoding processing on the splicing feature to be used to obtain the position encoding feature corresponding to the splicing feature to be used; finally, the feature processing module performs fusion processing on the splicing feature to be used and its corresponding position encoding feature to obtain the feature extraction result, so that the feature extraction result can better represent the lexical characteristics of the sample word.

[0215] Based on the relevant content of the above step 11, for the model 200 to be trained, after inputting the lexical description data of the sample word into the model 200 to be trained, the feature extraction layer 201 (for example, Figure 3 the feature extraction layer shown) in the model 200 to be trained can perform feature extraction processing on the lexical description data to obtain and output the feature extraction result corresponding to the sample word, so that the feature extraction result can represent the lexical characteristics of the sample word.

[0216] Step 12: Input the feature extraction result into the encoding layer 202 to obtain the feature encoding result output by the encoding layer 202.

[0217] Among them, the encoding layer 202 is used to perform encoding processing on the input data of the encoding layer 202; and the embodiments of the present application do not limit the implementation manner of the encoding layer 202. For example, it can be implemented by using any existing or future encoding network (such as the Encoder module in the transform model, etc.).

[0218] Based on the relevant content of step 12, for the model 200 to be trained, after the encoding layer 202 in the model 200 to be trained obtains the feature extraction result corresponding to the sample word, the encoding layer 202 can perform encoding processing on the feature extraction result to obtain and output the feature encoding result corresponding to the sample word.

[0219] Step 13: Input the feature encoding result into the decoding layer 203 to obtain the feature decoding result output by the decoding layer 203.

[0220] Among them, the decoding layer 203 is used to perform decoding processing on the input data of the decoding layer 203; and the embodiments of the present application do not limit the implementation manner of the decoding layer 203. For example, it can be implemented by using any existing or future decoding network (such as the Decoder module in the transform model, etc.).

[0221] Based on the relevant content of step 13, for the model 200 to be trained, after the decoding layer 203 in the model 200 to be trained obtains the feature encoding result corresponding to the sample vocabulary, the decoding layer 203 can perform decoding processing on the feature encoding result to obtain and output the feature decoding result corresponding to the sample vocabulary.

[0222] Step 14: Input the feature decoding result into the decision-making layer 204 to obtain the predicted splitting result of the sample vocabulary output by the decision-making layer 204.

[0223] Among them, the decision-making layer 204 is used to perform classification processing on the input data of the decision-making layer 204; and the embodiments of the present application do not limit the decision-making layer 204. For example, it can be implemented using any existing or future decoding network (for example, Figure 3 the decision-making layer shown, etc.).

[0224] In addition, for Figure 3 the decision-making layer shown, the decision-making layer may include a linear processing module and a classification module. Among them, the linear processing module is used to perform linear data processing on the input data of the linear processing module; and the embodiments of the present application do not limit the linear processing module. For example, it can be implemented using any existing or future linear processing network (for example, the linear module in the transform model, etc.). The classification module is used to perform classification processing on the input data of the classification module; and the embodiments of the present application do not limit the classification module. For example, it can be implemented using any existing or future classification network (for example, softmax, etc.).

[0225] The above "predicted splitting result of the sample vocabulary" is used to indicate which sub-words the sample vocabulary can be predicted to be split into, so that the "predicted splitting result of the sample vocabulary" can indicate the predicted position of the splitting mark symbol in the sample vocabulary.

[0226] In addition, the above "predicted splitting result of the sample vocabulary" can be represented in the form of a one-hot vector. For ease of understanding, an example is given below for illustration.

[0227] As an example, when the target language is English, the predicted splitting result of the sample vocabulary can be represented by a one-hot vector in a character space of 52 dimensions, so that the "predicted splitting result of the sample vocabulary" can be a one-hot vector of 52 dimensions. Among them, the 52-dimensional character space is constructed by 26 English letters and their forms with splitting mark symbols (that is, a, @@a, b, @@b,...); and for the predicted splitting result [p1, p2, p3, p4, p5, p6,... p 51 p52 It can be interpreted using the content shown in Table 1 below.

[0228] Character Predicted Splitting Result a p1 @@a <![CDATA[p2]]> b <![CDATA[p3]]> @@b <![CDATA[p4]]> c p5 @@c <![CDATA[p6]]> …… …… z p51 @@z <![CDATA[p 52 >

[0229] Table 1 Analytical content of the predicted splitting results

[0230] It should be noted that for Table 1, p1 represents the likelihood of the sub-word obtained by splitting the sample word containing a (or, represents the likelihood that the splitting marker symbol does not appear in front of a); p2 represents the likelihood of the sub-word obtained by splitting the sample word containing @@a (that is, the likelihood that the splitting marker symbol appears in front of a); p3 represents the likelihood of the sub-word obtained by splitting the sample word containing b (or, represents the likelihood that the splitting marker symbol does not appear in front of b); p4 represents the likelihood of the sub-word obtained by splitting the sample word containing @@b (that is, the likelihood that the splitting marker symbol appears in front of b); p5 represents the likelihood of the sub-word obtained by splitting the sample word containing c (or, represents the likelihood that the splitting marker symbol does not appear in front of c); p6 represents the likelihood of the sub-word obtained by splitting the sample word containing @@c (that is, the likelihood that the splitting marker symbol appears in front of c);... (and so on); p 51 represents the likelihood of the sub-word obtained by splitting the sample word containing z (or, represents the likelihood that the splitting marker symbol does not appear in front of z); p 52 represents the likelihood of the sub-word obtained by splitting the sample word containing @@z (that is, the likelihood that the splitting marker symbol appears in front of z).

[0231] Based on the relevant content of the above Step 11 to Step 14, for Figure 2 the to-be-trained model 200 shown, after the to-be-trained model 200 obtains the lexical description data of the sample word, the to-be-trained model 200 can perform lexical splitting processing on the lexical description data to obtain and output the predicted splitting result of the sample word, so that the predicted splitting result can indicate where to add the splitting marker symbol in the sample word, thereby enabling the predicted splitting result to represent which sub-words the sample word is predicted to be split into, so as to subsequently measure the lexical splitting performance of the to-be-trained model 200 based on the predicted splitting result.

[0232] Based on the relevant content of S1022 above, after obtaining the lexical description data of the sample vocabulary, the lexical description data can be input into the model to be trained, so that the model to be trained can, based on the lexical description data, perform lexical splitting processing on the sample vocabulary according to the pronunciation of the sample vocabulary, obtain and output the predicted splitting result of the sample vocabulary, so that the predicted splitting result can indicate which sub-words the sample vocabulary can be predicted to be split into, so as to determine the lexical splitting performance of the model to be trained based on the predicted splitting result subsequently.

[0233] S1023: Determine whether a preset stop condition is reached. If so, execute S1025; if not, execute S1024.

[0234] Among them, the preset stop condition can be set in advance; and the embodiments of the present application do not limit the preset stop condition. For example, specifically, it can be: the model loss value of the model to be trained is lower than the first threshold. Another example is that specifically, it can be: the change rate of the model loss value of the model to be trained is lower than the second threshold (that is, the model to be trained reaches convergence). Still another example is that specifically, it can be: the number of updates of the model to be trained reaches the third threshold. Among them, the first threshold, the second threshold, and the third threshold can all be set in advance.

[0235] The above-mentioned "model loss value of the model to be trained" is used to represent the lexical splitting performance of the model to be trained; and the embodiments of the present application do not limit the determination process of the "model loss value of the model to be trained". For example, any existing or future model loss calculation method can be used for implementation.

[0236] Based on the relevant content of S1023, for the model to be trained in the current round, it can be determined whether the model to be trained reaches the preset stop condition. If the preset stop condition is reached, it can be determined that the model to be trained has a high lexical splitting performance, so the model to be trained can be directly referred to and a lexical splitting model can be constructed; however, if the preset stop condition is not reached, it can be determined that the lexical splitting performance of the model to be trained is still relatively low. Therefore, the difference between the predicted splitting result of the sample vocabulary and the splitting annotation information of the sample vocabulary can be used to update the model to be trained, so that the updated model to be trained has better lexical splitting performance, so that it can be realized that under the guidance of the splitting annotation information of the sample vocabulary, the model to be trained can learn how to perform lexical splitting processing according to the lexical pronunciation from a large number of sample vocabularies and their pronunciation description information.

[0237] S1024: Update the model to be trained according to the predicted splitting result of the sample vocabulary and the splitting annotation information of the sample vocabulary, and return to execute S1022.

[0238] It should be noted that the embodiments of the present application do not limit the implementation manner of S1024. For example, any existing or future model update method can be used for implementation.

[0239] In addition, in order to further improve the update effect, the embodiments of the present application also provide another possible implementation manner for updating the model to be trained, which may specifically include steps 31 - 32:

[0240] Step 31: Determine the actual splitting result of the sample word according to the splitting annotation information of the sample word, so that the actual splitting result can represent the actual position of the splitting marker symbol in the sample word in the form of a one - hot vector.

[0241] As an example, step 31 may specifically include steps 311 - 312:

[0242] Step 311: Perform character extraction processing on the splitting annotation information of the sample word to obtain the splitting annotation sequence of the sample word.

[0243] Among them, the splitting annotation sequence of the sample word is used to represent the actual position of the splitting marker symbol in the sample word; and the embodiments of the present application do not limit the "splitting annotation sequence of the sample word". For example, when the splitting annotation information of the sample word includes com, @@for, @@ta, and @@ble, the splitting annotation sequence of the sample word can be the sequence {c o m@@f o r@@t a@@ble}.

[0244] Step 312: Map the splitting annotation sequence of the sample word to a 52 - dimensional character space to obtain the actual splitting result of the sample word.

[0245] In the embodiments of the present application, after obtaining the splitting annotation sequence of the sample word, the splitting annotation sequence can be mapped to the "52 - dimensional character space" above to obtain the actual splitting result of the sample word, so that the actual splitting result is represented by a 52 - dimensional one - hot vector, so that the actual splitting result can not only represent which minimum writing units (such as English letters) the sample word involves, but also represent the actual position of the splitting marker symbol in the sample word.

[0246] Based on the relevant content of step 31 above, after obtaining the splitting annotation information of the sample word, the splitting annotation information can be mapped to a 52 - dimensional character space to obtain the actual splitting result of the sample word, so that the actual splitting result can represent the actual position of the splitting marker symbol in the sample word in the form of a one - hot vector, so as to better determine the difference between the splitting annotation information of the sample word and the predicted splitting result of the sample word with the help of the actual splitting result.

[0247] It should be noted that the execution time of step 31 in the embodiments of the present application is not limited, as long as it is ensured that the execution time of this step 31 is earlier than the execution time of step 32.

[0248] Step 32: Update the model to be trained according to the actual splitting result and the predicted splitting result of the sample vocabulary, and return to execute S1022.

[0249] In the embodiments of the present application, after obtaining the actual splitting result and the predicted splitting result of the sample vocabulary, the difference representation value between the two (for example, the difference between the two, or the distance between the two, etc.) can be referred to to update the model to be trained, so that the updated model to be trained has better vocabulary splitting performance, which is beneficial to improving the model update effect.

[0250] Based on the relevant content of the above S1024, when it is determined that the model to be trained in the current round does not meet the preset stop condition, it can be determined that the vocabulary splitting performance of the model to be trained is still relatively low. Therefore, the difference between the predicted splitting result of the sample vocabulary and the splitting annotation information of the sample vocabulary can be referred to to update the model to be trained, so that the updated model to be trained has better vocabulary splitting performance, and the updated model to be trained is used to continue to execute S1022 and its subsequent steps above to implement the next round of training process for the model to be trained.

[0251] S1025: Determine a vocabulary splitting model according to the model to be trained.

[0252] In the embodiments of the present application, when it is determined that the model to be trained in the current round meets the preset stop condition, it can be determined that the model to be trained has relatively high vocabulary splitting performance, and thus it can be determined that the model to be trained has the ability to perform vocabulary splitting processing based on vocabulary pronunciation. Therefore, the model to be trained can be directly used to construct a vocabulary splitting model (for example, directly determine the model to be trained as the vocabulary splitting model; or, according to the model structure and model parameters of the model to be trained, set the model structure and model parameters of the vocabulary splitting model so that the model structure and model parameters of the vocabulary splitting model are consistent with the model structure and model parameters of the vocabulary splitting model), so that the vocabulary splitting model also has the ability to perform vocabulary splitting processing based on vocabulary pronunciation, so as to be able to use the vocabulary splitting model to perform vocabulary splitting processing on any minimum semantic unit in the target language in the future.

[0253] Based on the relevant content of S101 to S102 above, for the method for constructing a vocabulary splitting model provided by the embodiments of the present application, a large number of sample vocabulary in the target language, pronunciation description information of these sample vocabulary, and splitting annotation information of these sample vocabulary are first obtained; then, using these sample vocabulary, pronunciation description information of these sample vocabulary, and splitting annotation information of these sample vocabulary, a vocabulary splitting model is constructed, so that the constructed vocabulary splitting model can learn the ability to perform vocabulary splitting processing based on the pronunciation of the vocabulary from these sample vocabulary and their related information (that is, pronunciation description information and splitting annotation information), so as to be able to use the constructed vocabulary splitting model to automatically split a large number of minimum semantic units in the target language in the future.

[0254] Among them, since the vocabulary splitting model is constructed based on the pronunciation description information of a large number of sample vocabulary, the constructed vocabulary splitting model has the ability to perform vocabulary splitting processing based on the pronunciation of the vocabulary, so that the constructed vocabulary splitting model can perform a relatively reasonable splitting processing on each minimum semantic unit in the target language. In this way, it is beneficial to improve the vocabulary splitting effect, and thus when using the vocabulary splitting model to implement speech recognition processing in the target language, it is beneficial to improve the speech recognition effect in the target language.

[0255] Method Embodiment 2

[0256] See Figure 6 , this figure is a flowchart of a vocabulary splitting method provided by the embodiments of the present application.

[0257] The vocabulary splitting method provided by the embodiments of the present application includes S601 - S602:

[0258] S601: Obtain the vocabulary to be split in the target language and the pronunciation description information of the vocabulary to be split.

[0259] Among them, the vocabulary to be split is used to represent any minimum semantic unit in the target language. For example, when the target language is English, the vocabulary to be split can be any English word.

[0260] The pronunciation description information of the vocabulary to be split is used to describe the pronunciation situation of the vocabulary to be split; and the embodiments of the present application do not limit the "pronunciation description information of the vocabulary to be split". For example, it may include at least one of the pronunciation annotation information of the vocabulary to be split and the pronunciation audio data of the vocabulary to be split.

[0261] It should be noted that the relevant content of the above "pronunciation description information of the vocabulary to be split" is similar to the relevant content of the "pronunciation description information of the sample vocabulary" above. For the sake of brevity, it will not be elaborated here.

[0262] S602: Determine the splitting result of the word to be split according to the word to be split, the pronunciation description information of the word to be split, and a pre-constructed word splitting model.

[0263] Among them, the word splitting model is constructed by using any implementation manner of the word splitting model construction method provided above Method Embodiment 1 and described above.

[0264] The above "splitting result of the word to be split" is used to indicate which sub-words the word to be split can be split into, so that the "splitting result of the word to be split" can indicate the position of the splitting mark symbol in the word to be split.

[0265] In addition, the determination process of the above "splitting result of the word to be split" is similar to the determination process of the "predicted splitting result of the sample word" above. For the sake of easy understanding, an example is given below for illustration.

[0266] As an example, S602 may specifically include S6021 - S6022:

[0267] S6021: Determine the word description data of the word to be split according to the word to be split and the pronunciation description information of the word to be split.

[0268] The above "word description data of the word to be split" is used to describe the relevant information of the word to be split (for example, writing information, pronunciation information, etc.).

[0269] In addition, the relevant content of the above "word description data of the word to be split" is similar to the relevant content of the "word description data of the sample word" above. For the sake of brevity, it will not be elaborated here.

[0270] S6022: Input the word description data of the word to be split into the pre-constructed word splitting model, and obtain the splitting result of the word to be split output by the word splitting model.

[0271] It should be noted that the relevant content of S6022 is similar to the relevant content of S1022 above. For the sake of brevity, it will not be elaborated here.

[0272] Based on the relevant content of S601 to S602 above, for the word splitting method provided in the embodiments of the present application, after obtaining the word to be split in the target language, the word to be split can be split by means of a pre-constructed word splitting model and the pronunciation description information of the word to be split, and the splitting result of the word to be split can be obtained, so that the splitting result can indicate the position of the splitting mark symbol in the word to be split, thereby enabling the splitting result to indicate which sub-words the word to be split can be split into.

[0273] Among them, since the pre-constructed vocabulary splitting model has a good ability to split vocabulary according to vocabulary pronunciation, the splitting result determined by using the vocabulary splitting model for the vocabulary to be split can more reasonably represent which sub-words the vocabulary to be split is composed of. In this way, it is beneficial to improve the vocabulary splitting effect, and thus when using these sub-words to implement speech recognition processing in the target language, it is beneficial to improve the speech recognition effect in the target language.

[0274] Method Embodiment 3

[0275] See Figure 7 , which is a flowchart of a sub-word dictionary construction method provided by an embodiment of the present application.

[0276] The sub-word dictionary construction method provided by the embodiment of the present application includes S701-S704:

[0277] S701: Obtain at least one candidate vocabulary in the target language and pronunciation description information of each candidate vocabulary.

[0278] The above-mentioned "at least one candidate vocabulary" is used to represent the smallest semantic unit required when constructing a sub-word dictionary in the target language.

[0279] In addition, the embodiment of the present application does not limit the acquisition process of the above-mentioned "at least one candidate vocabulary". For example, specifically, it can be: statistically collecting all the smallest semantic units (for example, English words) in a large amount of corpus text in the target language to obtain at least one candidate vocabulary, so that these candidate vocabularies can cover all the smallest semantic units in the target language as much as possible.

[0280] In addition, the embodiment of the present application does not limit the relationship between the above-mentioned "at least one candidate vocabulary" and the "sample vocabulary" above. For example, specifically, a large number of sample vocabularies can be selected from the above-mentioned "at least one candidate vocabulary" so that these sample vocabularies can cover all pronunciation situations in the target language as much as possible.

[0281] The pronunciation description information of the j-th candidate vocabulary is used to describe the pronunciation situation of the j-th candidate vocabulary; and the embodiment of the present application does not limit the "pronunciation description information of the j-th candidate vocabulary". For example, it may include at least one of the pronunciation annotation information of the j-th candidate vocabulary and the pronunciation audio data of the j-th candidate vocabulary. Wherein, j is a positive integer, j≤J, J is a positive integer, and J represents the number of candidate vocabularies in the above-mentioned "at least one candidate vocabulary".

[0282] It should be noted that the relevant content of the above-mentioned "pronunciation description information of the j-th candidate vocabulary" is similar to the relevant content of the "pronunciation description information of the sample vocabulary" above. For the sake of brevity, it will not be repeated here.

[0283] S702: Obtain the splitting results of each candidate word according to each candidate word, the pronunciation description information of each candidate word, and a pre-constructed word splitting model.

[0284] Among them, the word splitting model is constructed by using any implementation manner of the word splitting model construction method provided above. Method Embodiment 1 The splitting result of the j-th candidate word is used to indicate which sub-words the j-th candidate word can be split into, so that the "splitting result of the j-th candidate word" can indicate the position of the splitting mark symbol in the j-th candidate word. Where j is a positive integer, j ≤ J, J is a positive integer, and J represents the number of candidate words in the above "at least one candidate word".

[0285] In addition, the determination process of the above "splitting result of the j-th candidate word" is similar to the determination process of the "splitting result of the word to be split" above. For the sake of brevity, it will not be elaborated here.

[0286]

[0287] S703: Determine at least one sub-word corresponding to each candidate word according to the splitting results of each candidate word.

[0288] Among them, at least one sub-word corresponding to the j-th candidate word is used to indicate the pronunciation unit involved in the j-th candidate word. For example, when the target language is English, the j-th candidate word is speak, and the splitting result of the j-th candidate word indicates that the splitting mark symbol is before p and before k, at least one sub-word corresponding to the j-th candidate word may include three sub-words: s, pea, and k, so that these sub-words can indicate the pronunciation unit involved in the j-th candidate word. j is a positive integer, j ≤ J, J is a positive integer, and J represents the number of candidate words in the above "at least one candidate word".

[0289] In addition, the embodiments of the present application do not limit the determination process of the above "at least one sub-word corresponding to the j-th candidate word". For example, it may specifically include S7031 - S7032:

[0290] S7031: Determine at least one splitting position corresponding to the j-th candidate word according to the splitting result of the j-th candidate word.

[0291] The above "at least one splitting position corresponding to the j-th candidate word" is used to indicate the splitting position for the j-th candidate word.

[0292] ​In addition, the above-mentioned "at least one splitting position corresponding to the j-th candidate word" is determined according to the positions of the splitting marker symbols represented by the splitting result of the j-th candidate word. For example, when the j-th candidate word is "speak", and the predicted probabilities for @@p and @@k are relatively high in the splitting result of the j-th candidate word, it can be determined that the at least one splitting position corresponding to the j-th candidate word includes the gap between s and p, and the gap between a and k.

[0293] S7032: Split the j-th candidate word according to the at least one splitting position corresponding to the j-th candidate word, to obtain at least one sub-word corresponding to the j-th candidate word.

[0294] As an example, when the j-th candidate word is "speak", and the at least one splitting position corresponding to the j-th candidate word includes the gap between s and p, and the gap between a and k, first for the j-th candidate word, a cut can be made at the gap between s and p and at the gap between a and k, to obtain three segments: s, pea, and k; then these three segments are all determined as the sub-words corresponding to the j-th candidate word, so that the "at least one sub-word corresponding to the j-th candidate word" includes three sub-words: s, pea, and k.

[0295] Based on the relevant content of the above S703, after obtaining the splitting result of the j-th candidate word, the position information of the splitting marker symbols represented by the splitting result can be referred to to split the j-th candidate word, to obtain at least one sub-word corresponding to the j-th candidate word, so that these sub-words can represent the pronunciation units involved in the j-th candidate word, for subsequent reference to these sub-words to construct a sub-word dictionary in the target language. Among them, j is a positive integer, j ≤ J, J is a positive integer, and J represents the number of candidate words in the above-mentioned "at least one candidate word".

[0296] S704: Perform statistical analysis processing on the at least one sub-word corresponding to the at least one candidate word, to obtain a sub-word dictionary in the target language.

[0297] In the embodiments of the present application, after obtaining at least one sub-word corresponding to all candidate words, statistical analysis processing (such as deduplication processing, frequency statistics processing, etc.) can be performed on at least one sub-word corresponding to these candidate words to obtain a sub-word dictionary in the target language, so that the sub-word dictionary can cover as many sub-words in the target language as possible, thereby enabling the sub-word dictionary to cover as many pronunciation units in the target language as possible, and further enabling the sub-word dictionary to meet the expression requirements of the smallest semantic units in the target language as much as possible, so as to use the sub-word dictionary to replace the smallest semantic unit database (such as an English word dictionary) in the target language later to construct a one-hot vector in the speech recognition scenario involving the target language.

[0298] Based on the relevant content of S701 to S704 above, for the sub-word dictionary construction method provided in the embodiments of the present application, after obtaining a large number of candidate words in the target language, first, with the help of a pre-constructed word splitting model and the pronunciation description information of each candidate word, perform word splitting processing on each candidate word to obtain the splitting result of each candidate word, so that the splitting result of each candidate word can represent the position information of the splitting marker symbols; then, according to the splitting results of each candidate word, determine at least one sub-word corresponding to each candidate word; finally, summarize at least one sub-word corresponding to these candidate words to obtain a sub-word dictionary in the target language, so that the sub-word dictionary can cover as many sub-words that may appear in the target language as possible, thereby enabling the sub-word dictionary to cover as many pronunciation units in the target language as possible, and further enabling the sub-word dictionary to meet the expression requirements of the smallest semantic units in the target language as much as possible. This is beneficial to improving the expression effect of the sub-word dictionary in the target language, and thus is beneficial to improving the speech recognition effect in the target language when using the sub-word dictionary to implement speech recognition processing in the target language.

[0299] Method Embodiment 4

[0300] See Figure 8 , which is a flowchart of a speech recognition model construction method provided in the embodiments of the present application.

[0301] The speech recognition model construction method provided in the embodiments of the present application includes S801 - S803:

[0302] S801: Obtain a sample speech in the target language and the actual speech content of the sample speech.

[0303] Among them, the sample speech refers to the speech data required when constructing a speech recognition model in the target language; and the embodiments of the present application do not limit the number of the sample speeches.

[0304] In addition, the embodiments of the present application do not limit the acquisition method of the sample speech. For example, any existing or future method capable of acquiring speech data in the target language can be used for implementation.

[0305] The "actual speech content of the sample speech" above is used to represent the speech information actually carried by the sample speech; moreover, the embodiments of the present application do not limit the acquisition method of the "actual speech content of the sample speech". For example, any existing or future method capable of acquiring the speech information actually carried by the sample speech (such as the manual annotation method, etc.) can be used for implementation.

[0306] S802: Determine the speech content annotation of the sample speech according to the sub-word dictionary in the target language and the actual speech content of the sample speech.

[0307] Among them, the sub-word dictionary in the target language is constructed by any implementation manner of the sub-word dictionary construction method provided above Method Embodiment 3 above.

[0308] The "speech content annotation of the sample speech" above is used to represent the mapping result of the actual speech content of the sample speech in the data representation space (hereinafter simply referred to as the "sub-word dictionary space") constructed by the above "sub-word dictionary in the target language", so that the "speech content annotation of the sample speech" is represented by a one-hot vector in the sub-word dictionary space, so that the data dimension of the "speech content annotation of the sample speech" is equal to the space dimension of the sub-word dictionary space (that is, the number of sub-words in the "sub-word dictionary in the target language").

[0309] In addition, the embodiments of the present application do not limit the determination process of the above "speech content annotation of the sample speech". For example, the manual annotation method can be used for implementation.

[0310] In addition, in order to further improve the construction efficiency of the speech recognition model, the embodiments of the present application also provide another possible implementation manner of the determination process of the above "speech content annotation of the sample speech", which may specifically include S8021-S8023:

[0311] S8021: Construct the data space to be used in the target language according to the sub-word dictionary in the target language.

[0312] In the embodiments of the present application, after obtaining the sub-word dictionary in the target language, each sub-word in the sub-word dictionary can be used as a data dimension to obtain the data space to be used in the target language, so that the data dimension of the data space to be used is equal to the number of sub-words in the sub-word dictionary, so that the data space to be used can represent the sub-word dictionary.

[0313] S8022: Determine each minimum semantic unit in the actual speech content of the sample speech as a vocabulary to be processed respectively.

[0314] As an example, when the target language is English, S8022 can specifically be: Determine each English word in the actual speech content of the sample speech as a vocabulary to be processed, so that these vocabularies to be processed can represent the speech information carried by the actual speech content.

[0315] S8023: Determine the splitting result of each vocabulary to be processed according to each vocabulary to be processed, the pronunciation description information of each vocabulary to be processed, and a pre-constructed vocabulary splitting model.

[0316] Among them, the vocabulary splitting model is constructed by using any implementation manner of the vocabulary splitting model construction method provided above. Method Embodiment 1

[0317] The pronunciation description information of the k-th vocabulary to be processed is used to describe the pronunciation situation of the k-th vocabulary to be processed; moreover, the embodiments of the present application do not limit the "pronunciation description information of the k-th vocabulary to be processed". For example, it may include at least one of the pronunciation annotation information of the k-th vocabulary to be processed and the pronunciation audio data of the k-th vocabulary to be processed. Wherein, k is a positive integer, k ≤ K, K is a positive integer, and K represents the number of vocabularies to be processed.

[0318] It should be noted that the relevant content of the above "pronunciation description information of the k-th vocabulary to be processed" is similar to the relevant content of the "pronunciation description information of the sample vocabulary" above. For the sake of brevity, it will not be elaborated here.

[0319] The splitting result of the k-th vocabulary to be processed is used to represent which sub-words the k-th vocabulary to be processed can be split into, so that the "splitting result of the k-th vocabulary to be processed" can represent the position where the splitting mark symbol is located in the k-th vocabulary to be processed. Wherein, k is a positive integer, k ≤ K, K is a positive integer, and K represents the number of vocabularies to be processed.

[0320] In addition, the determination process of the above "splitting result of the k-th vocabulary to be processed" is similar to the determination process of the "splitting result of the vocabulary to be split" above. For the sake of brevity, it will not be elaborated here.

[0321] S8024: Determine the vocabulary splitting result of the sample speech according to the splitting results of each vocabulary to be processed.

[0322] ​In an embodiment of the present application, after obtaining the splitting results of all the words to be processed in the sample speech, the splitting results of these words to be processed can be processed as a set according to the positions of these words to be processed in the sample speech, so as to obtain the word splitting result of the sample speech, so that the "word splitting result of the sample speech" can represent which sub-words the actual speech content of the sample speech can be split into.

[0323] S8025: Map the word splitting result of the sample speech to the data space to be used to obtain the speech content annotation of the sample speech.

[0324] In an embodiment of the present application, after obtaining the word splitting result of the sample speech, the word splitting result can be mapped to the data space to be used (that is, the data representation space constructed by the sub-word dictionary in the target language), so as to obtain the speech content annotation of the sample speech, so that the speech content annotation can represent the actual speech content of the sample speech in the form of a one-hot vector in the data space to be used, so that the data dimension of the speech content annotation is equal to the space dimension of the data space to be used (that is, the number of sub-words in the above-mentioned "sub-word dictionary in the target language").

[0325] Based on the relevant content of S802 above, after obtaining the actual speech content of the sample speech in the target language, the actual speech content can be mapped into the data representation space constructed by the sub-word dictionary in the target language to obtain the speech content annotation of the sample speech, so that the speech content annotation can represent the semantic information actually carried by the sample speech with the help of a one-hot vector in the data representation space, so that the subsequent speech recognition model can be guided to learn how to express the speech recognition result of a speech data in the data representation space.

[0326] S803: Use the sample speech and the speech content annotation of the sample speech to construct a speech recognition model.

[0327] Among them, the speech recognition model is used to perform speech recognition processing in the target language on the input data of the speech recognition model; and the embodiment of the present application does not limit the speech recognition model. For example, any existing or future network model with speech recognition ability can be used for implementation.

[0328] In addition, the speech recognition model can use the data representation space constructed by the sub-word dictionary in the target language to express the speech recognition result for a piece of speech data. It can be seen that when the speech recognition model classifies the acoustic feature extraction result of a piece of speech data into a one-hot encoded vector by means of a classification network, the number of classification nodes in the classification network is equal to the number of sub-words in the above-mentioned "sub-word dictionary in the target language", so that the classification network can express the speech recognition result for the speech data in the data representation space.

[0329] In addition, the embodiments of the present application do not limit the construction process of the speech recognition model. For example, any existing or future model construction method can be used for implementation.

[0330] Based on the relevant content of S801 to S803 above, for the speech recognition model construction method provided by the embodiments of the present application, after obtaining a large number of sample speeches in the target language and the actual speech content of these sample speeches, first use the pre-constructed sub-word dictionary in the target language to automatically label the actual speech content of each sample speech to obtain the speech content labels of these sample speeches, so that these speech content labels can represent the speech information actually carried by these sample speeches in the form of one-hot encoded vectors; then use these sample speeches and the speech content labels of these sample speeches to construct a speech recognition model, so that the speech recognition model can not only learn the ability to perform speech recognition on speech data, but also learn the ability to express speech recognition results using the sub-word dictionary, so that the classification network in the speech recognition model can use all sub-words in the sub-word dictionary as classification categories for classification processing, and further make the dimension of the one-hot encoded vector determined for a piece of speech data by using the speech recognition model equal to the number of sub-words in the sub-word dictionary. This is beneficial to reducing the dimension of the one-hot encoded vector in the target language, thus beneficial to improving the speech recognition ability of the speech recognition model, and further improving the speech recognition effect in the target language.

[0331] Method Embodiment 5

[0332] See Figure 9 , which is a flowchart of a speech recognition method provided by an embodiment of the present application.

[0333] The speech recognition method provided by the embodiments of the present application includes S901 - S902:

[0334] S901: Obtain the speech to be recognized in the target language.

[0335] Among them, the speech to be recognized refers to the speech data that needs to be processed for speech recognition in the target language.

[0336] In addition, the voice to be recognized in the embodiments of the present application is not limited. For example, it can be the voice stream data collected in real time or the voice data stored in a preset storage space.

[0337] S902: Use the pre-constructed speech recognition model to perform speech recognition processing on the voice to be recognized, and obtain the speech recognition result of the voice to be recognized.

[0338] Among them, the speech recognition model is constructed by using any implementation manner of the speech recognition model construction method shown above. Method Embodiment 4

[0339] The above "speech recognition result of the voice to be recognized" is used to represent the voice information carried by the voice to be recognized; and the embodiments of the present application do not limit the determination process of the "speech recognition result of the voice to be recognized". For example, specifically, it can be: input the voice to be recognized into the pre-constructed speech recognition model, and obtain the speech recognition result of the voice to be recognized output by the speech recognition model.

[0340] Based on the relevant content of S901 to S902 above, for the speech recognition method provided by the embodiments of the present application, after obtaining the voice to be recognized in the target language, use the pre-constructed speech recognition model to perform speech recognition processing on the voice to be recognized, and obtain the speech recognition result of the voice to be recognized. Among them, because the speech recognition model has the ability to express the speech recognition result using the sub-word dictionary, the speech recognition result determined for the voice to be recognized by using this speech recognition model can express the voice information carried by the voice to be recognized with the help of the sub-word dictionary, so that the dimension of the speech recognition result is equal to the number of sub-words in the sub-word dictionary, and further the dimension of the speech recognition result is much smaller than the total number of the smallest semantic units (for example, English words) in the target language. In this way, it can overcome the defect that the speech recognition ability is relatively poor due to the too large total number of the smallest semantic units in the target language, thereby improving the speech recognition ability of the speech recognition model, and further improving the speech recognition effect in the target language.

[0341] Based on the vocabulary splitting model construction method provided by the above method embodiments, the embodiments of the present application also provide a vocabulary splitting model construction device, which will be explained and described below with reference to the drawings.

[0342] Device Embodiment 1

[0343] Device Embodiment 1 introduces the vocabulary splitting model construction device. For relevant content, please refer to the above method embodiments.

[0344] See Figure 10 , this figure is a schematic structural diagram of a vocabulary splitting model construction device provided by the embodiments of the present application. ​

[0345] An apparatus 1000 for constructing a vocabulary splitting model provided by an embodiment of the present application includes:

[0346] A first acquisition unit 1001, configured to acquire sample vocabulary in a target language, pronunciation description information of the sample vocabulary, and splitting annotation information of the sample vocabulary;

[0347] A first construction unit 1002, configured to construct a vocabulary splitting model by using the sample vocabulary, the pronunciation description information of the sample vocabulary, and the splitting annotation information of the sample vocabulary.

[0348] In a possible implementation manner, the pronunciation description information includes at least one of pronunciation annotation information and pronunciation audio data.

[0349] In a possible implementation manner, the first construction unit 1002 includes:

[0350] A first determination subunit, configured to determine vocabulary description data of the sample vocabulary according to the sample vocabulary and the pronunciation description information of the sample vocabulary;

[0351] A first prediction subunit, configured to input the vocabulary description data of the sample vocabulary into a model to be trained, and obtain a predicted splitting result of the sample vocabulary output by the model to be trained;

[0352] A first update subunit, configured to update the model to be trained according to the predicted splitting result of the sample vocabulary and the splitting annotation information of the sample vocabulary, and return to the first prediction subunit to continue to execute the step of inputting the vocabulary description data of the sample vocabulary into the model to be trained to obtain the predicted splitting result of the sample vocabulary output by the model to be trained;

[0353] A second determination subunit, configured to determine the vocabulary splitting model according to the model to be trained after a preset stop condition is reached.

[0354] In a possible implementation manner, the first determination subunit includes:

[0355] A first splitting subunit, configured to split the sample vocabulary according to the smallest writing unit in the target language to obtain a vocabulary writing sequence of the sample vocabulary;

[0356] A first extraction subunit, configured to extract vocabulary phoneme features of the sample vocabulary from the pronunciation description information of the sample vocabulary;

[0357] A first processing subunit, configured to perform set processing on the vocabulary writing sequence of the sample vocabulary and the vocabulary phoneme features of the sample vocabulary to obtain the vocabulary description data of the sample vocabulary.

[0358] In a possible implementation manner, the pronunciation description information includes pronunciation annotation information and pronunciation audio data;

[0359] The first extraction subunit is specifically configured to: perform character extraction processing on the pronunciation annotation information of the sample vocabulary to obtain the vocabulary phonetic symbol sequence of the sample vocabulary; perform acoustic feature extraction processing on the pronunciation audio data of the sample vocabulary to obtain the vocabulary audio feature of the sample vocabulary; perform set processing on the vocabulary phonetic symbol sequence and the vocabulary audio feature of the sample vocabulary to obtain the vocabulary phoneme feature of the sample vocabulary.

[0360] In a possible implementation manner, the model to be trained includes a feature extraction layer, an encoding layer, a decoding layer, and a decision-making layer;

[0361] The process of determining the predicted splitting result of the sample vocabulary includes: inputting the vocabulary description data of the sample vocabulary into the feature extraction layer to obtain the feature extraction result output by the feature extraction layer; inputting the feature extraction result into the encoding layer to obtain the feature encoding result output by the encoding layer; inputting the feature encoding result into the decoding layer to obtain the feature decoding result output by the decoding layer; inputting the feature decoding result into the decision-making layer to obtain the predicted splitting result of the sample vocabulary output by the decision-making layer.

[0362] In a possible implementation manner, the feature extraction layer includes a writing feature extraction module, a phoneme feature extraction module, and a feature processing module;

[0363] The process of determining the feature extraction result includes: determining the vocabulary writing feature and the vocabulary pronunciation feature according to the vocabulary description data of the sample vocabulary, the writing feature extraction module, and the phoneme feature extraction module; inputting the vocabulary writing feature and the vocabulary pronunciation feature into the feature processing module to obtain the feature extraction result output by the feature processing module.

[0364] In a possible implementation manner, the vocabulary description data includes a vocabulary writing sequence and a vocabulary phoneme feature; wherein, the vocabulary writing sequence is used to describe the writing characteristics of the sample vocabulary; the vocabulary phoneme feature is used to describe the pronunciation characteristics of the sample vocabulary;

[0365] The process of determining the vocabulary writing feature includes: inputting the vocabulary writing sequence into the writing feature extraction module to obtain the vocabulary writing feature output by the writing feature extraction module;

[0366] The process of determining the vocabulary pronunciation feature includes: inputting the vocabulary phoneme feature into the phoneme feature extraction module to obtain the vocabulary pronunciation feature output by the phoneme feature extraction module.

[0367] In a possible implementation manner, the process of determining the feature extraction result includes: concatenating the lexical writing feature and the lexical pronunciation feature to obtain a to-be-used concatenated feature; performing position encoding processing on the to-be-used concatenated feature to obtain a position encoding feature corresponding to the to-be-used concatenated feature; and fusing the to-be-used concatenated feature and the position encoding feature corresponding to the to-be-used concatenated feature to obtain the feature extraction result.

[0368] In a possible implementation manner, the process of determining the to-be-used concatenated feature includes: determining to-be-used size description information and a to-be-used feature concatenation direction according to the size information of the lexical writing feature and the size information of the lexical pronunciation feature;

[0369] determining a to-be-used writing feature and a to-be-used pronunciation feature according to the to-be-used size description information, the lexical writing feature, and the lexical pronunciation feature, so that the size information of the to-be-used writing feature is consistent with the size information of the to-be-used pronunciation feature; and concatenating the to-be-used writing feature and the to-be-used pronunciation feature according to the to-be-used feature concatenation direction to obtain a to-be-used concatenated feature.

[0370] Based on the lexical splitting method provided in the above method embodiment, the embodiment of the present application further provides a lexical splitting device, which will be explained and described below with reference to the accompanying drawings.

[0371] Device Embodiment 2

[0372] Embodiment 2 of the device introduces the lexical splitting device, and for related content, please refer to the above method embodiment.

[0373] See Figure 11 , which is a schematic structural diagram of a lexical splitting device provided in the embodiment of the present application.

[0374] The lexical splitting device 1100 provided in the embodiment of the present application includes:

[0375] A second acquisition unit 1101, configured to acquire a to-be-split word in a target language and pronunciation description information of the to-be-split word;

[0376] A first splitting unit 1102, configured to determine a splitting result of the to-be-split word according to the to-be-split word, the pronunciation description information of the to-be-split word, and a pre-constructed lexical splitting model; wherein, the lexical splitting model is constructed by using any implementation manner of the lexical splitting model construction method provided in the embodiment of the present application.

[0377] Based on the sub-word dictionary construction method provided in the above method embodiments, the embodiments of the present application further provide a sub-word dictionary construction device, which will be explained and described below with reference to the accompanying drawings.

[0378] Device Embodiment 3

[0379] Embodiment 3 of the device introduces the sub-word dictionary construction device, and for related content, please refer to the above method embodiments.

[0380] See Figure 12 , this figure is a schematic structural diagram of a sub-word dictionary construction device provided by an embodiment of the present application.

[0381] The sub-word dictionary construction device 1200 provided by the embodiments of the present application includes:

[0382] A third acquisition unit 1201, configured to acquire at least one candidate word in the target language and pronunciation description information of each of the candidate words;

[0383] A second splitting unit 1202, configured to determine splitting results of each of the candidate words according to each of the candidate words, the pronunciation description information of each of the candidate words, and a pre-constructed word splitting model; wherein, the word splitting model is constructed by any implementation manner of the word splitting model construction method provided by the embodiments of the present application;

[0384] A sub-word determination unit 1203, configured to determine at least one sub-word corresponding to each of the candidate words according to the splitting results of each of the candidate words;

[0385] A dictionary construction unit 1204, configured to perform statistical analysis processing on at least one sub-word corresponding to the at least one candidate word to obtain a sub-word dictionary in the target language.

[0386] Based on the speech recognition model construction method provided in the above method embodiments, the embodiments of the present application further provide a speech recognition model construction device, which will be explained and described below with reference to the accompanying drawings.

[0387] Device Embodiment 4

[0388] Embodiment 4 of the device introduces the speech recognition model construction device, and for related content, please refer to the above method embodiments.

[0389] See Figure 13 , this figure is a schematic structural diagram of a speech recognition model construction device provided by an embodiment of the present application.

[0390] The speech recognition model construction device 1300 provided by the embodiments of the present application includes:

[0391] The fourth acquisition unit 1301 is configured to acquire a sample voice in a target language and the actual voice content of the sample voice;

[0392] The voice annotation unit 1302 is configured to determine a voice content annotation of the sample voice according to a sub-word dictionary in the target language and the actual voice content of the sample voice, so that the data dimension of the voice content annotation is equal to the number of sub-words in the sub-word dictionary; wherein, the sub-word dictionary is constructed by any implementation manner of the sub-word dictionary construction method provided in the embodiments of the present application;

[0393] The second construction unit 1303 is configured to construct a speech recognition model by using the sample voice and the voice content annotation of the sample voice.

[0394] Based on the speech recognition method provided in the above method embodiments, the embodiments of the present application further provide a speech recognition device, which will be explained and described below with reference to the accompanying drawings.

[0395] Device Embodiment 5

[0396] Embodiment 5 of the device introduces the speech recognition device, and for related content, please refer to the above method embodiments.

[0397] See Figure 14 , this figure is a schematic structural diagram of a speech recognition device provided in the embodiments of the present application.

[0398] The speech recognition device 1400 provided in the embodiments of the present application includes:

[0399] The fifth acquisition unit 1401 is configured to acquire a voice to be recognized in a target language;

[0400] The speech recognition unit 1402 is configured to perform speech recognition processing on the voice to be recognized by using a pre-constructed speech recognition model, and obtain a speech recognition result of the voice to be recognized; wherein, the speech recognition model is constructed by any implementation manner of the speech recognition model construction method provided in the present application.

[0401] Further, the embodiments of the present application further provide a device, the device includes: a processor, a memory, and a system bus;

[0402] The processor and the memory are connected through the system bus;

[0403] The memory is used to store one or more programs, and the one or more programs include instructions which, when executed by the processor, cause the processor to execute any implementation of the vocabulary splitting model construction method provided by the embodiments of the present application, or execute any implementation of the vocabulary splitting method provided by the embodiments of the present application, or execute any implementation of the sub-word dictionary construction method provided by the embodiments of the present application, or execute any implementation of the speech recognition model construction method provided by the embodiments of the present application, or execute any implementation of the speech recognition method provided by the embodiments of the present application.

[0404] Further, the embodiments of the present application also provide a computer-readable storage medium having instructions stored therein, which when run on a terminal device, cause the terminal device to execute any implementation of the vocabulary splitting model construction method provided by the embodiments of the present application, or execute any implementation of the vocabulary splitting method provided by the embodiments of the present application, or execute any implementation of the sub-word dictionary construction method provided by the embodiments of the present application, or execute any implementation of the speech recognition model construction method provided by the embodiments of the present application, or execute any implementation of the speech recognition method provided by the embodiments of the present application.

[0405] Further, the embodiments of the present application also provide a computer program product which, when run on a terminal device, causes the terminal device to execute any implementation of the vocabulary splitting model construction method provided by the embodiments of the present application, or execute any implementation of the vocabulary splitting method provided by the embodiments of the present application, or execute any implementation of the sub-word dictionary construction method provided by the embodiments of the present application, or execute any implementation of the speech recognition model construction method provided by the embodiments of the present application, or execute any implementation of the speech recognition method provided by the embodiments of the present application.

[0406] From the description of the above embodiments, those skilled in the art can clearly understand that all or part of the steps in the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0407] It should be noted that the various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0408] It should also be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element.

[0409] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a vocabulary splitting model, characterized in that, The method includes: Obtaining sample words in a target language, pronunciation description information of the sample words, and split annotation information of the sample words; Using the sample words, the pronunciation description information of the sample words, and the split annotation information of the sample words to construct a word splitting model; wherein, the word splitting model is used to perform word splitting processing on the sample words with reference to the pronunciation description information of the sample words; The construction process of the word splitting model includes: splitting the sample words according to the smallest writing unit in the target language to obtain a word writing sequence of the sample words; extracting word phoneme features of the sample words from the pronunciation description information of the sample words; performing set processing on the word writing sequence of the sample words and the word phoneme features of the sample words to obtain word description data of the sample words; inputting the word description data of the sample words into a model to be trained to obtain a predicted splitting result of the sample words output by the model to be trained; updating the model to be trained according to the predicted splitting result of the sample words and the split annotation information of the sample words, and continuing to execute the step of inputting the word description data of the sample words into the model to be trained to obtain the predicted splitting result of the sample words output by the model to be trained, until after reaching a preset stop condition, determining the word splitting model according to the model to be trained.

2. The method according to claim 1, wherein The pronunciation description information includes at least one of pronunciation annotation information and pronunciation audio data.

3. The method according to claim 1, wherein The pronunciation description information includes pronunciation annotation information and pronunciation audio data; The extracting the word phoneme features of the sample words from the pronunciation description information of the sample words includes: Performing character extraction processing on the pronunciation annotation information of the sample words to obtain a word phonetic symbol sequence of the sample words; Performing acoustic feature extraction processing on the pronunciation audio data of the sample words to obtain word audio features of the sample words; Performing set processing on the word phonetic symbol sequence of the sample words and the word audio features of the sample words to obtain the word phoneme features of the sample words.

4. The method according to claim 1, wherein The model to be trained includes a feature extraction layer, an encoding layer, a decoding layer, and a decision layer; The determination process of the predicted splitting result of the sample words includes: Inputting the word description data of the sample words into the feature extraction layer to obtain a feature extraction result output by the feature extraction layer; Inputting the feature extraction result into the encoding layer to obtain a feature encoding result output by the encoding layer; Inputting the feature encoding result into the decoding layer to obtain a feature decoding result output by the decoding layer; Inputting the feature decoding result into the decision layer to obtain the predicted splitting result of the sample words output by the decision layer.

5. A method for splitting words, characterized in that, The method includes: Obtaining a word to be split in a target language and pronunciation description information of the word to be split; Determine the splitting result of the vocabulary to be split according to the vocabulary to be split, the pronunciation description information of the vocabulary to be split, and a pre-constructed vocabulary splitting model; wherein, the vocabulary splitting model is constructed by using the vocabulary splitting model construction method according to any one of claims 1-4.

6. A method for constructing a subword dictionary, characterized in that, The method includes: Obtain at least one candidate vocabulary in the target language and the pronunciation description information of each candidate vocabulary; Determine the splitting result of each candidate vocabulary according to each candidate vocabulary, the pronunciation description information of each candidate vocabulary, and a pre-constructed vocabulary splitting model; wherein, the vocabulary splitting model is constructed by using the vocabulary splitting model construction method according to any one of claims 1-4; Determine at least one sub-word corresponding to each candidate vocabulary according to the splitting result of each candidate vocabulary; Perform statistical analysis processing on the at least one sub-word corresponding to the at least one candidate vocabulary to obtain a sub-word dictionary in the target language.

7. A method for constructing a speech recognition model, characterized in that The method includes: Obtain a sample speech in the target language and the actual speech content of the sample speech; Determine the speech content annotation of the sample speech according to the sub-word dictionary in the target language and the actual speech content of the sample speech, so that the data dimension of the speech content annotation is equal to the number of sub-words in the sub-word dictionary; wherein, the sub-word dictionary is constructed by using the sub-word dictionary construction method according to claim 6; Construct a speech recognition model by using the sample speech and the speech content annotation of the sample speech.

8. A speech recognition method, characterized in that, The method includes: Obtain a speech to be recognized in the target language; Perform speech recognition processing on the speech to be recognized by using a pre-constructed speech recognition model to obtain a speech recognition result of the speech to be recognized; wherein, the speech recognition model is constructed by using the speech recognition model construction method according to claim 7.

9. A device, characterized in that, The device includes: a processor, a memory, and a system bus; The processor and the memory are connected through the system bus; The memory is used to store one or more programs, and the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the vocabulary splitting model construction method according to any one of claims 1 to 4, or executes the vocabulary splitting method according to claim 5, or executes the sub-word dictionary construction method according to claim 6, or executes the speech recognition model construction method according to claim 7, or executes the speech recognition method according to claim 8.

10. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium, and when the instructions run on the terminal device, the terminal device executes the vocabulary splitting model construction method according to any one of claims 1 to 4, or executes the vocabulary splitting method according to claim 5, or executes the sub-word dictionary construction method according to claim 6, or executes the speech recognition model construction method according to claim 7, or executes the speech recognition method according to claim 8.

Images