A speech recognition method and related device

Abstract

The present application discloses a speech recognition method and related devices, which relate to the field of speech processing technology, including: after determining the universal decoding text sequence of the speech data to be recognized, first converting the universal decoding text sequence into a generalizable text sequence with a lower element granularity, then expanding the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence of corresponding granularity; and re-segmenting the generalizable text sequence and the expanded text sequence based on a preset long-tail vocabulary language model to obtain a long-tail decoding text sequence; finally, determining the final speech recognition result based on the universal decoding text sequence and the long-tail decoding text sequence. Since this solution can obtain a long-tail decoding text sequence, it can increase the possibility that the final speech recognition result is a long-tail decoding text sequence, and therefore can effectively improve the long-tail vocabulary recognition effect of the end-to-end speech recognition model.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to a speech recognition method and related devices. Background Art

[0002] With the rapid development of artificial intelligence technology, speech recognition, as a key method of human-computer interaction, has been widely used in mobile devices, customer service systems, smart homes, and other fields. End-to-end speech recognition models, which jointly model acoustic, pronunciation, and speech models, are currently the most popular speech recognition frameworks. However, current end-to-end speech recognition models have poor recognition performance for long-tail vocabulary (such as rare or proper nouns).

[0003] At present, the recognition effect of the end-to-end speech recognition model for long-tail words is generally improved by methods such as re-score, hot word incentives, or external language model incentives. However, the improvement of the long-tail word recognition effect of the end-to-end speech recognition model by re-score, hot word incentives, or external language model incentives is not obvious.

[0004] Therefore, how to provide a speech recognition method to effectively improve the long-tail vocabulary recognition effect of the end-to-end speech recognition model has become an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, this application provides a speech recognition method and related devices to effectively improve the long-tail vocabulary recognition effect of the end-to-end speech recognition model. The specific solution is as follows:

[0006] The first aspect of the present application provides a speech recognition method, comprising:

[0007] Determine a universal decoded text sequence of the speech data to be recognized, wherein each element in the universal decoded text sequence is a text unit of a first granularity;

[0008] Converting the universal decoded text sequence into a generalizable text sequence, wherein each element in the generalizable text sequence is a text unit of a second granularity, and the second granularity is smaller than the first granularity;

[0009] Expanding the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence, wherein each element in the expanded text sequence is a text unit of a second granularity;

[0010] Performing a re-scaling operation on the generalizable text sequence and the expanded text sequence based on a preset long-tail vocabulary language model to obtain a long-tail decoded text sequence;

[0011] A final speech recognition result is determined based on the universal decoded text sequence and the long-tail decoded text sequence.

[0012] In a possible implementation, the method for determining similar elements corresponding to each element in the generalizable text sequence includes:

[0013] Determining a similarity matrix, wherein the similarity matrix includes similarities between text units of the second granularity;

[0014] Based on the similarity matrix, similar elements corresponding to each element in the generalizable text sequence are determined.

[0015] In a possible implementation, the expanding the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence includes:

[0016] The similar elements corresponding to the elements in the generalizable text sequence are used to replace elements in the generalizable text sequence to obtain an expanded text sequence.

[0017] In a possible implementation, determining the similarity matrix includes:

[0018] Obtaining text sequence pairs, each text sequence pair comprising a decoded text sequence and a corresponding text sequence label; each element in the text sequence pair is a text unit of the first granularity;

[0019] Converting the text sequence pair into a generalizable text sequence pair, wherein each element in the generalizable text sequence pair is a text unit of a second granularity, where the second granularity is smaller than the first granularity;

[0020] Based on the similarity between the pairs of error elements in the generalizable text sequence pairs, the similarity between the text units of the second granularity is statistically obtained;

[0021] The similarity matrix is ​​constructed based on the similarities between the text units of the second granularity.

[0022] In a possible implementation, determining the similar elements corresponding to each element in the generalizable text sequence based on the similarity matrix includes:

[0023] For each element in the generalizable text sequence, determining whether there is a text unit of a second granularity with a similarity greater than a preset threshold in the similarity matrix;

[0024] If so, the text unit of the second granularity with a similarity greater than a preset threshold is taken as a similar element corresponding to the element.

[0025] In a possible implementation, determining a final speech recognition result based on the universal decoded text sequence and the long-tail decoded text sequence includes:

[0026] Obtaining the score of the universal decoding text sequence, the score of the long-tail decoding text sequence, a preset universal decoding weight, and a preset long-tail decoding weight;

[0027] Determining a final score of the universal decoding text sequence based on the score of the universal decoding text sequence and the preset universal decoding weight;

[0028] Determining a final score of the long-tail decoding text sequence based on the score of the long-tail decoding text sequence and the preset long-tail decoding weight;

[0029] The decoded text sequence with the highest final score is determined as the final speech recognition result.

[0030] A second aspect of the present application provides a speech recognition device, comprising:

[0031] a universal decoding text sequence determining unit, configured to determine a universal decoding text sequence of the speech data to be recognized; each element in the universal decoding text sequence being a text unit of a first granularity;

[0032] a text sequence conversion unit, configured to convert the universal decoding text sequence into a generalizable text sequence, wherein each element in the generalizable text sequence is a text unit of a second granularity, the second granularity being smaller than the first granularity;

[0033] a text sequence expansion unit, configured to expand the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence, wherein each element in the expanded text sequence is a text unit of a second granularity;

[0034] a long-tail decoding text sequence determining unit, configured to perform a re-scaling operation on the generalizable text sequence and the expanded text sequence based on a preset long-tail vocabulary language model to obtain a long-tail decoding text sequence;

[0035] The final speech recognition result determination unit is used to determine the final speech recognition result based on the universal decoded text sequence and the long-tail decoded text sequence.

[0036] In a possible implementation, the text sequence expansion unit includes:

[0037] A similarity matrix determining unit, configured to determine a similarity matrix, wherein the similarity matrix includes similarities between text units of the second granularity;

[0038] A similar element determination unit is configured to determine, based on the similarity matrix, the similar elements corresponding to the elements in the generalizable text sequence.

[0039] In a possible implementation, the text sequence expansion unit is specifically configured to:

[0040] The similar elements corresponding to the elements in the generalizable text sequence are used to replace elements in the generalizable text sequence to obtain an expanded text sequence.

[0041] In a possible implementation, the similarity matrix determination unit is specifically configured to:

[0042] Obtaining text sequence pairs, each text sequence pair comprising a decoded text sequence and a corresponding text sequence label; each element in the text sequence pair is a text unit of the first granularity;

[0043] Converting the text sequence pair into a generalizable text sequence pair, wherein each element in the generalizable text sequence pair is a text unit of a second granularity, where the second granularity is smaller than the first granularity;

[0044] Based on the similarity between the pairs of error elements in the generalizable text sequence pairs, the similarity between the text units of the second granularity is statistically obtained;

[0045] The similarity matrix is ​​constructed based on the similarities between the text units of the second granularity.

[0046] In a possible implementation, the similar element determining unit is specifically configured to:

[0047] For each element in the generalizable text sequence, determining whether there is a text unit of a second granularity with a similarity greater than a preset threshold in the similarity matrix;

[0048] If so, the text unit of the second granularity with a similarity greater than a preset threshold is taken as a similar element corresponding to the element.

[0049] In a possible implementation, the final speech recognition result determination unit is specifically configured to:

[0050] Obtaining the score of the universal decoding text sequence, the score of the long-tail decoding text sequence, a preset universal decoding weight, and a preset long-tail decoding weight;

[0051] Determining a final score of the universal decoding text sequence based on the score of the universal decoding text sequence and the preset universal decoding weight;

[0052] Determining a final score of the long-tail decoding text sequence based on the score of the long-tail decoding text sequence and the preset long-tail decoding weight;

[0053] The decoded text sequence with the highest final score is determined as the final speech recognition result.

[0054] The third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the speech recognition method of the first aspect or any implementation of the first aspect.

[0055] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0056] The memory is used to store computer programs;

[0057] The processor is used to execute the computer program so that the electronic device can implement the speech recognition method of the first aspect or any implementation manner of the first aspect.

[0058] In a fifth aspect, the present application provides a computer-readable storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the speech recognition method of the first aspect or any implementation of the first aspect.

[0059] By means of the above technical solution, the speech recognition method and related devices provided by the present application. After determining the universal decoding text sequence of the speech data to be recognized, the universal decoding text sequence is first converted into a generalizable text sequence with a lower element granularity, and then the generalizable text sequence is expanded using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence of corresponding granularity; and based on a preset long-tail vocabulary language model, the generalizable text sequence and the expanded text sequence are re-pointed to obtain a long-tail decoding text sequence; finally, based on the universal decoding text sequence and the long-tail decoding text sequence, the final speech recognition result is determined. Since this solution can obtain a long-tail decoding text sequence, it can increase the possibility that the final speech recognition result is a long-tail decoding text sequence, and therefore, can effectively improve the long-tail vocabulary recognition effect of the end-to-end speech recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0061] Figure 1 A flowchart of a speech recognition method provided in an embodiment of the present application;

[0062] Figure 2 A flowchart of a method for determining similar elements corresponding to each element in a generalized text sequence provided in an embodiment of the present application;

[0063] Figure 3 A flowchart of a method for determining a similarity matrix provided in an embodiment of the present application;

[0064] Figure 4 A flowchart illustrating a method for determining a final speech recognition result based on a general decoded text sequence and a long-tail decoded text sequence provided in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of the structure of a speech recognition device provided in an embodiment of the present application;

[0066] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0068] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0069] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0070] With the rapid development of artificial intelligence technology, voice recognition, as one of the important ways of human-computer interaction, has been widely used in mobile devices, customer service systems, smart homes and other fields.

[0071] Traditional speech recognition systems typically require pre-built pronunciation systems tailored to the specific language, resulting in accurate phoneme segmentation and pronunciation dictionaries. This process is often time-consuming and error-prone. Furthermore, because the system consists of multiple independent modules, errors in each module can be passed on to the next, leading to cumulative errors.

[0072] Consequently, over the past few years, end-to-end models have garnered widespread attention in the field of speech recognition. In speech recognition tasks, end-to-end models replace the traditional combination of acoustic, pronunciation, and language models with a single neural network. They can be optimized with a unified goal, significantly simplifying the construction and maintenance of speech recognition systems. Furthermore, leveraging the unique advantages of internet big data, end-to-end speech recognition models, which combine acoustic, pronunciation, and language models, have already outperformed traditional speech recognition systems and become the most sought-after speech recognition framework.

[0073] The most effective end-to-end speech recognition model currently available is mainly based on a neural network structure of an attention model. This structure can convert a speech sequence word by word into a corresponding recognition result sequence. It consists of an encoder and a decoder. The encoder can be a popular network structure such as DNN, CNN, RNN, LSTM, TRANSFORMER, etc., which is used to extract high-order latent features from speech features; the decoder can be a network structure such as RNN, LSTM, etc., and it also contains a set of attention modules inside to convert the high-order latent features extracted by the encoder into a probability distribution of a set of character sequences, and then obtain the speech recognition sequence through this probability distribution.

[0074] End-to-end speech recognition models avoid the initial manual design process and can achieve good performance through large-scale data training. However, for long-tail words (such as rare or proper nouns), they generally appear less frequently in standard training sets, which may cause the end-to-end speech recognition model to fail to learn the appropriate spelling and contextual usage of long-tail words. As a result, the end-to-end speech recognition model finds it difficult to capture enough information to accurately identify long-tail words, resulting in current end-to-end speech recognition models having poor recognition effects on long-tail words (such as rare or proper nouns).

[0075] At present, the recognition effect of the end-to-end speech recognition model on long-tail words is generally improved through re-score, hot word incentives or external language model incentives.

[0076] Rescoring refers to retaining several of the most likely recognition results after model decoding, and then rescoring and ranking them using a language model. Hot word incentives provide additional scoring incentives for specific words during the model decoding process, ensuring they are not clipped during the decoding process. External language model incentives constrain decoding results using a language model generated by customized sentences that are highly relevant to long-tail vocabulary.

[0077] However, in the end-to-end speech recognition model, long-tail words have extremely low scores during speech recognition and are easily pruned during decoding, making the re-scoring method unusable. Hot word excitation or external language model excitation methods, on the one hand, have difficulty balancing the boundary between successful excitation and false triggering during decoding due to the low scores of the acoustic model. On the other hand, there may be situations where long-tail words do not appear in all candidates and have no chance of being excited at all.

[0078] It can be seen that the long-tail vocabulary recognition effect of the end-to-end speech recognition model is not significantly improved by re-score, hot word incentives, or external language model incentives.

[0079] To solve the above problems, the present invention provides a speech recognition method that can effectively improve the long-tail vocabulary recognition performance of an end-to-end speech recognition model.

[0080] The speech recognition method of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0081] Reference Figure 1 , Figure 1 A flow chart of a speech recognition method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, a speech recognition method provided in an embodiment of the present application may include the following steps, which are described in detail below.

[0082] S101: Determine a universal decoded text sequence of speech data to be recognized; each element in the universal decoded text sequence is a text unit of a first granularity;

[0083] In the present application, the speech data to be recognized can be speech data in any language and in any field. The decoded text sequence of the speech data to be recognized can be obtained after an end-to-end speech recognition model performs speech recognition on the speech data to be recognized.

[0084] In one possible implementation, determining the universal decoding text sequence of the speech data to be recognized includes: first extracting speech features from the speech data to be recognized, the speech features can be common speech features such as FilterBank, MFCC, etc., and FilterBank features can be used in the application. Then the speech features of the speech data to be recognized are input into an end-to-end speech recognition model, and the end-to-end speech recognition model is combined with a universal language model for decoding. The top-ranked preset number of text sequences obtained in the last decoding step are used as the universal decoding text sequences of the speech data to be recognized, and the scores of each universal decoding text sequence are also determined. The value of the preset number can be set based on the scenario requirements, and is not limited in this application.

[0085] In one possible implementation, the end-to-end speech recognition model, when combined with a general language model for decoding, can adopt a beam search decoding method, which is consistent with the standard end-to-end speech recognition model decoding process and is not described in detail in this application.

[0086] S102: Converting the universal decoded text sequence into a generalizable text sequence, wherein each element in the generalizable text sequence is a text unit of a second granularity, and the second granularity is smaller than the first granularity;

[0087] In the present application, there may be multiple universal decoding text sequences, and for each universal decoding text sequence, the universal decoding text sequence needs to be converted into a generalizable text sequence.

[0088] In one possible implementation, taking Chinese as an example, the first granularity can be the word level, and the second granularity can be the single-character level, syllable level, or phoneme level; taking English as an example, the first granularity can be the word level, and the second granularity can be the subword level or letter level.

[0089] In one possible implementation, the decoded text sequence can be converted into a generalizable text sequence by using a whole word splitting method or a pronunciation prediction method. It should be noted that the whole word splitting method and the pronunciation prediction method are existing mature technologies and will not be described in detail in this application.

[0090] S103: Expanding the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence, wherein each element in the expanded text sequence is a text unit of a second granularity;

[0091] In the present application, the number of universal decoding text sequences can be multiple. For each of the universal decoding text sequences, the universal decoding text sequence needs to be converted into a generalizable text sequence. Therefore, multiple generalizable text sequences can be obtained. Therefore, in the present application, for each of the generalizable text sequences, the generalizable text sequence needs to be expanded using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence.

[0092] In one possible implementation, the method of expanding the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence includes: replacing elements in the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence.

[0093] Specifically, for each element of each generalizable text sequence, it is determined whether the element has a similar element. If the element has a similar element, the element is replaced with the similar element to obtain an expanded text sequence. It should be noted that multiple elements in the generalizable sequence may have similar elements, and the multiple elements can be fully or partially replaced to obtain one or more expanded text sequences.

[0094] S104: performing a re-scaling operation on the generalizable text sequence and the expanded text sequence based on a preset long-tail vocabulary language model to obtain a long-tail decoded text sequence;

[0095] In this application, the preset long-tail vocabulary language model is an existing mature technology and will not be described in detail in this application. The re-score operation on the text sequence based on the preset long-tail vocabulary language model is also an existing mature technology and will not be described in detail in this application. The re-score operation is performed on the generalizable text sequence and the expanded text sequence based on the preset long-tail vocabulary language model to obtain the long-tail decoded text sequence and the score of each long-tail decoded text sequence.

[0096] S105: Determine a final speech recognition result based on the universal decoded text sequence and the long-tail decoded text sequence.

[0097] In the present application, the final speech recognition result may be determined based on the score of the universal decoded text sequence and the score of the long-tail decoded text sequence.

[0098] The speech recognition method provided in this embodiment, after determining the universal decoding text sequence of the speech data to be recognized, first converts the universal decoding text sequence into a generalizable text sequence with a lower element granularity, then expands the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence of corresponding granularity; and based on a preset long-tail vocabulary language model, performs a re-segmentation operation on the generalizable text sequence and the expanded text sequence to obtain a long-tail decoding text sequence; finally, based on the universal decoding text sequence and the long-tail decoding text sequence, determines the final speech recognition result. Since this solution can obtain a long-tail decoding text sequence, it can increase the possibility that the final speech recognition result is a long-tail decoding text sequence, and therefore can effectively improve the long-tail vocabulary recognition effect of the end-to-end speech recognition model.

[0099] In another embodiment of the present application, a method for determining similar elements corresponding to each element in the generalizable text sequence is described in detail.

[0100] Reference Figure 2 , Figure 2 A flowchart of a method for determining similar elements corresponding to each element in a generalized text sequence provided in an embodiment of the present application may include the following steps:

[0101] S201: Determine a similarity matrix, wherein the similarity matrix includes similarities between text units of the second granularity;

[0102] S202: Determine similar elements corresponding to each element in the generalizable text sequence based on the similarity matrix.

[0103] In a possible implementation, determining the similar elements corresponding to each element in the generalizable text sequence based on the similarity matrix includes:

[0104] For each element in the generalizable text sequence, determining whether there is a text unit of a second granularity with a similarity greater than a preset threshold in the similarity matrix;

[0105] If so, the text unit of the second granularity with a similarity greater than a preset threshold is taken as a similar element corresponding to the element.

[0106] In another embodiment of the present application, a specific implementation method of determining the similarity matrix is ​​described in detail.

[0107] Reference Figure 3 , Figure 3 A flowchart of a method for determining a similarity matrix provided in an embodiment of the present application may include the following steps:

[0108] S301: Acquire text sequence pairs, each text sequence pair including a decoded text sequence and a corresponding text sequence label; each element in the text sequence pair is a text unit of the first granularity;

[0109] In this application, text sequence pairs can be collected in different scenarios, and different collection strategies can be adopted for different scenarios. For languages ​​with large data volumes or general scenarios, multi-domain scenarios can be collected to improve the generalization of text sequence pairs; for specific scenarios, the TopN decoded text sequences and corresponding text sequence labels in the scenario can be collected to improve the targetedness of text sequence pairs.

[0110] S302: Convert the text sequence pair into a generalizable text sequence pair, wherein each element in the generalizable text sequence pair is a text unit of a second granularity, and the second granularity is smaller than the first granularity;

[0111] In this application, reference may be made to the related description of converting the universal decoding text sequence into a generalizable text sequence, which will not be repeated here.

[0112] S303: Based on the similarity between the pairs of error elements in the generalizable text sequence pairs, statistically obtain the similarity between the text units of the second granularity;

[0113] In the present application, the incorrect element pairing in the generalizable text sequence pair can be determined based on the edit distance of the generalizable text sequence pair. The specific implementation method is an existing mature technology and will not be described in detail in this application.

[0114] In a possible implementation, the similarity between each pair of error elements may be measured using the cosine distance.

[0115] It should be noted that in the present application, there may be multiple generalizable text sequence pairs containing the same erroneous element pairing, and the similarity of the erroneous element pairing in each generalizable text sequence pair may be different. In this case, the similarity that meets the requirements can be screened out by presetting a similarity threshold, and then the similarities that meet the requirements are averaged to obtain the final similarity of the erroneous element pairing.

[0116] S304: Constructing the similarity matrix based on the similarities between the text units of the second granularity.

[0117] In another embodiment of the present application, a specific implementation method of determining the final speech recognition result based on the universal decoding text sequence and the long-tail decoding text sequence is described.

[0118] In a possible implementation, the decoded text sequence with the highest score may be determined as the final speech recognition result.

[0119] Taking into account that different scenarios require different recognition effects for long-tail words, in another possible implementation, general decoding weights and preset long-tail decoding weights can be preset, and the scores of each decoded text sequence can be re-determined by combining the preset general decoding weights and the preset long-tail decoding weights to determine the final speech recognition result.

[0120] Reference Figure 4 , Figure 4 A flowchart of a method for determining a final speech recognition result based on a general decoded text sequence and a long-tail decoded text sequence provided in an embodiment of the present application may include the following steps:

[0121] S401: Obtaining the score of the general decoding text sequence, the score of the long-tail decoding text sequence, a preset general decoding weight, and a preset long-tail decoding weight;

[0122] In this application, the specific values ​​of the preset general decoding weight and the preset long-tail decoding weight can be set based on scenario requirements and are not limited in this application.

[0123] S402: Determine a final score of the universal decoding text sequence based on the score of the universal decoding text sequence and the preset universal decoding weight;

[0124] S403: Determine a final score of the long-tail decoding text sequence based on the score of the long-tail decoding text sequence and the preset long-tail decoding weight;

[0125] S404: Determine the decoded text sequence with the highest final score as the final speech recognition result.

[0126] In summary, the present application proposes a speech recognition method, which degenerates the initial speech recognition results into smaller units during the decoding process of the end-to-end speech recognition model, and then combines them with the long-tail vocabulary language model for re-stimulation, thereby avoiding the problem that neither direct use of long-tail word excitation nor long-tail vocabulary language model excitation can decode the correct results, and ultimately can effectively improve the long-tail vocabulary recognition effect of the end-to-end speech recognition model.

[0127] The above describes a speech recognition method provided in an embodiment of the present application. The following describes a device for executing the above speech recognition method.

[0128] See also Figure 5 , Figure 5 This is a structural diagram of a speech recognition device provided in an embodiment of the present application. Figure 5 As shown, the speech recognition device includes:

[0129] The universal decoding text sequence determining unit 11 is used to determine the universal decoding text sequence of the speech data to be recognized; each element in the universal decoding text sequence is a text unit of the first granularity;

[0130] a text sequence conversion unit 12, configured to convert the universal decoded text sequence into a generalizable text sequence, wherein each element in the generalizable text sequence is a text unit of a second granularity, the second granularity being smaller than the first granularity;

[0131] a text sequence expansion unit 13, configured to expand the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence, wherein each element in the expanded text sequence is a text unit of a second granularity;

[0132] A long-tail decoded text sequence determining unit 14 is configured to perform a re-scaling operation on the generalizable text sequence and the expanded text sequence based on a preset long-tail vocabulary language model to obtain a long-tail decoded text sequence;

[0133] The final speech recognition result determining unit 15 is configured to determine a final speech recognition result based on the universal decoded text sequence and the long-tail decoded text sequence.

[0134] In a possible implementation, the text sequence expansion unit includes:

[0135] A similarity matrix determining unit, configured to determine a similarity matrix, wherein the similarity matrix includes similarities between text units of the second granularity;

[0136] A similar element determination unit is configured to determine, based on the similarity matrix, the similar elements corresponding to the elements in the generalizable text sequence.

[0137] In a possible implementation, the text sequence expansion unit is specifically configured to:

[0138] The similar elements corresponding to the elements in the generalizable text sequence are used to replace elements in the generalizable text sequence to obtain an expanded text sequence.

[0139] In a possible implementation, the similarity matrix determination unit is specifically configured to:

[0140] Obtaining text sequence pairs, each text sequence pair comprising a decoded text sequence and a corresponding text sequence label; each element in the text sequence pair is a text unit of the first granularity;

[0141] Converting the text sequence pair into a generalizable text sequence pair, wherein each element in the generalizable text sequence pair is a text unit of a second granularity, where the second granularity is smaller than the first granularity;

[0142] Based on the similarity between the pairs of error elements in the generalizable text sequence pairs, the similarity between the text units of the second granularity is statistically obtained;

[0143] The similarity matrix is ​​constructed based on the similarities between the text units of the second granularity.

[0144] In a possible implementation, the similar element determining unit is specifically configured to:

[0145] For each element in the generalizable text sequence, determining whether there is a text unit of a second granularity with a similarity greater than a preset threshold in the similarity matrix;

[0146] If so, the text unit of the second granularity with a similarity greater than a preset threshold is taken as a similar element corresponding to the element.

[0147] In a possible implementation, the final speech recognition result determination unit is specifically configured to:

[0148] Obtaining the score of the universal decoding text sequence, the score of the long-tail decoding text sequence, a preset universal decoding weight, and a preset long-tail decoding weight;

[0149] Determining a final score of the universal decoding text sequence based on the score of the universal decoding text sequence and the preset universal decoding weight;

[0150] Determining a final score of the long-tail decoding text sequence based on the score of the long-tail decoding text sequence and the preset long-tail decoding weight;

[0151] The decoded text sequence with the highest final score is determined as the final speech recognition result.

[0152] An electronic device is also provided in an embodiment of the present application. Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0153] like Figure 6As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 602 or programs loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing device 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0154] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 6 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0155] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the speech recognition methods provided in the embodiments of the present application.

[0156] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any speech recognition method provided in the embodiment of the present application.

[0157] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0158] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0159] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0160] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A speech recognition method, characterized in that: include: Determine a universal decoded text sequence of the speech data to be recognized, wherein each element in the universal decoded text sequence is a text unit of a first granularity; Converting the universal decoded text sequence into a generalizable text sequence, wherein each element in the generalizable text sequence is a text unit of a second granularity, and the second granularity is smaller than the first granularity; Expanding the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence, wherein each element in the expanded text sequence is a text unit of a second granularity; Performing a re-scaling operation on the generalizable text sequence and the expanded text sequence based on a preset long-tail vocabulary language model to obtain a long-tail decoded text sequence; A final speech recognition result is determined based on the universal decoded text sequence and the long-tail decoded text sequence.

2. The method according to claim 1, characterized in that The method for determining similar elements corresponding to each element in the generalizable text sequence includes: Determining a similarity matrix, wherein the similarity matrix includes similarities between text units of the second granularity; Based on the similarity matrix, similar elements corresponding to each element in the generalizable text sequence are determined.

3. The method according to claim 1, characterized in that The method of expanding the generalizable text sequence by using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence includes: The similar elements corresponding to the elements in the generalizable text sequence are used to replace elements in the generalizable text sequence to obtain an expanded text sequence.

4. The method according to claim 2, characterized in that Determining the similarity matrix includes: Obtaining text sequence pairs, each text sequence pair comprising a decoded text sequence and a corresponding text sequence label; each element in the text sequence pair is a text unit of the first granularity; Converting the text sequence pair into a generalizable text sequence pair, wherein each element in the generalizable text sequence pair is a text unit of a second granularity, where the second granularity is smaller than the first granularity; Based on the similarity between the pairs of error elements in the generalizable text sequence pairs, the similarity between the text units of the second granularity is statistically obtained; The similarity matrix is ​​constructed based on the similarities between the text units of the second granularity.

5. The method according to claim 2, characterized in that The determining, based on the similarity matrix, the similar elements corresponding to the elements in the generalizable text sequence includes: For each element in the generalizable text sequence, determining whether there is a text unit of a second granularity with a similarity greater than a preset threshold in the similarity matrix; If so, the text unit of the second granularity with a similarity greater than a preset threshold is taken as a similar element corresponding to the element.

6. The method according to claim 1, characterized in that Determining a final speech recognition result based on the universal decoded text sequence and the long-tail decoded text sequence includes: Obtaining the score of the universal decoding text sequence, the score of the long-tail decoding text sequence, a preset universal decoding weight, and a preset long-tail decoding weight; Determining a final score of the universal decoding text sequence based on the score of the universal decoding text sequence and the preset universal decoding weight; Determining a final score of the long-tail decoding text sequence based on the score of the long-tail decoding text sequence and the preset long-tail decoding weight; The decoded text sequence with the highest final score is determined as the final speech recognition result.

7. A speech recognition device, characterized in that: include: a universal decoding text sequence determining unit, configured to determine a universal decoding text sequence of the speech data to be recognized; each element in the universal decoding text sequence being a text unit of a first granularity; a text sequence conversion unit, configured to convert the universal decoding text sequence into a generalizable text sequence, wherein each element in the generalizable text sequence is a text unit of a second granularity, the second granularity being smaller than the first granularity; a text sequence expansion unit, configured to expand the generalizable text sequence using similar elements corresponding to each element in the generalizable text sequence to obtain an expanded text sequence, wherein each element in the expanded text sequence is a text unit of a second granularity; a long-tail decoding text sequence determining unit, configured to perform a re-scaling operation on the generalizable text sequence and the expanded text sequence based on a preset long-tail vocabulary language model to obtain a long-tail decoding text sequence; The final speech recognition result determination unit is used to determine the final speech recognition result based on the universal decoded text sequence and the long-tail decoded text sequence.

8. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the speech recognition method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program so that the electronic device can implement the speech recognition method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the speech recognition method according to any one of claims 1 to 6.

Images