A speech recognition method, device and storage medium

By dynamically loading domain models and combining the scores of the first and second models for a weighted average, the efficiency and accuracy issues of speech recognition when domain-related corpora are insufficient are solved, and efficient and accurate speech recognition is achieved.

CN110797026BActive Publication Date: 2024-11-26TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910880013.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-09-17
Publication Date
2024-11-26
Estimated Expiration
2039-10-31

AI Technical Summary

Technical Problem

Existing speech recognition technologies suffer from low efficiency and accuracy when there is a lack of domain-specific corpora.

Method used

By dynamically loading domain models, the first model is used to identify words in common expressions, and when the first model cannot identify them, the second domain model is loaded. The identification is performed according to the domain type to which the language words belong. The first and second scores are combined and weighted to determine the final identification result.

Benefits of technology

It improves the accuracy and efficiency of speech recognition, especially in the recognition of domain-specific vocabulary, without affecting the recognition of general expressions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110797026B_ABST
    Figure CN110797026B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a speech recognition method, device and storage medium, including: extracting the features of a speech signal stream and inputting them into a first model to obtain a first recognition result, and determining a first score of the first recognition result, wherein the first model is used to recognize the vocabulary of general statements; when the first model cannot recognize the language vocabulary in the speech signal stream, loading a second model according to the field type to which the language vocabulary belongs, inputting the language vocabulary into the second model to obtain a second recognition result, and determining a second score of the second recognition result, wherein the second model is used to recognize the vocabulary of field statements; determining the final recognition result according to the first score and the second score. The embodiment of the present application improves the accuracy and efficiency of speech recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a speech recognition method, device and storage medium. Background Art

[0002] The key technologies of speech technology include automatic speech recognition technology (ASR), text to speech (TTS) and voiceprint recognition technology. Enabling computers to listen, see, speak and feel is the future development direction of human-computer interaction, among which speech becomes one of the most promising human-computer interaction methods in the future. Speech recognition can be performed in the following ways: First, combine the corpus training related to each field and the general corpus to retrain the language model and resynthesize the decoding graph HCLG. Second, add domain-related audio and text to the end-to-end speech recognition system and retrain the relevant model on a large scale of corpus. However, in actual scenarios, the entire model needs to be retrained and deployed, resulting in low speech recognition efficiency. And because there are not many domain-related corpora, the recognition accuracy is low. Summary of the invention

[0003] The embodiments of the present application provide a speech recognition method, device and storage medium, which can improve the accuracy and efficiency of speech recognition.

[0004] In a first aspect, an embodiment of the present application provides a speech recognition method, comprising:

[0005] Extracting features of the speech signal stream and inputting them into a first model to obtain a first recognition result, and determining a first score of the first recognition result, wherein the first model is used to recognize vocabulary of common sayings;

[0006] When the first model cannot recognize the language vocabulary in the speech signal stream, loading a second model according to the domain type to which the language vocabulary belongs, inputting the language vocabulary into the second model to obtain a second recognition result, and determining a second score of the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement;

[0007] A final recognition result is determined according to the first score and the second score.

[0008] The vocabulary of the first model includes a special identifier, and the loading of the second model according to the domain type to which the language vocabulary belongs includes:

[0009] Determine a special identifier corresponding to the domain type to which the language vocabulary belongs;

[0010] According to the special identifier, the second model is searched for and loaded from a plurality of preset domain models.

[0011] The first model includes a first commonly used language model and a first re-scoring language model, the first commonly used language model is generated based on a bigram language model, and the first re-scoring language model is generated based on a pentagram language model;

[0012] The extracting the features of the speech signal stream and inputting them into the first model to obtain a first recognition result, and determining a first score of the first recognition result comprises:

[0013] Recognize the speech signal stream according to the first common language model to obtain a plurality of first recognition results, where each first recognition result corresponds to one first score;

[0014] The plurality of first recognition results are sorted according to the first re-scoring language model, and the first recognition result corresponding to the highest first score is selected.

[0015] The second model includes a second domain language model and a second re-scoring language model, and the second domain language model is generated according to a unigram language model;

[0016] The step of inputting the language vocabulary into the second model to obtain a second recognition result, and determining a second score of the second recognition result comprises:

[0017] Recognize the language vocabulary according to the second domain language model to obtain a plurality of second recognition results, each second recognition result corresponding to one second score;

[0018] The plurality of second recognition results are sorted according to the second scoring language model, and the second recognition result corresponding to the highest second score is selected.

[0019] Wherein, determining the final recognition result according to the first score and the second score includes:

[0020] calculating a weighted average of the first score and the second score;

[0021] The final recognition result is determined according to the weighted average value.

[0022] Wherein, the second domain language model includes the vocabulary to be strengthened of the first model; and the method further includes:

[0023] The recognition result for the vocabulary to be strengthened identified by the second model is used to replace the recognition result for the vocabulary to be strengthened identified by the first model to obtain the final recognition result.

[0024] In a second aspect, an embodiment of the present application provides a speech recognition device, including:

[0025] a processing module, configured to extract features of a speech signal stream and input the features into a first model to obtain a first recognition result, and determine a first score of the first recognition result, wherein the first model is used to recognize vocabulary of a common saying;

[0026] The processing module is further configured to, when the first model cannot recognize the language vocabulary in the speech signal stream, load a second model according to the domain type to which the language vocabulary belongs, input the language vocabulary into the second model to obtain a second recognition result, and determine a second score for the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement;

[0027] A determination module is used to determine a final recognition result according to the first score and the second score.

[0028] The processing module is further used to determine a special identifier corresponding to the domain type to which the language vocabulary belongs; and according to the special identifier, search for the second model from a plurality of preset domain models and load it.

[0029] The first model includes a first commonly used language model and a first re-scoring language model, the first commonly used language model is generated based on a bigram language model, and the first re-scoring language model is generated based on a pentagram language model;

[0030] The processing module is further used to recognize the speech signal stream according to the first commonly used language model to obtain multiple first recognition results, one first recognition result corresponding to one first score; sort the multiple first recognition results according to the first re-scoring language model, and select the first recognition result corresponding to the highest first score.

[0031] The second model includes a second domain language model and a second re-scoring language model, and the second domain language model is generated according to a unigram language model;

[0032] The processing module is further used to recognize the language vocabulary according to the second domain language model to obtain multiple second recognition results, one second recognition result corresponds to one second score; sort the multiple second recognition results according to the second re-scoring language model, and select the second recognition result corresponding to the highest second score.

[0033] Wherein, the determination module is further used to calculate a weighted average of the first score and the second score; and determine the final recognition result according to the weighted average.

[0034] wherein the second domain language model includes the vocabulary to be strengthened of the first model;

[0035] The processing module is further configured to replace the recognition result for the vocabulary to be strengthened identified by the first model with the recognition result for the vocabulary to be strengthened identified by the second model, so as to obtain the final recognition result.

[0036] In a third aspect, an embodiment of the present application provides a speech recognition device, comprising: a processor, a memory and a communication bus, wherein the communication bus is used to realize connection and communication between the processor and the memory, and the processor executes a program stored in the memory to implement the steps in a speech recognition method provided in the first aspect above.

[0037] In a possible design, the speech recognition device provided by the present application may include a module for executing the corresponding behavior in the above method. The module may be software and / or hardware.

[0038] Another aspect of the embodiments of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are suitable for being loaded by a processor and executing the methods described in the above aspects.

[0039] Another aspect of the embodiments of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the methods described in the above aspects.

[0040] In the implementation of the present application, the features of the speech signal stream are extracted and input into the first model to obtain a first recognition result, and a first score of the first recognition result is determined, and the first model is used to identify the vocabulary of general statements; when the first model cannot recognize the language vocabulary in the speech signal stream, a second model is loaded according to the domain type to which the language vocabulary belongs, and the language vocabulary is input into the second model to obtain a second recognition result, and a second score of the second recognition result is determined, and the second model is used to identify the vocabulary of domain statements; according to the first score and the second score, the final recognition result is determined. By dynamically loading the domain model, the efficiency and accuracy of speech recognition are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1is a structural schematic diagram of a speech recognition system proposed in an embodiment of the present application;

[0043] Figure 2 It is a flowchart of a speech recognition method provided by an embodiment of the present application;

[0044] Figure 3 This is a state rollback schematic diagram provided in an embodiment of the present application;

[0045] Figure 4 It is a schematic diagram of a replacement operation provided in an embodiment of the present application;

[0046] Figure 5 is a flow chart of another speech recognition method provided in an embodiment of the present application;

[0047] Figure 6 is a structural schematic diagram of a speech recognition device provided in an embodiment of the present application;

[0048] Figure 7 It is a structural diagram of a speech recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] like Figure 1 As shown, Figure 1It is a structural diagram of a speech recognition system proposed in an embodiment of the present application. The speech recognition system includes a ROOT model and a domain model, wherein the ROOT model is used to identify the vocabulary of general statements, the ROOT model includes a ROOT model (base) and a ROOT model (big), the ROOT model (base) includes an acoustic model and a decoding graph HCLG, the acoustic model and the decoding graph HCLG are used to identify the features of the extracted speech signal stream, and the ROOT model (big) is used to sort and re-score the recognition results. The domain model is used to identify the vocabulary of domain statements, and the domain model may include a domain model (OOV) and a domain re-scoring (rescore) model, wherein the domain model (OOV) includes a domain model 1, a domain model 2..., a domain model n, and the like. If it is necessary to identify the vocabulary of a certain domain, the corresponding domain model can be loaded. Each domain model includes an acoustic model and a decoding graph OOV-HCLG. The acoustic model and the decoding graph OOV-HCLG are used to identify the vocabulary of the domain, and the domain re-scoring model can be used to sort and re-score the recognition results. The speech recognition system can be applied to Yun Xiaowei speech recognition assistant, WeChat applets, and the like.

[0051] like Figure 2 As shown, Figure 2 : is a flow chart of a speech recognition method provided in an embodiment of the present application. The steps in the embodiment of the present application at least include:

[0052] S201, extracting features of a speech signal stream and inputting them into a first model to obtain a first recognition result, and determining a first score of the first recognition result, wherein the first model is used to recognize vocabulary of common sayings.

[0053] Among them, the first model includes a first commonly used language model and a first re-scoring language model, the first commonly used language model is generated based on a bigram language model, and the first re-scoring language model is generated based on a five-gram language model; the speech signal stream can be recognized according to the first commonly used language model to obtain multiple first recognition results, and one first recognition result corresponds to one first score; then the multiple first recognition results are sorted according to the first re-scoring language model, and the first recognition result corresponding to the highest first score is selected.

[0054] like Figure 1As shown, the first model can be a ROOT model, which obtains an acoustic model A and a language model G1 from audio and corpus in a general field, respectively, wherein the language model G1 is split into 2-gram G11 (a first commonly used language model) and 5-gram G12 (a first re-scored language model), and a decoding graph HCLG can be generated according to G11, and the acoustic model and the decoding graph HCLG are used to recognize the speech of a general statement, and then the recognition results of the acoustic model and the decoding graph HCLG are re-scored through G12.

[0055] For example, the corpus "I want to know in which hall Zhu Yuanzhang attends court" is segmented. I want / know / Zhu Yuanzhang / in / which hall / attends court is a 1-gram. I want / attend court / which hall to attend court / I want to know / Zhu Yuanzhang is / know Zhu Yuanzhang is a 2-gram. I want to know Zhu Yuanzhang is / I want to know in which hall Zhu Yuanzhang is / which hall Zhu Yuanzhang attends court is a 5-gram. The above examples only illustrate part of the vocabulary. Among them, G11 includes 1-gram and 2-gram, and G12 includes 1-gram, 2-gram, 3-gram, 4-gram and 5-gram.

[0056] S202, when the first model cannot recognize the language vocabulary in the speech signal stream, load a second model according to the domain type to which the language vocabulary belongs, input the language vocabulary into the second model to obtain a second recognition result, and determine a second score of the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement.

[0057] In a specific implementation, a special identifier corresponding to the domain type to which the language vocabulary belongs may be determined; and according to the special identifier, the second model is searched from a plurality of preset domain models and loaded.

[0058] For example, Figure 3 As shown, Figure 3 This is a state fallback diagram provided by an embodiment of the present application. After feature extraction of the signal stream, the ROOT model is entered for online decoding. If the ROOT model cannot recognize one or some domain vocabulary, it needs to fall back. Among them, $OOV represents a 1-gram state fallback identifier. During the decoding graph search, the $OOV identifier is included in the fallback state of the ROOT language model. If a 1-gram fallback state is searched, the pre-specified domain OOV-HCLG will be entered for a decoding graph search, and then the search results will be replaced (replace) through a weighted finite state transducer (Weighted Finite-State Transducer, WFST). Another example Figure 4 As shown, Figure 4It is a schematic diagram of a replacement operation provided by an embodiment of the present application, which replaces the edges of $OOV in the search results.

[0059] Among them, the second model includes a second domain language model and a second rescoring language model. The second domain language model is generated according to a unigram language model. Multiple second recognition results can be obtained by recognizing the language vocabulary according to the second domain language model, and one second recognition result corresponds to one second score. The multiple second recognition results are sorted according to the second rescoring language model, and the second recognition result corresponding to the highest second score is selected.

[0060] As Figure 1 shown, the second model can be a domain model, which is divided into two parts: OOV-HCLG and rescore-HCLG. The language model of OOV-HCLG consists of the set of words {oov} not registered in the ROOT model in the domain scenario and the set of words { <s> word< / s>} of the words to be strengthened word in the ROOT model, which is composed of 1-grams. For example, words such as Duanmen / Xianximen / Yongkangzuomen are all in the set of words {oov} not registered in the ROOT model, which belong to the vocabulary in the historical domain and are not commonly used words. Another example is that "dian" is easily recognized as "dian", so "dian" is a word to be strengthened.

[0061] Among them, the language model of rescore-HCLG is generated by using the above-mentioned word set {oov} and specifying sentence patterns, and is used to rescore the recognition results of the language model of OOV-HCLG to strengthen the recognition of domain expressions. When decoding through the ROOT model (base) and the language model of OOV-HCLG, the 5-gram language model (big) of the ROOT model and the rescore-HCLG of the domain model are respectively used for rescoring synchronously.

[0062] It should be noted that models in different domains can be integrated into a speech recognition system. For independent retrieval (query) requests in each domain, the domain model is hot-loaded in real time, that is, when loading a new domain model, the already loaded domain model is not released.

[0063] S203. Determine the final recognition result according to the first score and the second score.

[0064] In a specific implementation, a weighted average of the first score and the second score may be calculated; and the final recognition result may be determined based on the weighted average. For example, the recognition result with the highest weighted average may be selected as the final recognition result. For example, the first score and the second score are score1 and score2, respectively, and weights W1 and W2 are set, respectively, where W1+W2=1, and the final score final_score=W1*score1+W2*score2. The value range of W1 may be (±0.9-0.99), and the value range of W2 may be (±0.01-0.1).

[0065] In the embodiment of the present application, by dynamically loading the domain model, not only can the unregistered words in the domain scenario be efficiently recognized, but also the specified words in the original model can be strongly recognized without affecting the recognition of common expressions in speech recognition. The model training of the entire process is at the minute level, and hot loading is used to merge it with the general domain speech recognition model, thereby improving the accuracy and efficiency of speech recognition.

[0066] like Figure 5 As shown, Figure 5 : is a flow chart of another speech recognition method provided in an embodiment of the present application. The steps in the embodiment of the present application at least include:

[0067] S501, extracting features of a speech signal stream and inputting them into a first model to obtain a first recognition result, and determining a first score of the first recognition result, wherein the first model is used to recognize vocabulary of common sayings.

[0068] Among them, the first model includes a first commonly used language model and a first re-scoring language model, the first commonly used language model is generated based on a bigram language model, and the first re-scoring language model is generated based on a five-gram language model; the speech signal stream can be recognized according to the first commonly used language model to obtain multiple first recognition results, and one first recognition result corresponds to one first score; then the multiple first recognition results are sorted according to the first re-scoring language model, and the first recognition result corresponding to the highest first score is selected.

[0069] like Figure 1 As shown, the first model can be a ROOT model, which obtains an acoustic model A and a language model G1 from audio and corpus in a general field, respectively, wherein the language model G1 is split into 2-gram G11 (a first commonly used language model) and 5-gram G12 (a first re-scored language model), and a decoding graph HCLG can be generated according to G11, and the acoustic model and the decoding graph HCLG are used to recognize the speech of a general statement, and then the recognition results of the acoustic model and the decoding graph HCLG are re-scored through G12.

[0070] For example, the corpus "I want to know in which hall Zhu Yuanzhang attends court" is segmented. I want / know / Zhu Yuanzhang / in / which hall / attends court is a 1-gram. I want / attend court / which hall to attend court / I want to know / Zhu Yuanzhang is / know Zhu Yuanzhang is a 2-gram. I want to know Zhu Yuanzhang is / I want to know in which hall Zhu Yuanzhang is / which hall Zhu Yuanzhang attends court is a 5-gram. The above examples only illustrate part of the vocabulary. Among them, G11 includes 1-gram and 2-gram, and G12 includes 1-gram, 2-gram, 3-gram, 4-gram and 5-gram.

[0071] S502, when the first model cannot recognize the language vocabulary in the speech signal stream, load a second model according to the domain type to which the language vocabulary belongs, input the language vocabulary into the second model to obtain a second recognition result, and determine a second score of the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement.

[0072] In a specific implementation, a special identifier corresponding to the domain type to which the language vocabulary belongs may be determined; and according to the special identifier, the second model is searched from a plurality of preset domain models and loaded.

[0073] For example, Figure 3 As shown, Figure 3 This is a state fallback diagram provided by an embodiment of the present application. After feature extraction of the signal stream, the ROOT model is entered for online decoding. If the ROOT model cannot recognize one or some domain vocabulary, it needs to fall back. Among them, $OOV represents a 1-gram state fallback identifier. During the decoding graph search, the $OOV identifier is included in the fallback state of the ROOT language model. If a 1-gram fallback state is searched, the pre-specified domain OOV-HCLG will be entered for a decoding graph search, and then the search results will be replaced (replace) through a weighted finite state transducer (Weighted Finite-State Transducer, WFST). Another example Figure 4 As shown, Figure 4 This is a replacement operation diagram provided in an embodiment of the present application, which replaces the edge of $OOV in the search results.

[0074] Among them, the second model includes a second domain language model and a second rescoring language model, and the second domain language model is generated according to a unigram language model. Multiple second recognition results can be obtained by recognizing the language vocabulary according to the second domain language model, and one second recognition result corresponds to one second score; the multiple second recognition results are sorted according to the second rescoring language model, and the second recognition result corresponding to the highest second score is selected.

[0075] As Figure 1 shown, the second model can be a domain model, and the domain model is divided into two parts: OOV-HCLG and rescore-HCLG. The language model of OOV-HCLG consists of the set of words ${oov} not covered by the ROOT model in the domain scenario and the set of words ${ <s> word< / s>} of the words word to be strengthened in the ROOT model, which is composed of 1-grams. For example, words such as Duanmen / Xianximen / Yongkangzuomen are all in the set of words ${oov} not covered by the ROOT model, which belong to the historical domain and are infrequently used words. Another example is that "dian" is easily recognized as "dian", so "dian" is a word to be strengthened.

[0076] Among them, the language model of rescore-HCLG is generated by using the above-mentioned word set ${oov} and constructing sentences with specified sentence patterns, and is used to rescore the recognition results of the language model of OOV-HCLG to strengthen the recognition of domain expressions. When decoding through the ROOT model (base) and the language model of OOV-HCLG, the 5-gram language model (big) of the ROOT model and the rescore-HCLG of the domain model will be used for rescoring synchronously respectively.

[0077] It should be noted that models in different domains can be integrated into a speech recognition system. For independent retrieval (query) requests in each domain, the domain model is hot-loaded in real time, that is, the already loaded domain model is not released, and a new domain model is loaded.

[0078] S503, determine the final score according to the first score and the second score.

[0079] In a specific implementation, a weighted average of the first score and the second score can be calculated; based on the weighted average, the weighted average is used as the final score. For example, the recognition result with the highest weighted average can be selected as the final recognition result. For example, if the first score and the second score are score1 and score2 respectively, weights W1 and W2 are set respectively, where W1 + W2 = 1, and the final score final_score = W1 * score1 + W2 * score2. Among them, the value range of W1 can be (±0.9 - 0.99), and the value range of W2 can be (±0.01 - 0.1).

[0080] S504, replace the recognition result of the to-be-strengthened word recognized by the first model with the recognition result of the to-be-strengthened word recognized by the second model to obtain the final recognition result.

[0081] In a specific implementation, the recognition result with the highest final score can be selected. For this recognition result, according to the identifier of the to-be-strengthened word <s> word< / s> for replacement to obtain the final recognition result. For example, if the recognition result includes "store", then "store" is replaced with "hall".

[0082] As Figure 6 shown Figure 6 is a schematic structural diagram of a voice recognition device provided by an embodiment of the present application. The device in the embodiment of the present application at least includes:

[0083] A processing module 601, configured to extract features of a voice signal stream and input them into a first model to obtain a first recognition result, and determine a first score of the first recognition result. The first model is used to recognize words in common expressions.

[0084] Among them, the first model includes a first common language model and a first re-scoring language model. The first common language model is generated according to a bigram language model, and the first re-scoring language model is generated according to a 5-gram language model; the voice signal stream can be recognized according to the first common language model to obtain a plurality of the first recognition results, and one of the first recognition results corresponds to one of the first scores; then, the plurality of first recognition results are sorted according to the first re-scoring language model, and the first recognition result corresponding to the highest first score is selected.

[0085] As Figure 1As shown, the first model can be a ROOT model, which obtains an acoustic model A and a language model G1 from audio and corpus in a general field, respectively, wherein the language model G1 is split into 2-gram G11 (a first commonly used language model) and 5-gram G12 (a first re-scored language model), and a decoding graph HCLG can be generated according to G11, and the acoustic model and the decoding graph HCLG are used to recognize the speech of a general statement, and then the recognition results of the acoustic model and the decoding graph HCLG are re-scored through G12.

[0086] For example, the corpus "I want to know in which hall Zhu Yuanzhang attends court" is segmented. I want / know / Zhu Yuanzhang / in / which hall / attends court is a 1-gram. I want / attend court / which hall to attend court / I want to know / Zhu Yuanzhang is / know Zhu Yuanzhang is a 2-gram. I want to know Zhu Yuanzhang is / I want to know in which hall Zhu Yuanzhang is / which hall Zhu Yuanzhang attends court is a 5-gram. The above examples only illustrate part of the vocabulary. Among them, G11 includes 1-gram and 2-gram, and G12 includes 1-gram, 2-gram, 3-gram, 4-gram and 5-gram.

[0087] The processing module 601 is also used to load a second model according to the domain type to which the language vocabulary belongs when the first model cannot recognize the language vocabulary in the speech signal stream, input the language vocabulary into the second model to obtain a second recognition result, and determine a second score of the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement.

[0088] In a specific implementation, a special identifier corresponding to the domain type to which the language vocabulary belongs may be determined; and according to the special identifier, the second model is searched from a plurality of preset domain models and loaded.

[0089] For example, Figure 3 As shown, Figure 3 This is a state fallback diagram provided by an embodiment of the present application. After feature extraction of the signal stream, the ROOT model is entered for online decoding. If the ROOT model cannot recognize one or some domain vocabulary, it needs to fall back. Among them, $OOV represents a 1-gram state fallback identifier. During the decoding graph search, the $OOV identifier is included in the fallback state of the ROOT language model. If a 1-gram fallback state is searched, the pre-specified domain OOV-HCLG will be entered for a decoding graph search, and then the search results will be replaced (replace) through a weighted finite state transducer (Weighted Finite-State Transducer, WFST). Another example Figure 4 As shown, Figure 4It is a schematic diagram of a replacement operation provided by an embodiment of the present application, which replaces the edge of $OOV in the search results.

[0090] Among them, the second model includes a second domain language model and a second rescoring language model, and the second domain language model is generated according to a unigram language model. Multiple second recognition results can be obtained by recognizing the language vocabulary according to the second domain language model, and one second recognition result corresponds to one second score; the multiple second recognition results are sorted according to the second rescoring language model, and the second recognition result corresponding to the highest second score is selected.

[0091] As Figure 1 shown, the second model can be a domain model, and the domain model is divided into two parts: OOV-HCLG and rescore-HCLG. The language model of OOV-HCLG consists of the set of words ${oov} not covered by the ROOT model in the domain scenario and the set of words ${ <s> word< / s>} of 1-gram of the words to be strengthened word in the ROOT model. For example, words such as Duanmen / Xianximen / Yongkangzuomen are all in the set of words ${oov} not covered by the ROOT model, which belong to the vocabulary in the historical field and are not commonly used words. Another example is that "dian" is easily recognized as "dian", so "dian" is a word to be strengthened.

[0092] Among them, the language model of rescore-HCLG is generated by using the above-mentioned word set ${oov} and constructing sentences with specified sentence patterns, and is used to rescore the recognition results of the language model of OOV-HCLG to strengthen the recognition of domain expressions. When decoding through the ROOT model (base) and the language model of OOV-HCLG, the 5-gram language model (big) of the ROOT model and the rescore-HCLG of the domain model will be used for rescoring synchronously.

[0093] It should be noted that models in different domains can be integrated into a speech recognition system. For independent retrieval (query) requests in each domain, the domain model is hot-loaded in real time, that is, the already loaded domain model is not released, and a new domain model is loaded.

[0094] A determination module 602, configured to determine a final recognition result according to the first score and the second score.

[0095] In a specific implementation, the weighted average of the first score and the second score can be calculated; according to the weighted average, the weighted average is used as the final score. For example, the recognition result with the highest weighted average can be selected as the final recognition result. For example, if the first score and the second score are score1 and score2 respectively, weights W1 and W2 are set respectively, where W1 + W2 = 1, and the final score final_score = W1 * score1 + W2 * score2. Among them, the value range of W1 can be (±0.9 - 0.99), and the value range of W2 can be (±0.01 - 0.1).

[0096] Optionally, the recognition result of the second model for the to-be-strengthened word is used to replace the recognition result of the first model for the to-be-strengthened word to obtain the final recognition result. Specifically, the recognition result with the highest final score can be selected, and for this recognition result, replacement is performed according to the identifier of the to-be-strengthened word <s> word< / s> to obtain the final recognition result. For example, if the recognition result includes "store", then "store" is replaced with "hall".

[0097] In the embodiments of the present application, by means of dynamically loading a domain model, not only can unregistered words in the domain scenario be efficiently recognized, but also specified words in the original model can be strongly recognized, and it does not affect the recognition of common expressions in speech recognition. The model training of the whole process is in minutes, and the hot loading method is used to merge with the general domain speech recognition model, improving the accuracy and efficiency of speech recognition.

[0098] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of a speech recognition device provided by the embodiments of the present application. As shown in the figure, the device may include: at least one processor 701, at least one communication interface 702, at least one memory 703, and at least one communication bus 704.

[0099] Among them, the processor 701 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in combination with the disclosure of the present application. The processor may also be a combination that implements a computing function, such as a combination including one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. The communication bus 704 may be a peripheral component interconnect standard PCI bus or an extended industry standard architecture EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 7Only one thick line is used to represent it, but it does not mean that there is only one bus or one type of bus. The communication bus 704 is used to realize the connection communication between these components. Among them, the communication interface 702 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 703 may include volatile memory, such as non-volatile dynamic random access memory (Nonvolatile Random Access Memory, NVRAM), phase change random access memory (PhaseChange RAM, PRAM), magnetoresistive random access memory (Magetoresistive RAM, MRAM), etc., and may also include non-volatile memory, such as at least one disk storage device, electronically erasable programmable read-only memory (ElectricallyErasable Programmable Read-Only Memory, EEPROM), flash memory devices, such as NOR flash memory or NAND flash memory, semiconductor devices, such as solid state disk (SolidState Disk, SSD), etc. The memory 703 may also be optionally at least one storage device located away from the aforementioned processor 701. A set of program codes is stored in the memory 703, and the processor 701 executes the program in the memory 703:

[0100] Extracting features of the speech signal stream and inputting them into a first model to obtain a first recognition result, and determining a first score of the first recognition result, wherein the first model is used to recognize vocabulary of common sayings;

[0101] When the first model cannot recognize the language vocabulary in the speech signal stream, loading a second model according to the domain type to which the language vocabulary belongs, inputting the language vocabulary into the second model to obtain a second recognition result, and determining a second score of the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement;

[0102] A final recognition result is determined according to the first score and the second score.

[0103] wherein the vocabulary of the first model includes special identifiers,

[0104] Optionally, the processor 701 is further configured to perform the following operation steps:

[0105] Determine a special identifier corresponding to the domain type to which the language vocabulary belongs;

[0106] According to the special identifier, the second model is searched for and loaded from a plurality of preset domain models.

[0107] The first model includes a first commonly used language model and a first re-scoring language model, the first commonly used language model is generated based on a bigram language model, and the first re-scoring language model is generated based on a pentagram language model;

[0108] Optionally, the processor 701 is further configured to perform the following operation steps:

[0109] Recognize the speech signal stream according to the first common language model to obtain a plurality of first recognition results, where each first recognition result corresponds to one first score;

[0110] The plurality of first recognition results are sorted according to the first re-scoring language model, and the first recognition result corresponding to the highest first score is selected.

[0111] The second model includes a second domain language model and a second re-scoring language model, and the second domain language model is generated according to a unigram language model;

[0112] Optionally, the processor 701 is further configured to perform the following operation steps:

[0113] Recognize the language vocabulary according to the second domain language model to obtain a plurality of second recognition results, each second recognition result corresponding to one second score;

[0114] The plurality of second recognition results are sorted according to the second scoring language model, and the second recognition result corresponding to the highest second score is selected.

[0115] Optionally, the processor 701 is further configured to perform the following operation steps:

[0116] calculating a weighted average of the first score and the second score;

[0117] The final recognition result is determined according to the weighted average value.

[0118] wherein the second domain language model includes the vocabulary to be strengthened of the first model;

[0119] Optionally, the processor 701 is further configured to perform the following operation steps:

[0120] The recognition result for the vocabulary to be strengthened identified by the second model is used to replace the recognition result for the vocabulary to be strengthened identified by the first model to obtain the final recognition result.

[0121] Furthermore, the processor may also cooperate with the memory and the communication interface to execute the operations performed by the speech recognition device in the above-mentioned application embodiment.

[0122] The embodiment of the present application also provides a chip system, which includes a processor for supporting a network device or a terminal device to implement the functions involved in any of the above embodiments, such as generating or processing the data and / or information involved in the above method. In one possible design, the chip system may also include a memory, which is used for program instructions and data necessary for the speech recognition device. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0123] The embodiment of the present application also provides a processor, which is coupled to a memory and is used to execute any method and function related to a speech recognition device in any of the above embodiments.

[0124] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute any method and function related to the speech recognition device in any of the above embodiments.

[0125] The embodiment of the present application also provides a device for executing any method and function related to the speech recognition device in any of the above embodiments.

[0126] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0127] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of this application in detail. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application shall be included in the protection scope of this application.

Claims

1. A speech recognition method, It is characterized in that The invention is applied to a speech recognition system, wherein the speech recognition system integrates a plurality of domain models, wherein the domain models are used to identify vocabulary of domain statements, wherein the speech recognition system comprises a first model and a second model, wherein the first model comprises a first common language model and a first re-scoring language model, wherein the second model comprises a second domain language model and a second re-scoring language model, wherein the second domain language model is composed of a word set not registered in the first model in a domain scenario and a set of vocabulary to be strengthened in the first model, wherein the second re-scoring language model is generated by a word set not registered in the first model in a domain scenario and a specified sentence pattern, and is used to re-score the recognition result of the second domain language model to strengthen the recognition of domain statements, wherein the method comprises: Extracting features of the speech signal stream and inputting them into a first model to obtain a first recognition result, and determining a first score of the first recognition result, wherein the first model is used to recognize vocabulary of common sayings, and the vocabulary of the first model includes special identifiers; When the first model cannot recognize the language vocabulary in the speech signal stream, determine the special identifier corresponding to the domain type to which the language vocabulary belongs, search and load the second model from the preset multiple domain models according to the special identifier, input the language vocabulary into the second model to obtain a second recognition result, and determine a second score of the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement; According to the first score and the second score, the recognition result for the vocabulary to be strengthened identified by the second model is replaced with the recognition result for the vocabulary to be strengthened identified by the first model to determine a final recognition result.

2. The method according to claim 1, It is characterized in that The first common language model is generated based on a bigram language model, and the first re-scoring language model is generated based on a pentagram language model; The extracting the features of the speech signal stream and inputting them into the first model to obtain a first recognition result, and determining a first score of the first recognition result comprises: Recognize the speech signal stream according to the first common language model to obtain a plurality of first recognition results, where each first recognition result corresponds to one first score; The plurality of first recognition results are sorted according to the first re-scoring language model, and the first recognition result corresponding to the highest first score is selected.

3. The method according to claim 1, It is characterized in that The second domain language model is generated based on a unigram language model; The step of inputting the language vocabulary into the second model to obtain a second recognition result, and determining a second score of the second recognition result comprises: Recognize the language vocabulary according to the second domain language model to obtain a plurality of second recognition results, each second recognition result corresponding to one second score; The plurality of second recognition results are sorted according to the second scoring language model, and the second recognition result corresponding to the highest second score is selected.

4. The method according to any one of claims 1 to 3, It is characterized in that The determining a final recognition result according to the first score and the second score comprises: calculating a weighted average of the first score and the second score; The final recognition result is determined according to the weighted average value.

5. A speech recognition device, It is characterized in that The invention is applied to a speech recognition system, wherein the speech recognition system integrates a plurality of domain models, wherein the domain models are used to identify vocabulary of domain statements, wherein the speech recognition system comprises a first model and a second model, wherein the first model comprises a first common language model and a first re-scoring language model, wherein the second model comprises a second domain language model and a second re-scoring language model, wherein the second domain language model is composed of a word set not registered in the first model in a domain scenario and a set of vocabulary to be strengthened in the first model, wherein the second re-scoring language model is generated by a word set not registered in the first model in a domain scenario and a specified sentence pattern, and is used to re-score the recognition result of the second domain language model to strengthen the recognition of domain statements, wherein the device comprises: a processing module, configured to extract features of a speech signal stream and input the features into a first model to obtain a first recognition result, and determine a first score of the first recognition result, wherein the first model is used to recognize vocabulary of common sayings, and the vocabulary of the first model includes special identifiers; The processing module is further configured to, when the first model cannot recognize the language vocabulary in the speech signal stream, determine a special identifier corresponding to the domain type to which the language vocabulary belongs, search for a second model from the preset multiple domain models and load it according to the special identifier, input the language vocabulary into the second model to obtain a second recognition result, and determine a second score for the second recognition result, wherein the second model is used to recognize the vocabulary of the domain statement; A determination module is used to replace the recognition result for the to-be-strengthened vocabulary identified by the first model with the recognition result for the to-be-strengthened vocabulary identified by the second model according to the first score and the second score, so as to determine a final recognition result.

6. The device as claimed in claim 5, It is characterized in that The determination module is further configured to calculate a weighted average of the first score and the second score; and determine the final recognition result according to the weighted average.

7. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 4.