A speech recognition method, device, equipment and storage medium

Abstract

The embodiment of the present invention discloses a speech recognition method, device, equipment and storage medium, the method comprising: determining the decoding output score of the speech to be recognized and the second candidate word corresponding to the speech to be recognized based on the first candidate word obtained by decoding the speech to be recognized; determining the decoding features of each of the second candidate words, and determining the decoding confidence of the speech to be recognized based on the decoding features of each of the second candidate words; determining the noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized based on the noise confidence of each of the speech frames; determining the comprehensive confidence of the speech to be recognized based on the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determining the speech recognition result based on the comprehensive confidence. The above technical solution improves the accuracy of speech recognition without increasing the cost.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of speech processing technology, and in particular to a speech recognition method, apparatus, device and storage medium. Background Art

[0002] Speech recognition has been widely used in smart customer service, smart home, car assistant and other fields. Speech recognition systems are often affected by noise interference from the environment or telephone channels, which can easily lead to speech recognition errors. For example, when the noise and speech segments do not coincide in time, speech recognition insertion errors will occur; when the speech segments are corrupted by noise, deletion or modification errors will occur. Speech recognition errors bring huge challenges to subsequent voice interactions.

[0003] In the prior art, the speech to be recognized can be processed by a front-end noise reduction module to reduce the influence of noise on the characteristics of the speech to be recognized, and then the processed speech to be recognized is recognized based on the speech recognition module to determine the speech recognition result.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following technical problems in the prior art:

[0005] The front-end noise reduction module and speech recognition module need to be adapted, which increases the speech recognition cost. Summary of the invention

[0006] The present invention provides a speech recognition method, device, equipment and storage medium to reduce the cost of speech recognition.

[0007] In a first aspect, an embodiment of the present invention provides a speech recognition method, comprising:

[0008] Determining, based on a first candidate word obtained by decoding the speech to be recognized, a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized;

[0009] Determining a decoding feature of each of the second candidate words, and determining a decoding confidence of the speech to be recognized according to the decoding feature of each of the second candidate words;

[0010] Determining the noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized according to the noise confidence of each speech frame;

[0011] The comprehensive confidence of the speech to be recognized is determined according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and the speech recognition result is determined according to the comprehensive confidence.

[0012] In a second aspect, an embodiment of the present invention further provides a speech recognition device, comprising:

[0013] A decoding module, configured to determine a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized based on a first candidate word obtained by decoding the speech to be recognized;

[0014] A decoding confidence determination module, used to determine a decoding feature of each of the second candidate words, and determine a decoding confidence of the speech to be recognized according to the decoding feature of each of the second candidate words;

[0015] A noise confidence determination module, used to determine the noise confidence of each speech frame contained in the speech to be recognized, and determine the noise confidence of the speech to be recognized according to the noise confidence of each speech frame;

[0016] The execution module is used to determine the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determine the speech recognition result according to the comprehensive confidence.

[0017] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a speech recognition method as described in any one of the first aspects is implemented.

[0018] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to perform the speech recognition method as described in any one of the first aspects.

[0019] The embodiments of the above invention have the following advantages or beneficial effects:

[0020] An embodiment of the present invention provides a speech recognition method, comprising: determining a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized based on a first candidate word obtained by decoding the speech to be recognized; determining decoding features of each of the second candidate words, and determining a decoding confidence of the speech to be recognized based on the decoding features of each of the second candidate words; determining a noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized based on the noise confidence of each of the speech frames; determining a comprehensive confidence of the speech to be recognized based on the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determining a speech recognition result based on the comprehensive confidence. The above technical solution can first determine the decoding output score of the speech to be recognized according to the first candidate word obtained by decoding the speech to be recognized, provide a data basis for determining the comprehensive confidence, determine the second candidate word corresponding to the speech to be recognized according to the first candidate word, and after determining the decoding features of each second candidate word, determine a more accurate decoding confidence of the speech to be recognized according to each decoding feature, and then determine a more accurate frame-level noise confidence of the speech to be recognized according to the noise confidence of each speech frame contained in the speech to be recognized, and further determine the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determine a more accurate comprehensive confidence of the speech to be recognized by combining the segment-level decoding confidence of the speech to be recognized, the frame-level noise confidence and the above-determined decoding output score. The above process does not need to re-optimize or retrain the speech recognition model used for decoding the speech to be recognized, and obtains a more accurate comprehensive confidence of the speech to be recognized. Therefore, the determined speech recognition result is also more accurate. The accuracy of speech recognition is improved without increasing the cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a speech recognition module provided by an embodiment of the present invention;

[0022] Figure 2 A flow chart of a speech recognition method provided by an embodiment of the present invention;

[0023] Figure 3 A first word graph including a first candidate word obtained by decoding a speech to be recognized in a speech recognition method provided by an embodiment of the present invention;

[0024] Figure 4 A flowchart of another speech recognition method provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of a second preset network model in another speech recognition method provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a speech recognition system provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a speech recognition device provided by an embodiment of the present invention;

[0028] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0030] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0031] Figure 1 A schematic diagram of a speech recognition module provided by an embodiment of the present invention, such as Figure 1 As shown, the speech recognition module 100 may include a language model 110 and an acoustic model 120. The speech recognition module 100 may use a decoding algorithm to obtain an optimal sequence through Viterbi search during the recognition process, and generate a decoding output corresponding to the speech, that is, a word graph corresponding to the speech. Since noise infects speech, speech recognition errors are easily caused. Therefore, an embodiment of the present invention proposes a speech recognition method, which improves the accuracy of speech recognition and reduces the error rate of speech recognition without increasing costs.

[0032] The following will be combined Figure 1 The speech recognition module and the embodiments provide a detailed description of the speech recognition method proposed in the embodiments of the present invention.

[0033] Figure 2The flowchart of a speech recognition method provided by an embodiment of the present invention is applicable to situations where the accuracy of speech recognition needs to be improved without increasing the cost. The method can be executed by a speech recognition device, which can be implemented by software and / or hardware. Figure 2 The method specifically comprises the following steps:

[0034] Step 210: Determine, based on the first candidate word obtained by decoding the speech to be recognized, a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized.

[0035] like Figure 1 The speech recognition module including the language model and the acoustic model shown in FIG. 1 can be used to decode the speech to be recognized and generate the first candidate word corresponding to the speech. Figure 1 The speech recognition module shown in the figure enables the speech recognition module to decode the speech to be recognized and obtain the first candidate word corresponding to the speech to be recognized. Due to the interference of noise, the error rate of the first candidate word obtained by decoding the speech to be recognized by the speech recognition module is high, and it is necessary to further determine whether the speech to be recognized contains speech.

[0036] The first candidate word corresponding to the speech to be recognized can be represented based on the first word graph. The first word graph is a compressed representation of the first candidate words and time information in the process of decoding the speech to be recognized. The node represents the state, and the value in the node bracket represents the time corresponding to the state. There are different candidate paths from the initial state to the terminal state in the first word graph. The value on the candidate path represents the score of the first candidate word, that is, the posterior probability of the first candidate word. The score of each candidate path and the time information of each first candidate word in each candidate path can be determined from the first word graph. Figure 3 A first word graph including a first candidate word is obtained by decoding a speech to be recognized in a speech recognition method provided by an embodiment of the present invention, such as Figure 3 As shown, for example, from the initial state 0 to the terminal state 4, "Beijing", "background", "mobilization", "sports", "Olympic Games" and "meeting" are the first candidate words, the posterior probability of "Beijing" is 0.5, the posterior probability of "background" is 0.5, the posterior probability of "mobilization" is 0.5, the posterior probability of "sports" is 0.4, the posterior probability of "Olympic Games" is 0.2, and the posterior probability of "meeting" is 0.4. The two endpoint values ​​of the edge where "Winter Olympics" are located are 6 and 20 respectively, indicating that the speech content corresponding to the time period of 6s-20s is "Winter Olympics".

[0037] Specifically, after the first candidate word is obtained by decoding the speech to be recognized based on the speech recognition module, on the one hand, the decoding output score of each first candidate word can be determined first, and the decoding output score of each first candidate word can be sorted. Secondly, the three largest decoding output scores can be normalized, and then the processing result can be determined as the decoding output score of the speech to be recognized. On the other hand, the second candidate word corresponding to the speech to be recognized can also be determined. Specifically, a secondary decoding based on the minimum Bayesian risk and with the edit distance as the criterion can be performed on the first word graph to obtain the second candidate word corresponding to the speech to be recognized.

[0038] In an embodiment of the present invention, after the speech to be recognized is decoded based on the speech recognition module to obtain the first candidate word, the decoding output score of the speech to be recognized can be determined according to the first candidate word, and the first word graph containing the first candidate words can be decoded again to obtain the second candidate word. The second candidate word can be used to determine the decoding confidence of the speech to be recognized, providing a data basis for determining the decoding confidence of the speech to be recognized.

[0039] Step 220: Determine the decoding features of each of the second candidate words, and determine the decoding confidence of the speech to be recognized based on the decoding features of each of the second candidate words.

[0040] The decoding features include the confidence score, word category, probability distribution, word length and word graph depth of the second candidate word.

[0041] Specifically, after determining the posterior probability of each second candidate word, the posterior probability of each second candidate word is normalized to obtain the confidence score of each second candidate word, and the confidence score of the second candidate word is the one-dimensional feature of the second candidate word. After determining the field of the speech to be recognized, determine the classification of words in the field, and the number of classifications is N+1, and map each second candidate word to any N+1 category to represent the word category of the second candidate word based on the N+1-dimensional feature, and the word category of the second candidate word is the N+1-dimensional feature of the second candidate word. According to the number of occurrences of the second candidate word in all the second candidate words corresponding to the speech to be recognized and the total number of second candidate words corresponding to the speech to be recognized, determine the probability distribution of the second candidate word, and the probability distribution of the second candidate word is the one-dimensional feature of the second candidate word. The word length of the second candidate word can be determined according to the number of phonemes contained in the second candidate word, and the word length of the second candidate word is the one-dimensional feature of the second candidate word. In the second word graph including the second candidate word, the word graph depth of the second candidate word is determined according to the number of outgoing edges of all nodes in the time period corresponding to the second candidate word and the length of the time period. The word graph depth of the second candidate word is a one-dimensional feature of the second candidate word.

[0042] Therefore, it can be determined that the decoding features of the second candidate words are N+5-dimensional features, and the N+5-dimensional decoding features corresponding to each second candidate word are respectively input into the pre-trained decoding confidence model to obtain the decoding confidence of each second candidate word, and then the arithmetic mean of the decoding confidence of each second candidate word can be determined, and the arithmetic mean is determined as the decoding confidence of the speech to be recognized.

[0043] The decoding confidence of the speech to be recognized reflects the reliability of the speech recognition result. Generally speaking, the decoding confidence of the speech to be recognized ranges from 0 to 1. The closer it is to 1, the more reliable the speech recognition result is.

[0044] In an embodiment of the present invention, after a second word graph containing a second candidate word is obtained by performing secondary decoding based on the first word graph containing the first candidate word, the decoding features of each second candidate word can be determined according to the second word graph, and the decoding confidence of the second candidate word can be determined according to the decoding features, and then the decoding confidence of the speech to be recognized can be determined according to the decoding confidence of each second candidate word. The decoding confidence of the speech to be recognized can indicate the reliability of the speech recognition result, and provide a data basis for determining the speech recognition result.

[0045] Step 230: Determine the noise confidence of each speech frame included in the speech to be recognized, and determine the noise confidence of the speech to be recognized according to the noise confidence of each speech frame.

[0046] Before determining the noise confidence of the speech to be recognized, the speech to be recognized may be divided into frames to obtain speech frames, wherein the frame length of each speech frame is 25 milliseconds and the frame shift is 10 milliseconds.

[0047] Specifically, first, the Mel-scale Frequency Cepstral Coefficients (MFCC) of each speech frame can be extracted, and the MFCC of the speech frame can be input into a pre-trained noise confidence model to obtain the probability p(t) that the speech frame contains speech, and the probability that the speech frame does not contain speech is 1-p(t), and then the noise confidence of the speech frame is determined to be 1-p(t). After determining the noise confidence of each speech frame contained in the speech to be recognized, the maximum noise confidence, the minimum noise confidence, the mean noise confidence and the variance of the noise confidence are determined, and the maximum noise confidence and the minimum noise confidence of the noise confidence of each speech frame contained in the speech to be recognized, as well as the mean noise confidence and the variance of the noise confidence are determined as the noise confidence of the speech to be recognized.

[0048] In the embodiment of the present invention, the noise confidence at the segment level of the speech to be recognized may be determined based on the noise confidence at the frame level of each speech frame included in the speech to be recognized.

[0049] Step 240: Determine the comprehensive confidence of the speech to be recognized based on the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determine the speech recognition result based on the comprehensive confidence.

[0050] Specifically, the decoding confidence, noise confidence and decoding output score of the speech to be recognized are input into the pre-trained recognition model as the recognition features of the speech to be recognized, and the output result obtained is the comprehensive confidence of the speech to be recognized. The comprehensive confidence integrates the decoding confidence at the segment level of the speech to be recognized, the noise confidence at the segment level of the speech to be recognized determined according to the noise confidence at the frame level, and the decoding output score of the speech to be recognized. According to the comprehensive confidence, it can be determined whether it is a valid recognition result and whether the speech recognition result contains speech.

[0051] In the embodiment of the present invention, the probability that the speech to be recognized contains speech and the probability that the speech does not contain speech can be determined according to the comprehensive confidence, and then the speech recognition result is determined according to the probability that the speech to be recognized contains speech. The problem of speech recognition insertion errors caused by noise is effectively solved, and there is no need to re-optimize and retrain the speech recognition module, so that different speech recognition modules can be adapted.

[0052] A speech recognition method provided by an embodiment of the present invention includes: determining a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized based on a first candidate word obtained by decoding the speech to be recognized; determining decoding features of each of the second candidate words, and determining a decoding confidence of the speech to be recognized based on the decoding features of each of the second candidate words; determining a noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized based on the noise confidence of each of the speech frames; determining a comprehensive confidence of the speech to be recognized based on the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determining a speech recognition result based on the comprehensive confidence. The above technical solution can first determine the decoding output score of the speech to be recognized according to the first candidate word obtained by decoding the speech to be recognized, provide a data basis for determining the comprehensive confidence, determine the second candidate word corresponding to the speech to be recognized according to the first candidate word, and after determining the decoding features of each second candidate word, determine a more accurate decoding confidence of the speech to be recognized according to each decoding feature, and then determine a more accurate frame-level noise confidence of the speech to be recognized according to the noise confidence of each speech frame contained in the speech to be recognized, and further determine the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determine a more accurate comprehensive confidence of the speech to be recognized by combining the segment-level decoding confidence of the speech to be recognized, the frame-level noise confidence and the above-determined decoding output score. The above process does not need to re-optimize or retrain the speech recognition model used for decoding the speech to be recognized, and obtains a more accurate comprehensive confidence of the speech to be recognized. Therefore, the determined speech recognition result is also more accurate. The accuracy of speech recognition is improved without increasing the cost.

[0053] Figure 4 This is a flowchart of another speech recognition method provided by an embodiment of the present invention. The embodiment of the present invention can be applied to situations where the accuracy of speech recognition needs to be improved without increasing the cost. The explanations of the terms in the embodiment of the present invention that are the same or corresponding to the above embodiments are not repeated here. Figure 4 , the speech recognition method provided by the embodiment of the present invention includes:

[0054] Step 410: Determine a decoding output score of the speech to be recognized based on the first candidate word obtained by decoding the speech to be recognized.

[0055] In one implementation, step 410 may specifically include:

[0056] Based on a speech recognition module composed of a language model and an acoustic model, the speech to be recognized is decoded once to obtain the first candidate words; a language score and an acoustic score of each of the first candidate words are determined, and the decoding output score of the speech to be recognized is determined according to the language score and the acoustic score of each of the first candidate words.

[0057] Specifically, the speech recognition module including the language model and the acoustic model can be used to decode the speech to be recognized once, and the first candidate words corresponding to the speech to be recognized can be obtained by decoding once, and then the language score and acoustic score of each first candidate word can be determined, and after the language score and the acoustic score are integrated, the decoding output score of each first candidate word can be obtained. After determining the decoding output score of each first candidate word, the decoding output scores of each first candidate word can be sorted, and the three largest decoding output scores can be normalized, and the processing result is determined as the decoding output score of the speech to be recognized.

[0058] In an embodiment of the present invention, after the speech to be recognized is decoded based on the speech recognition module to obtain the first candidate word, the decoding output score of the speech to be recognized can be determined according to the first candidate word, providing a data basis for determining the comprehensive confidence of the speech to be recognized.

[0059] Step 420: Determine a second candidate word corresponding to the speech to be recognized based on the first candidate word obtained by decoding the speech to be recognized.

[0060] In one implementation, step 420 may specifically include:

[0061] Based on a speech recognition module composed of a language model and an acoustic model, the speech to be recognized is decoded once to obtain a first word graph including the first candidate words; on the first word graph, a second decoding is performed based on the minimum Bayesian risk with the edit distance as a criterion to obtain the second candidate words corresponding to the speech to be recognized and the posterior probability of each second candidate word.

[0062] Specifically, after the speech recognition module including the language model and the acoustic model decodes the speech to be recognized once to obtain the first candidate word corresponding to the speech to be recognized, it can also determine the first word graph based on the first candidate word, and perform a secondary decoding on the first word graph based on the minimum Bayesian risk with the edit distance as the criterion to obtain the second candidate word corresponding to the speech to be recognized.

[0063] The process of secondary decoding on the first word graph is as follows: 1) select any candidate path from the initial state to the terminal state from the first word graph; 2) take the candidate path as the benchmark, calculate the edit distance between the candidate path and the entire first word graph (the edit distance can be the minimum number of insertions, deletions, and substitutions to convert one text into another text), and obtain the posterior probabilities of all first candidate words in the time period corresponding to each first candidate word in the candidate path through the edit distance; 3) select the word with the highest probability at each moment to obtain a new word sequence, i.e., the second candidate word; 4) if the second candidate word is different from the first candidate word corresponding to the candidate path in 2), return to execute 2), otherwise, determine that the secondary decoding is over, and determine the word sequence containing each second candidate word as the secondary decoding result. In addition, each time period in the word sequence containing each second candidate word has the posterior probability of all second candidate words in the time period.

[0064] In the embodiment of the present invention, after the speech to be recognized is decoded based on the speech recognition module to obtain the first candidate word, the second candidate word of the speech to be recognized can be determined based on the first candidate word, providing a data basis for determining the decoding confidence of the speech to be recognized.

[0065] Step 430: Determine the decoding features of each of the second candidate words.

[0066] The decoding features include the confidence score, word category, probability distribution, word length and word graph depth of the second candidate word.

[0067] In one implementation, step 430 may specifically include:

[0068] The posterior probability of each second candidate word is normalized to obtain a confidence score of each second candidate word; the word category of each second candidate word is determined according to the category information of each second candidate word; the probability distribution of each second candidate word is determined according to the number of occurrences of each second candidate word in all second candidate words corresponding to the speech to be recognized; the word length of each second candidate word is determined according to the number of phonemes contained in each second candidate word; in the second word graph obtained by secondary decoding of the first candidate word, the word graph depth of each second candidate word is determined according to the number of outgoing edges of all nodes in the time period corresponding to each second candidate word and the length of the time period.

[0069] Specifically, when the second candidate word is obtained by performing secondary decoding on the first word graph, a second word graph containing the second candidate word can be generated. The second word graph also contains the posterior probability of each second candidate word. Therefore, after normalizing the posterior probability of each second candidate word, the confidence score of each second candidate word is obtained. The confidence score of the second candidate word can be determined as a one-dimensional feature of the second candidate word. The word category can indicate the category information of the second candidate word. First, the field of the speech to be recognized and the classification of words in the field can be determined, and the categories of words in the field can be sorted according to the word frequency of each category of words. The words of the first N categories are each treated as a separate category, with a total of N categories, and the words of other categories are unified as a special category. Therefore, the words in the field can be divided into N+1 categories, and each second candidate word can be mapped to any N+1 category. Therefore, the word category of the second candidate word can be represented based on the N+1-dimensional feature. For example, when the words in the field of speech to be recognized are divided into N+1=3+1=4 categories, the word category of the second candidate word can be determined as: (1,0,0,0), (0,1,0,0), (0,0,1,0) or (0,0,0,1). The probability distribution can indicate the number of occurrences of the second candidate word in all the second candidate words corresponding to the speech to be recognized. Therefore, the probability distribution of the second candidate word, that is, the unigram probability of the second candidate word, can be determined according to the number of occurrences of the second candidate word in all the second candidate words corresponding to the speech to be recognized and the total number of second candidate words corresponding to the speech to be recognized. The probability distribution of the second candidate word can be determined as the one-dimensional feature of the second candidate word. The word length can indicate the number of phonemes contained in the second candidate word. Therefore, the word length of the second candidate word can be determined according to the number of phonemes contained in the second candidate word. The word length of the second candidate word can be determined as the one-dimensional feature of the second candidate word. The word graph depth can indicate the average word graph depth of the time period corresponding to the second candidate word. Therefore, in the second word graph containing the second candidate word, the word graph depth of the second candidate word, that is, the average lattice depth, can be determined according to the number of outgoing edges of all nodes in the time period corresponding to the second candidate word and the length of the time period. The word graph depth of the second candidate word can be determined as a one-dimensional feature of the second candidate word.

[0070] Therefore, it can be determined that the decoding feature of the second candidate word is an N+5-dimensional feature. As mentioned above, when N+1=3+1=4, it can be determined that the decoding feature of the second candidate word is an eight-dimensional feature.

[0071] In an embodiment of the present invention, the N+5-dimensional decoding features of each second candidate word obtained by secondary decoding can be determined, and the decoding features of the second candidate words are used to determine the decoding confidence of the speech to be recognized, providing a data basis for determining the decoding confidence of the speech to be recognized.

[0072] Step 440: Determine the decoding confidence of the speech to be recognized based on the decoding features of each of the second candidate words.

[0073] In one implementation, step 440 may specifically include:

[0074] The decoding features of each of the second candidate words are respectively input into a pre-trained decoding confidence model to obtain the decoding confidence of each of the second candidate words; and the decoding confidence of the speech to be recognized is determined according to the decoding confidence of each of the second candidate words.

[0075] Before respectively inputting the decoding features of each of the second candidate words into the pre-trained decoding confidence model, the method further includes:

[0076] A first preset network model is constructed based on a deep neural network and a cross entropy function, and the N+5-dimensional decoding features of the second candidate word corresponding to the speech data containing noise and valid speech after annotation and the annotation information of the speech data are used as the first training data. The first preset network model is trained and a first loss function is calculated. The network is optimized based on the back propagation algorithm until the first loss function converges to obtain a decoding confidence model.

[0077] The first loss function may be a cross entropy function.

[0078] Specifically, after constructing the first preset network model based on the deep neural network and the cross entropy function, the speech data containing noise and valid speech are annotated, the noise is annotated as 0, the valid speech is annotated as 1, and the annotated speech data containing noise and valid speech is determined as the first training speech set. After decoding each training speech contained in the first training speech set based on the speech recognition module to obtain the first training candidate word corresponding to the training speech, it is further decoded twice to obtain the second training candidate word corresponding to the training speech, and the N+5-dimensional decoding features of each second training candidate word are determined based on the aforementioned step 430. The N+5-dimensional decoding features of each second candidate word corresponding to each training speech contained in the training speech set and the annotation information of the training speech are used as training data to perform network training on the first preset network model, and the cross entropy function is calculated according to the sigmoid activation function value representing the confidence score of the second candidate word output by the first preset network model and the annotation information of the training speech; the network is optimized based on the back propagation algorithm until the cross entropy function converges to obtain a decoding confidence model.

[0079] Then, the decoding features of each second candidate word can be respectively input into the decoding confidence model, and the output result obtained is the decoding confidence of each second candidate word. The arithmetic mean of the decoding confidence of each second candidate word corresponding to the speech to be recognized is determined, and the arithmetic mean is determined as the decoding confidence of the speech to be recognized.

[0080] In an embodiment of the present invention, the decoding confidence of each second candidate word corresponding to the speech to be recognized can be determined based on the decoding confidence model, and then the decoding confidence of the speech to be recognized can be determined according to the decoding confidence of each second candidate word. The decoding confidence of the speech to be recognized can indicate the reliability of the speech recognition result and provide a data basis for determining the speech recognition result.

[0081] Step 450: Determine the noise confidence of each speech frame included in the speech to be recognized.

[0082] In one implementation, step 450 may specifically include:

[0083] The speech to be recognized is divided into frames to obtain speech frames contained in the speech to be recognized; the Mel-frequency cepstral coefficients of each speech frame are determined, and the Mel-frequency cepstral coefficients of each speech frame are respectively input into a pre-trained noise confidence model to obtain the noise confidence of each speech frame.

[0084] Before the Mel-frequency cepstral coefficients of each speech frame are respectively input into the pre-trained noise confidence model, the method further includes:

[0085] A second preset network model is constructed based on a gated recurrent unit (GRU), and the Mel-cepstral coefficients of the frame training speech corresponding to each training speech in a second training speech set composed of training speech containing pure noise and pure speech and the annotation information of each frame training speech are used as second training data. Network training is performed on the second preset network model, and a second loss function is calculated. The weights of the second preset network model are iterated based on stochastic gradient descent until the second loss function converges to obtain a noise confidence model.

[0086] The second loss function may also be a cross entropy function.

[0087] Figure 5 A schematic diagram of a second preset network model in another speech recognition method provided by an embodiment of the present invention, such as Figure 5 As shown, the second preset network model includes a first fully connected layer (FC), a first GRU, a second GRU, a third GRU and a second FC.

[0088] Specifically, after constructing the second preset network model based on GRU, pure noise and pure speech are collected, and pure noise is randomly added to the pure speech at a preset signal-to-noise ratio to obtain training speech. A preset number of training speech is determined as the second training speech set, and each training speech contained in the second training speech set is framed with a frame length of 25 milliseconds and a frame shift of 10 milliseconds to obtain a frame training speech corresponding to the training speech. For each frame training speech, when the phoneme is non-silent, it is marked as 1, otherwise, it is marked as 0. Then, the Mel cepstral coefficients of the frame training speech corresponding to each training speech in the second training speech set and the annotation information of each frame training speech are used as the second training data to perform network training on the second preset network model. Specifically, the Mel cepstral coefficients of L frames of frame training speech can be used as a training sequence and input into the second preset network model, and the output result corresponding to each frame training speech is a vector with a dimension of 2, in which one dimension represents the probability that the current frame contains speech, and the other dimension represents the probability that the current frame does not contain speech. The annotation information of L frames of training speech is used as the target sequence, the second preset network model is trained, and the cross entropy function is calculated; the weights of the second preset network model are iterated based on stochastic gradient descent until the second loss function converges to obtain a noise confidence model.

[0089] Furthermore, the speech to be recognized can be divided into frames with a frame length of 25 milliseconds and a frame shift of 10 milliseconds to obtain the speech frames contained in the speech to be recognized, determine the Mel-cepstral coefficients of each speech frame, and input the Mel-cepstral coefficients of the speech frame into the noise confidence model. The output result is the probability p(t) that the speech frame contains speech, and the probability that it does not contain speech is 1-p(t), and then the noise confidence of the speech frame is determined to be 1-p(t).

[0090] In an embodiment of the present invention, the noise confidence of each speech frame contained in the speech to be recognized can be determined based on the noise confidence model, and the noise confidence of each speech frame is used to determine the noise confidence of the speech to be recognized. The noise confidence of the speech to be recognized can be further used to determine the comprehensive confidence of the speech to be recognized, thereby providing a data basis for determining the speech recognition result.

[0091] Step 460: Determine the noise confidence of the speech to be recognized according to the noise confidence of each speech frame.

[0092] In one implementation, step 460 may specifically include:

[0093] The noise confidence of the speech to be recognized is determined according to the maximum noise confidence, the minimum noise confidence, the noise confidence mean and the noise confidence variance among the noise confidences of the speech frames contained in the speech to be recognized.

[0094] Specifically, after determining the noise confidence of each speech frame contained in the speech to be recognized, the noise confidence of each frame of speech can be sorted, the mean and variance can be calculated, and the maximum noise confidence, minimum noise confidence, noise confidence mean and noise confidence variance among the noise confidence of each speech frame contained in the speech to be recognized are determined as the noise confidence of the speech to be recognized.

[0095] In the embodiment of the present invention, the noise confidence at the segment level of the speech to be recognized may be determined based on the noise confidence at the frame level of each speech frame included in the speech to be recognized.

[0096] Step 470: Determine the comprehensive confidence of the speech to be recognized based on the decoding confidence, noise confidence and decoding output score of the speech to be recognized.

[0097] In one implementation, step 470 may specifically include:

[0098] The decoding confidence, noise confidence and decoding output score of the speech to be recognized are input into a pre-trained speech recognition model to obtain a comprehensive confidence of the speech to be recognized.

[0099] Before inputting the decoding confidence, noise confidence and decoding output score of the speech to be recognized into the pre-trained speech recognition model, the method further includes:

[0100] A third preset network model is constructed based on a logistic regressor, and the decoding confidence, noise confidence, decoding output score corresponding to each training speech in a third training speech set constructed from speech containing noise and the annotation information of each training speech are used as third training data. Network training is performed on the third preset network model, and a third loss function is calculated. Network optimization is performed based on a back propagation algorithm until the third loss function converges to obtain a speech recognition model.

[0101] Then, the decoding confidence, noise confidence and decoding output score of the speech to be recognized can be input into the speech recognition model, and the output result obtained is the comprehensive confidence of the speech to be recognized.

[0102] In the embodiment of the present invention, the segment-level decoding confidence, frame-level noise confidence and decoding output score input of the speech to be recognized are integrated to determine the comprehensive confidence of the speech to be recognized.

[0103] Step 480: Determine the speech recognition result according to the comprehensive confidence.

[0104] The comprehensive confidence includes the probability that the speech to be recognized contains speech and the probability that the speech to be recognized does not contain speech.

[0105] In one implementation, step 480 may specifically include:

[0106] If the probability that the speech to be recognized contains speech is greater than or equal to a first preset threshold, the speech recognition result is determined to be that the speech to be recognized contains speech; if the probability that the speech to be recognized contains speech is greater than or equal to a second preset threshold, and is less than the first preset threshold, the speech recognition result is determined to be that the speech to be recognized does not contain speech; if the probability that the speech to be recognized contains speech is less than the second preset threshold, it is determined that the speech recognition is an error, or the speech to be recognized is optimized to obtain optimized speech, and speech recognition is performed again based on the optimized speech.

[0107] The first preset threshold is greater than the second preset threshold, and both the first preset threshold and the second preset threshold are less than 1.

[0108] Specifically, after the comprehensive confidence of the speech to be recognized is determined based on the speech recognition model, the probability that the speech to be recognized contains speech can be compared with the second preset threshold.

[0109] On the one hand, if the probability that the speech to be recognized contains speech is greater than or equal to the second preset threshold, the speech recognition result can be determined based on the probability that the speech to be recognized contains speech, that is, the decoding result of the speech recognition module can be used. Then, the probability that the speech to be recognized contains speech and the first preset threshold are compared. If the probability that the speech to be recognized contains speech is greater than or equal to the first preset threshold, the speech recognition result is determined to be that the speech to be recognized contains speech; if the probability that the speech to be recognized contains speech is less than the first preset threshold, the speech recognition result is determined to be that the speech to be recognized does not contain speech.

[0110] On the other hand, if the probability that the speech to be recognized contains speech is less than the second preset threshold, the speech recognition result cannot be determined based on the probability that the speech to be recognized contains speech, and the decoding result of the speech to be recognized by the speech recognition module may not be used, and then the speech recognition error may be determined. Alternatively, the speech to be recognized may be optimized to obtain optimized speech, and speech recognition may be performed again based on the optimized speech.

[0111] In one implementation, optimizing the speech to be recognized to obtain optimized speech includes:

[0112] The speech frames whose noise confidence in the speech to be recognized is greater than a preset confidence are set to be silent to obtain the optimized speech.

[0113] Specifically, the noise confidence of each speech frame contained in the speech to be recognized is compared with the preset confidence. If the noise confidence of any speech frame is greater than the preset confidence, the noise confidence of the speech frame is set to 0, so as to optimize the speech to be recognized and obtain the optimized speech. The speech frames whose noise confidence of each speech frame contained in the speech to be recognized is greater than the preset confidence are set to silence, so as to achieve noise reduction of the speech to be recognized, and then continue to perform speech recognition based on the optimized speech obtained by noise reduction, so as to improve the accuracy of speech recognition.

[0114] In an embodiment of the present invention, it can be determined whether the speech to be recognized contains speech or not based on the comprehensive confidence level, and it is also possible not to use the decoding result of the speech to be recognized by the speech recognition module to further determine the speech recognition error, or, after the speech to be recognized is denoised to optimize the speech to be recognized to obtain the optimized speech, the optimized speech is further decoded based on the speech recognition module to obtain the speech recognition result.

[0115] The speech recognition method provided by the embodiment of the present invention includes: determining the decoding output score of the speech to be recognized based on the first candidate word obtained by decoding the speech to be recognized; determining the second candidate word corresponding to the speech to be recognized based on the first candidate word obtained by decoding the speech to be recognized; determining the decoding features of each of the second candidate words, and determining the decoding confidence of the speech to be recognized based on the decoding features of each of the second candidate words; determining the noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized based on the noise confidence of each of the speech frames; determining the comprehensive confidence of the speech to be recognized based on the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determining the speech recognition result based on the comprehensive confidence. The above technical solution can first determine the decoding output score of the speech to be recognized according to the first candidate word obtained by decoding the speech to be recognized, provide a data basis for determining the comprehensive confidence, determine the second candidate word corresponding to the speech to be recognized according to the first candidate word, and after determining the decoding features of each second candidate word, determine a more accurate decoding confidence of the speech to be recognized according to each decoding feature, and then determine a more accurate frame-level noise confidence of the speech to be recognized according to the noise confidence of each speech frame contained in the speech to be recognized, and further determine the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determine a more accurate comprehensive confidence of the speech to be recognized by combining the segment-level decoding confidence of the speech to be recognized, the frame-level noise confidence and the above-determined decoding output score. The above process does not need to re-optimize or retrain the speech recognition model used for decoding the speech to be recognized, and obtains a more accurate comprehensive confidence of the speech to be recognized. Therefore, the determined speech recognition result is also more accurate. The accuracy of speech recognition is improved without increasing the cost.

[0116] In addition, after determining not to use the speech recognition module for the decoding result of the speech to be recognized based on the comprehensive confidence, a speech recognition error can be determined, or the speech to be recognized can be denoised to optimize the speech to be recognized to obtain the optimized speech, and then the optimized speech can be decoded based on the speech recognition module to obtain the speech recognition result.

[0117] Figure 6 A schematic diagram of a speech recognition system provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the speech recognition system may include a speech recognition module 100, a decoding confidence module 200, a noise confidence module 300, a result determination module 400 and a processing module 500, wherein the speech recognition module 100 is used to decode the speech to be recognized once to determine the first candidate word of the speech to be recognized and the first word graph containing the first candidate word; the decoding confidence module 200 is used to determine the decoding output score of the speech to be recognized and the second candidate word corresponding to the speech to be recognized according to the first candidate word, and after determining the decoding features of each second candidate word, determine the decoding confidence of the speech to be recognized according to the decoding features of each second candidate word; the noise confidence module 300 is used to determine the noise confidence of each speech frame contained in the speech to be recognized, and determine the speech to be recognized according to the noise confidence of each speech frame. The noise confidence of the identified speech; the result determination module 400 is used to determine the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized; the processing module 500 is used to determine the speech recognition result according to the comprehensive confidence, specifically for determining that the speech recognition result is that the speech to be recognized contains speech when the probability that the speech to be recognized contains speech is greater than or equal to a first preset threshold; when the probability that the speech to be recognized contains speech is greater than or equal to a second preset threshold and is less than the first preset threshold, determining that the speech recognition result is that the speech to be recognized does not contain speech; when the probability that the speech to be recognized contains speech is less than the second preset threshold, determining that the speech recognition is an error, or optimizing the speech to be recognized to obtain an optimized speech, and re-performing speech recognition based on the optimized speech.

[0118] The speech recognition system provided in the embodiment of the present invention can execute the speech recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the speech recognition method.

[0119] Figure 7 This is a schematic diagram of the structure of a speech recognition device provided by an embodiment of the present invention. The device and the speech recognition methods of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiments of the speech recognition device can refer to the embodiments of the above speech recognition methods.

[0120] The specific structure of the speech recognition device is as follows: Figure 7As shown, including:

[0121] A decoding module 710, configured to determine a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized according to the first candidate word obtained by decoding the speech to be recognized;

[0122] A decoding confidence determination module 720, configured to determine a decoding feature of each of the second candidate words, and determine a decoding confidence of the speech to be recognized according to the decoding feature of each of the second candidate words;

[0123] A noise confidence determination module 730, configured to determine the noise confidence of each speech frame included in the speech to be recognized, and determine the noise confidence of the speech to be recognized according to the noise confidence of each speech frame;

[0124] The execution module 740 is used to determine the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determine the speech recognition result according to the comprehensive confidence.

[0125] Based on the above embodiment, the decoding module 710 is specifically used for:

[0126] Decoding the speech to be recognized once based on a speech recognition module composed of a language model and an acoustic model to obtain a first word graph including the first candidate word;

[0127] Determine a language score and an acoustic score of each of the first candidate words, and determine the decoding output score of the speech to be recognized according to the language score and the acoustic score of each of the first candidate words;

[0128] On the first word graph, secondary decoding is performed based on the minimum Bayesian risk and the edit distance is used as a criterion to obtain the second candidate word corresponding to the speech to be recognized and the posterior probability of each second candidate word.

[0129] On the basis of the above embodiment, the decoding features include the confidence score, word category, probability distribution, word length and word graph depth of the second candidate word. Accordingly, the decoding confidence determination module 720 is specifically used to:

[0130] Normalizing the posterior probabilities of the second candidate words to obtain confidence scores of the second candidate words; determining the word categories of the second candidate words according to the category information of the second candidate words; determining the probability distribution of the second candidate words according to the number of occurrences of the second candidate words in all the second candidate words corresponding to the speech to be recognized; determining the word length of the second candidate words according to the number of phonemes contained in the second candidate words; determining the word graph depth of the second candidate words according to the number of outgoing edges of all nodes in the time period corresponding to the second candidate words and the length of the time period in the second word graph obtained by secondary decoding of the first candidate words;

[0131] Inputting the decoding features of each of the second candidate words into a pre-trained decoding confidence model to obtain the decoding confidence of each of the second candidate words;

[0132] The decoding confidence of the speech to be recognized is determined according to the decoding confidence of each of the second candidate words.

[0133] Based on the above embodiment, the noise confidence determination module 730 is specifically used for:

[0134] Dividing the speech to be recognized into frames to obtain speech frames contained in the speech to be recognized;

[0135] Determine the Mel-cepstral coefficients of each speech frame, and input the Mel-cepstral coefficients of each speech frame into a pre-trained noise confidence model to obtain the noise confidence of each speech frame;

[0136] The noise confidence of the speech to be recognized is determined according to the maximum noise confidence, the minimum noise confidence, the noise confidence mean and the noise confidence variance among the noise confidences of the speech frames contained in the speech to be recognized.

[0137] Based on the above embodiment, the execution module 740 is specifically used to:

[0138] Inputting the decoding confidence, noise confidence and decoding output score of the speech to be recognized into a pre-trained speech recognition model to obtain a comprehensive confidence of the speech to be recognized;

[0139] The speech recognition result is determined according to the comprehensive confidence.

[0140] In one implementation, the comprehensive confidence includes the probability that the speech to be recognized contains speech. Accordingly, determining the speech recognition result according to the comprehensive confidence includes:

[0141] If the probability that the speech to be recognized contains speech is greater than or equal to a first preset threshold, determining that the speech recognition result is that the speech to be recognized contains speech;

[0142] If the probability that the speech to be recognized contains speech is greater than or equal to the second preset threshold and less than the first preset threshold, determining that the speech recognition result is that the speech to be recognized does not contain speech;

[0143] If the probability that the speech to be recognized contains speech is less than the second preset threshold, it is determined that the speech recognition is wrong, or the speech to be recognized is optimized to obtain optimized speech, and speech recognition is performed again based on the optimized speech.

[0144] Further, optimizing the speech to be recognized to obtain optimized speech includes:

[0145] The speech frames whose noise confidence in the speech to be recognized is greater than a preset confidence are set to be silent to obtain the optimized speech.

[0146] The speech recognition device provided in the embodiment of the present invention can execute the speech recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the speech recognition method.

[0147] It is worth noting that in the embodiment of the above-mentioned speech recognition device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0148] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 8 A block diagram of an exemplary computer device 8 suitable for use in implementing embodiments of the present invention is shown. Figure 8 The computer device 8 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0149] like Figure 8 As shown, the computer device 8 is in the form of a general purpose computing computer device. The components of the computer device 8 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0150] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0151] The computer device 8 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 8, including volatile and non-volatile media, removable and non-removable media.

[0152] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 8 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0153] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0154] The computer device 8 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the computer device 8, and / or any device that enables the computer device 8 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the computer device 8 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. Figure 8 As shown, the network adapter 20 communicates with other modules of the computer device 8 via the bus 18. It should be understood that although Figure 8 Not shown, other hardware and / or software modules may be used in conjunction with computer device 8, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0155] The processing unit 16 executes various functional applications and page displays by running the program stored in the system memory 28, for example, implementing the speech recognition method provided in the embodiment of the present invention, which includes:

[0156] Determining, based on a first candidate word obtained by decoding the speech to be recognized, a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized;

[0157] Determining a decoding feature of each of the second candidate words, and determining a decoding confidence of the speech to be recognized according to the decoding feature of each of the second candidate words;

[0158] Determining the noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized according to the noise confidence of each speech frame;

[0159] The comprehensive confidence of the speech to be recognized is determined according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and the speech recognition result is determined according to the comprehensive confidence.

[0160] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the speech recognition method provided by any embodiment of the present invention.

[0161] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the speech recognition method provided by the embodiment of the present invention is implemented, for example, and the method includes:

[0162] Determining, based on a first candidate word obtained by decoding the speech to be recognized, a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized;

[0163] Determining a decoding feature of each of the second candidate words, and determining a decoding confidence of the speech to be recognized according to the decoding feature of each of the second candidate words;

[0164] Determining the noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized according to the noise confidence of each speech frame;

[0165] The comprehensive confidence of the speech to be recognized is determined according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and the speech recognition result is determined according to the comprehensive confidence.

[0166] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0167] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0168] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0169] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0170] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0171] In addition, the acquisition, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations.

[0172] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A speech recognition method, characterized in that: include: Determining, based on a first candidate word obtained by decoding the speech to be recognized, a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized; Determine a decoding feature of each of the second candidate words, and determine a decoding confidence of the speech to be recognized according to the decoding feature of each of the second candidate words, wherein the decoding feature includes a confidence score, a word category, a probability distribution, a word length, and a word graph depth of the second candidate words; Determining the noise confidence of each speech frame contained in the speech to be recognized, and determining the noise confidence of the speech to be recognized according to the noise confidence of each speech frame; The comprehensive confidence of the speech to be recognized is determined according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and the speech recognition result is determined according to the comprehensive confidence.

2. The speech recognition method according to claim 1, characterized in that: Determining a decoding output score of the speech to be recognized according to a first candidate word obtained by decoding the speech to be recognized, comprising: Decoding the speech to be recognized once based on a speech recognition module composed of a language model and an acoustic model to obtain the first candidate word; Determine a language score and an acoustic score of each of the first candidate words, and determine the decoding output score of the speech to be recognized according to the language score and the acoustic score of each of the first candidate words.

3. The speech recognition method according to claim 1, characterized in that: Determining, according to a first candidate word obtained by decoding the speech to be recognized, a second candidate word corresponding to the speech to be recognized, comprising: Decoding the speech to be recognized once based on a speech recognition module composed of a language model and an acoustic model to obtain a first word graph including the first candidate word; On the first word graph, secondary decoding is performed based on the minimum Bayesian risk and the edit distance is used as a criterion to obtain the second candidate word corresponding to the speech to be recognized and the posterior probability of each second candidate word.

4. The speech recognition method according to claim 3, characterized in that: Determining a decoding feature of each of the second candidate words includes: Normalizing the posterior probability of each of the second candidate words to obtain a confidence score of each of the second candidate words; Determining the word category of each of the second candidate words according to the category information of each of the second candidate words; Determining the probability distribution of each second candidate word according to the number of occurrences of each second candidate word in all second candidate words corresponding to the speech to be recognized; Determining the word length of each of the second candidate words according to the number of phonemes contained in each of the second candidate words; In the second word graph obtained by performing secondary decoding on the first candidate words, the word graph depth of each second candidate word is determined according to the number of outgoing edges of all nodes in the time period corresponding to each second candidate word and the length of the time period.

5. The speech recognition method according to claim 1, characterized in that: Determining the decoding confidence of the speech to be recognized according to the decoding features of each of the second candidate words, including: Inputting the decoding features of each of the second candidate words into a pre-trained decoding confidence model to obtain the decoding confidence of each of the second candidate words; The decoding confidence of the speech to be recognized is determined according to the decoding confidence of each of the second candidate words.

6. The speech recognition method according to claim 1, characterized in that: Determining the noise confidence of each speech frame included in the speech to be recognized includes: Dividing the speech to be recognized into frames to obtain speech frames contained in the speech to be recognized; The Mel-cepstral coefficients of each speech frame are determined, and the Mel-cepstral coefficients of each speech frame are respectively input into a pre-trained noise confidence model to obtain the noise confidence of each speech frame.

7. The speech recognition method according to claim 1, characterized in that: Determining the noise confidence of the speech to be recognized according to the noise confidence of each speech frame includes: The noise confidence of the speech to be recognized is determined according to the maximum noise confidence, the minimum noise confidence, the noise confidence mean and the noise confidence variance among the noise confidences of the speech frames contained in the speech to be recognized.

8. The speech recognition method according to claim 1, characterized in that: Determining the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, including: The decoding confidence, noise confidence and decoding output score of the speech to be recognized are input into a pre-trained speech recognition model to obtain a comprehensive confidence of the speech to be recognized.

9. The speech recognition method according to claim 1, characterized in that: The comprehensive confidence includes the probability that the speech to be recognized contains speech. Accordingly, determining the speech recognition result according to the comprehensive confidence includes: If the probability that the speech to be recognized contains speech is greater than or equal to a first preset threshold, determining that the speech recognition result is that the speech to be recognized contains speech; If the probability that the speech to be recognized contains speech is greater than or equal to the second preset threshold and less than the first preset threshold, determining that the speech recognition result is that the speech to be recognized does not contain speech; If the probability that the speech to be recognized contains speech is less than the second preset threshold, it is determined that the speech recognition is wrong, or the speech to be recognized is optimized to obtain optimized speech, and speech recognition is performed again based on the optimized speech.

10. The speech recognition method according to claim 9, characterized in that: Optimizing the speech to be recognized to obtain optimized speech includes: The speech frames whose noise confidence in the speech to be recognized is greater than a preset confidence are set to be silent to obtain the optimized speech.

11. A speech recognition device, characterized in that: include: A decoding module, configured to determine a decoding output score of the speech to be recognized and a second candidate word corresponding to the speech to be recognized based on a first candidate word obtained by decoding the speech to be recognized; A decoding confidence determination module, used to determine the decoding features of each of the second candidate words, and determine the decoding confidence of the speech to be recognized according to the decoding features of each of the second candidate words, wherein the decoding features include the confidence score, word category, probability distribution, word length and word graph depth of the second candidate words; A noise confidence determination module, used to determine the noise confidence of each speech frame contained in the speech to be recognized, and determine the noise confidence of the speech to be recognized according to the noise confidence of each speech frame; The execution module is used to determine the comprehensive confidence of the speech to be recognized according to the decoding confidence, noise confidence and decoding output score of the speech to be recognized, and determine the speech recognition result according to the comprehensive confidence.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the speech recognition method as described in any one of claims 1 to 10 is implemented.

13. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the speech recognition method according to any one of claims 1 to 10 when executed by a computer processor.

Images