A speech recognition method, apparatus, electronic device, and storage medium

Abstract

The application discloses a speech recognition method and device, electronic equipment and storage medium. According to the acoustic score of each first phoneme corresponding to each speech frame in the speech signal to be recognized, the decoding result of the speech signal to be recognized is determined. When the decoding result contains a preset keyword, for each second phoneme in the keyword, the weighted acoustic score is determined according to the acoustic score of the second phoneme in each speech frame corresponding to the second phoneme. The maximum weighted acoustic score is determined according to the maximum acoustic score of each speech frame in each speech frame. Then, the acoustic score confidence is determined according to the weighted acoustic score and the maximum weighted acoustic score. The acoustic score confidence obtained in this way can more accurately measure the credibility of the decoding result. Therefore, according to the acoustic score confidence, the accuracy of speech recognition can be ensured.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a speech recognition method, apparatus, electronic device and storage medium. Background Technology

[0002] Keyword recognition, as an important branch of speech recognition technology, is widely used in human-computer interaction and smart homes. For example, in some smart home scenarios, users wake up smart devices with a wake word and then speak a voice command they want the device to perform, instructing it to complete a specific action, such as "turn on the air conditioner." This process involves both voice wake-up and command word recognition, both of which are important applications of keyword recognition technology. However, in practical applications, false wake-ups and misrecognitions can occur. For instance, a user might say something completely unrelated to the command word, but it could be misrecognized as "turn on the air conditioner," which significantly impacts the user experience. Therefore, improving the accuracy of speech recognition is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] This application provides a speech recognition method, apparatus, electronic device, and storage medium to improve the accuracy of speech recognition.

[0004] This application provides a speech recognition method, the method comprising:

[0005] Acquire the speech signal to be recognized, extract the acoustic feature information of each speech frame in the speech signal, input the acoustic feature information of each speech frame into the trained acoustic model respectively, and obtain the acoustic score of each speech frame corresponding to each first phoneme respectively.

[0006] The acoustic fraction corresponding to each first phoneme of each speech frame is input into the trained decoding network to obtain the decoding result;

[0007] Determine whether the decoding result contains a preset keyword. If so, proceed to the following steps:

[0008] For each second phoneme in the keyword, a weighted acoustic score is determined based on the acoustic score of the second phoneme in each speech frame corresponding to the second phoneme; and a maximum weighted acoustic score is determined based on the maximum acoustic score of each speech frame in each speech frame.

[0009] Based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme, the acoustic score confidence level is determined, and the speech recognition result is determined based on the acoustic score confidence level.

[0010] Furthermore, before determining the speech recognition result based on the acoustic confidence score, the method further includes:

[0011] Based on the decoding results, determine the confidence level of at least one feature of the word graph lattice;

[0012] The step of determining the speech recognition result based on the acoustic confidence level includes:

[0013] If the confidence scores of the acoustic component and the confidence scores of at least one feature both meet the confidence threshold condition, the keyword is determined to be a speech recognition result; otherwise, the keyword is determined to be a misrecognition result.

[0014] Further, the step of determining the weighted acoustic score based on the acoustic score of the second phoneme in each speech frame corresponding to the second phoneme, and determining the maximum weighted acoustic score based on the maximum acoustic score of each speech frame, includes:

[0015] The weighted acoustic score is determined based on the sum of the acoustic scores of the second phoneme in each speech frame corresponding to the second phoneme.

[0016] The maximum weighted acoustic score is determined based on the sum of the maximum acoustic scores of each speech frame in each speech frame.

[0017] Further, determining the acoustic score confidence level based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme includes:

[0018] The total weighted acoustic score is determined based on the sum of the weighted acoustic scores corresponding to each of the second phonemes.

[0019] The total maximum weighted acoustic score is determined based on the sum of the maximum weighted acoustic scores corresponding to each of the second phonemes.

[0020] The confidence level of the acoustic score is determined based on the ratio of the total weighted acoustic score to the total maximum weighted acoustic score.

[0021] Further, determining the keyword as a speech recognition result when both the acoustic confidence score and the confidence score of at least one feature meet the confidence threshold condition includes:

[0022] If the acoustic score confidence level is greater than a preset first acoustic score confidence level threshold, and the confidence level of the at least one feature is greater than a preset second confidence level threshold, or if the acoustic score confidence level is greater than a preset second acoustic score confidence level threshold, and the confidence level of the at least one feature is greater than a preset first confidence level threshold, then the keyword is determined to be a speech recognition result; wherein the preset first acoustic score confidence level threshold is greater than the preset second acoustic score confidence level threshold, and the preset first confidence level threshold is greater than the preset second confidence level threshold.

[0023] Furthermore, the at least one feature includes at least one of the following: decoding path disorder, edit distance, total cost score difference, and language model backoff probability.

[0024] Furthermore, if the at least one feature includes the total cost score difference, the process for determining the confidence level of the at least one feature includes:

[0025] Obtain the total cost score of each path in the word graph lattice of the decoding result, and determine the confidence level of the total cost score difference based on the ratio of the lowest total cost score to the second lowest total cost score.

[0026] On the other hand, this application provides a voice recognition device, the device comprising:

[0027] The acquisition module is used to acquire the speech signal to be recognized, extract the acoustic feature information of each speech frame in the speech signal, input the acoustic feature information of each speech frame into the trained acoustic model respectively, and obtain the acoustic score of each speech frame corresponding to each first phoneme.

[0028] The determination module is used to input the acoustic fraction of each speech frame corresponding to each first phoneme into the trained decoding network to obtain the decoding result;

[0029] Determine whether the decoding result contains a preset keyword. If so, proceed to the following steps:

[0030] For each second phoneme in the keyword, a weighted acoustic score is determined based on the acoustic score of the second phoneme in each speech frame corresponding to the second phoneme; and a maximum weighted acoustic score is determined based on the maximum acoustic score of each speech frame in each speech frame.

[0031] The recognition module is used to determine the acoustic score confidence level based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme, and to determine the speech recognition result based on the acoustic score confidence level.

[0032] Furthermore, the determining module is also used to determine the confidence level of at least one feature of the word graph lattice based on the decoding result;

[0033] The recognition module is specifically used to determine the keyword as a speech recognition result if both the acoustic confidence score and the confidence score of the at least one feature meet the confidence score threshold condition; otherwise, it determines the keyword as a misrecognition result.

[0034] Furthermore, the determining module is specifically used to determine the weighted acoustic score based on the sum of the acoustic scores of the second phoneme in each speech frame corresponding to the second phoneme; and to determine the maximum weighted acoustic score based on the sum of the maximum acoustic scores of each speech frame in each speech frame.

[0035] Further, the determining module is specifically used to determine the total weighted acoustic score based on the sum of the weighted acoustic scores corresponding to each of the second phonemes; to determine the total maximum weighted acoustic score based on the sum of the maximum weighted acoustic scores corresponding to each of the second phonemes; and to determine the acoustic score confidence level based on the ratio of the total weighted acoustic score to the total maximum weighted acoustic score.

[0036] Further, the recognition module is specifically configured to determine the keyword as a speech recognition result if the acoustic score confidence level is greater than a preset first acoustic score confidence threshold and the confidence level of the at least one feature is greater than a preset second confidence threshold, or if the acoustic score confidence level is greater than a preset second acoustic score confidence threshold and the confidence level of the at least one feature is greater than a preset first confidence threshold; wherein, the preset first acoustic score confidence threshold is greater than the preset second acoustic score confidence threshold, and the preset first confidence threshold is greater than the preset second confidence threshold.

[0037] Furthermore, the at least one feature includes at least one of the following: decoding path disorder, edit distance, total cost score difference, and language model backoff probability.

[0038] Furthermore, the recognition module is specifically used to obtain the total cost score of each path in the word graph lattice of the decoding result, and to determine the confidence level of the total cost score difference based on the ratio of the lowest total cost score to the second lowest total cost score.

[0039] In another aspect, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0040] Memory, used to store computer programs;

[0041] When a processor executes a program stored in memory, it implements the steps of the method described in any of the preceding statements.

[0042] In another aspect, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any of the above claims.

[0043] This application provides a speech recognition method, apparatus, electronic device, and storage medium. The method includes: acquiring a speech signal to be recognized; extracting acoustic feature information of each speech frame in the speech signal; inputting the acoustic feature information of each speech frame into a trained acoustic model to obtain an acoustic score for each speech frame corresponding to each first phoneme; inputting the acoustic score of each speech frame corresponding to each first phoneme into a trained decoding network to obtain a decoding result; determining whether the decoding result contains a preset keyword; if so, proceeding to the following operations: for each second phoneme in the keyword, determining a weighted acoustic score based on the acoustic score of the second phoneme in each speech frame corresponding to the second phoneme, and determining a maximum weighted acoustic score based on the maximum acoustic score of each speech frame in each speech frame; determining an acoustic score confidence level based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme; and determining a speech recognition result based on the acoustic score confidence level.

[0044] The above technical solution has the following advantages or beneficial effects:

[0045] In this application, considering that a correctly pronounced speech segment identified as a keyword has a high acoustic score, the decoding result of the speech signal to be recognized is determined based on the acoustic score of each first phoneme corresponding to each speech frame in the speech signal to be recognized. When it is determined that the decoding result contains a preset keyword, a weighted acoustic score is determined for each second phoneme in the keyword based on the acoustic score of that second phoneme in each corresponding speech frame. A maximum weighted acoustic score is determined based on the maximum acoustic score of each speech frame. Then, an acoustic score confidence level is determined based on the weighted acoustic score and the maximum weighted acoustic score. The acoustic score confidence level obtained in this way can more accurately measure the reliability of the decoding result. Therefore, determining the speech recognition result based on the acoustic score confidence level can ensure the accuracy of speech recognition. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A schematic diagram of the speech recognition process provided in this application;

[0048] Figure 2 Detailed flowchart of speech recognition provided for this application;

[0049] Figure 3A schematic diagram of the speech recognition device provided in this application;

[0050] Figure 4 A schematic diagram of the electronic device structure provided in this application. Detailed Implementation

[0051] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0052] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0053] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0054] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0055] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0057] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0058] Figure 1 The speech recognition process provided in this application includes the following steps:

[0059] S101: Obtain the speech signal to be recognized, extract the acoustic feature information of each speech frame in the speech signal, input the acoustic feature information of each speech frame into the trained acoustic model respectively, and obtain the acoustic score of each speech frame corresponding to each first phoneme.

[0060] S102: Input the acoustic score of each first phoneme corresponding to each speech frame into the trained decoding network to obtain the decoding result; determine whether the decoding result contains a preset keyword; if so, proceed to the following operation: for each second phoneme in the keyword, determine the weighted acoustic score based on the acoustic score of the second phoneme in each speech frame corresponding to the second phoneme; and determine the maximum weighted acoustic score based on the maximum acoustic score of each speech frame in each speech frame.

[0061] S103: Determine the acoustic score confidence level based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme, and determine the speech recognition result based on the acoustic score confidence level.

[0062] The speech recognition method provided in this application is applied to electronic devices, which may be PCs, tablets, or servers.

[0063] After acquiring the speech signal to be recognized, the electronic device first performs frame segmentation on the speech signal to obtain each speech frame. Then, acoustic features are extracted from each speech frame to obtain its acoustic feature information. The electronic device stores a trained acoustic model, which can be trained within the electronic device itself or trained on another electronic device and then configured into this device. The acoustic feature information of each speech frame is then input into the trained acoustic model to obtain the acoustic score for each first phoneme corresponding to each speech frame.

[0064] The electronic device stores a trained decoding network. The acoustic score corresponding to each first phoneme in each speech frame is input into the trained decoding network to obtain a decoding result. The decoding result may or may not contain preset keywords. If the decoding result does not contain preset keywords, speech recognition ends. If the decoding result contains preset keywords, then for each second phoneme in the keyword, firstly, each speech frame containing that second phoneme is identified as the speech frames corresponding to that second phoneme. A weighted acoustic score is determined based on the acoustic score of that second phoneme in each speech frame corresponding to that second phoneme. Specifically, the weighted acoustic score is determined based on the sum of the acoustic scores of that second phoneme in each speech frame corresponding to that second phoneme. Optionally, the sum of the acoustic scores of that second phoneme in each speech frame corresponding to that second phoneme is used as the weighted acoustic score, or the sum of the acoustic scores of that second phoneme in each speech frame corresponding to that second phoneme is divided by the number of speech frames corresponding to that second phoneme to obtain the weighted acoustic score.

[0065] For each second phoneme in the keyword, determine the corresponding speech frames, then determine the maximum acoustic score of each speech frame. Based on the maximum acoustic score of each speech frame corresponding to the second phoneme, determine the maximum weighted acoustic score. Specifically, determine the maximum weighted acoustic score based on the sum of the maximum acoustic scores of each speech frame. Optionally, use the sum of the maximum acoustic scores of each speech frame as the maximum weighted acoustic score, or divide the sum of the maximum acoustic scores of each speech frame by the number of speech frames corresponding to the second phoneme to obtain the maximum weighted acoustic score.

[0066] Finally, based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme, the acoustic score confidence level is determined, and based on the acoustic score confidence level, the speech recognition result is determined. Specifically, determining the acoustic score confidence level based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme includes: determining the total weighted acoustic score based on the sum of the weighted acoustic scores corresponding to each second phoneme; determining the total maximum weighted acoustic score based on the sum of the maximum weighted acoustic scores corresponding to each second phoneme; and determining the acoustic score confidence level based on the ratio of the total weighted acoustic score to the total maximum weighted acoustic score.

[0067] Optionally, the sum of the weighted acoustic scores corresponding to each second phoneme can be used as the total weighted acoustic score, and the sum of the maximum weighted acoustic scores corresponding to each second phoneme can be used as the total maximum weighted acoustic score. The ratio of the total weighted acoustic score to the total maximum weighted acoustic score can be used as the acoustic score confidence level.

[0068] Finally, the speech recognition result is determined based on the acoustic score confidence score. Optionally, if the acoustic score confidence score is greater than a set threshold, the speech recognition result is determined to be a keyword contained in the decoding result; if the acoustic score confidence score is not greater than the set threshold, the speech recognition result is determined not to be a keyword contained in the decoding result.

[0069] In this application, considering that a correctly pronounced speech segment identified as a keyword has a high acoustic score, the decoding result of the speech signal to be recognized is determined based on the acoustic score of each first phoneme corresponding to each speech frame in the speech signal to be recognized. When it is determined that the decoding result contains a preset keyword, a weighted acoustic score is determined for each second phoneme in the keyword based on the acoustic score of that second phoneme in each corresponding speech frame. A maximum weighted acoustic score is determined based on the maximum acoustic score of each speech frame. Then, an acoustic score confidence level is determined based on the weighted acoustic score and the maximum weighted acoustic score. The acoustic score confidence level obtained in this way can more accurately measure the reliability of the decoding result. Therefore, determining the speech recognition result based on the acoustic score confidence level can ensure the accuracy of speech recognition.

[0070] In this application, to improve the accuracy of speech recognition, before determining the speech recognition result based on the acoustic confidence level, the method further includes:

[0071] Based on the decoding results, determine the confidence level of at least one feature of the word graph lattice;

[0072] The step of determining the speech recognition result based on the acoustic confidence level includes:

[0073] If the confidence scores of the acoustic component and the confidence scores of at least one feature both meet the confidence threshold condition, the keyword is determined to be a speech recognition result; otherwise, the keyword is determined to be a misrecognition result.

[0074] Specifically, if the acoustic score confidence level is greater than a preset first acoustic score confidence level threshold, and the confidence level of the at least one feature is greater than a preset second confidence level threshold, or if the acoustic score confidence level is greater than a preset second acoustic score confidence level threshold, and the confidence level of the at least one feature is greater than a preset first confidence level threshold, then the keyword is determined to be a speech recognition result; wherein, the preset first acoustic score confidence level threshold is greater than the preset second acoustic score confidence level threshold, and the preset first confidence level threshold is greater than the preset second confidence level threshold.

[0075] The at least one feature includes at least one of the following: decoding path disorder, edit distance, total cost score difference, and language model backoff probability.

[0076] If the at least one feature includes the total cost score difference, the process for determining the confidence level of the at least one feature includes:

[0077] Obtain the total cost score of each path in the word graph lattice of the decoding result, and determine the confidence level of the total cost score difference based on the ratio of the lowest total cost score to the second lowest total cost score.

[0078] The speech recognition process provided in this application will be described in detail below. Figure 2 The detailed flowchart of speech recognition provided for this application includes the following steps:

[0079] S201: The speech signal received by the keyword recognition algorithm is segmented into frames. After extracting acoustic features from the speech frames, the data is fed into a pre-trained acoustic model to obtain the acoustic score of each speech frame corresponding to each first phoneme, as well as the optimal acoustic score of each speech frame.

[0080] Specifically:

[0081] The received speech is generally a discrete-time signal, requiring processing such as framing, windowing, and pre-emphasis before acoustic feature extraction. This converts the time-domain speech signal into a speech feature vector that can be processed by the speech recognition system. The acoustic features can be a single feature, such as MFCC, FBANK, pitch, or i-vector features, or a fusion of multiple features, such as MFCC + Iverctor features. The extracted acoustic features are fed into a pre-trained acoustic model to obtain the acoustic score for each phoneme. Each node in the output layer of the acoustic model corresponds to one phoneme, and the value output by each node is the acoustic score for that phoneme. Furthermore, the maximum value of all output nodes is calculated as the maximum acoustic score for that speech frame.

[0082] Taking the simplest example of a single phoneme and a single node, assuming there are 200 phonemes in total, with each phoneme corresponding to one node, the output layer of the acoustic model will have a total of 200 nodes. When there is an input speech frame, the acoustic score of that frame corresponding to these 200 phonemes can be calculated. The maximum acoustic score of these 200 phonemes is then obtained.

[0083] S202: The acoustic component corresponding to each first phoneme of each speech frame is fed into a pre-trained decoding network to obtain the decoding result corresponding to the speech signal.

[0084] Specifically:

[0085] The decoding process is essentially the speech recognition process. In the decoding network, the Viterbi algorithm can be used to search for the optimal path, and multiple optimal recognition results can be preserved by saving the Lattice graph. Both the decoding graph and the Lattice graph are based on the fst format and consist of state nodes connected by transition arcs. The basic structure of a transition arc can be represented as: input label, output label, and weights. The input label is the output node of the acoustic model, i.e., the phoneme, and the output label is the final decoded word or character sequence. The weights include two parts: acoustic score and graph score. The graph score consists of the transition probabilities between the language model and the HMM state, while the acoustic score is the output value of the output node of the acoustic model. During the decoding process, the sequentially input speech frames are equivalent to the mutual jumps between state nodes in the decoding graph. The path with the lowest weight is the optimal decoding path.

[0086] S203: Check whether the decoding result contains preset keywords. If it does not contain keywords, end speech recognition. If it contains keywords, perform confidence judgment.

[0087] Specifically:

[0088] The recognition result is a sequence of words or characters consisting of output labels along the optimal path. In the decoding graph, the output labels will also be marked with a single quote ('). <sil>'to characterize silent or noisy audio segments, and'<SPOKEN_NOISE> 'or' <unk>The term '' represents out-of-series words that have speech but cannot be decoded. These output labels are often included in the recognition results, requiring post-processing to remove them. Simultaneously, spaces between valid word sequences used to represent the segmentation results are removed to obtain the final recognition result. The final recognition result is then compared with preset keywords. If the recognition result contains preset keywords, the confidence level of the recognition result needs to be calculated.

[0089] S204: Acoustic score confidence CM1 calculation: Obtain the valid speech frames corresponding to the keywords from the decoding results, obtain the number of speech frames and the acoustic score of each second phoneme in the keywords, calculate the weighted acoustic score of the keywords, and at the same time obtain the maximum acoustic score corresponding to the valid speech frames, calculate the maximum weighted acoustic score, and calculate the acoustic score confidence CM1 based on the weighted acoustic score and the maximum weighted acoustic score.

[0090] Specifically:

[0091] The Viterbi algorithm selects the optimal path based on the final total cost score, which includes both acoustic and graph scores. Therefore, an optimal total cost score does not necessarily mean an optimal acoustic score. Thus, for a speech segment misidentified as a keyword, even if the total cost score is optimal after decoding, there's still a possibility that pronunciation differences could result in a very low acoustic score. Considering that a correctly pronounced speech segment identified as a keyword will undoubtedly have a high acoustic score, the ratio between the decoded acoustic score and the optimal acoustic score can be used to measure the reliability of the decoding result.

[0092] Assume there are N valid speech frames corresponding to the identified keywords, and M phonemes corresponding to the keywords. The acoustic score and number of frames for each phoneme are as follows: and F i (i = 1, 2, ..., M), then the weighted acoustic score corresponding to the decoding result is:

[0093]

[0094] Regarding the aforementioned frame number F i Its optimal acoustic subhypothesis is S i For all valid speech frames corresponding to the keyword, the optimal weighted acoustic division is:

[0095]

[0096] Therefore, the confidence level CM1 is calculated as follows:

[0097]

[0098] For the effective number of speech frames N, whether or not to divide by N does not affect the final confidence result, so it can be ignored, that is, no frame normalization is performed.

[0099] For example, suppose the current decoding result of the keyword algorithm is "turn on the air conditioner", which corresponds to the phonemes "d, a, k, ai, k, ong, t, iao", a total of 8 phonemes. These 8 phonemes correspond to a total of 24 effective speech frames, with each phoneme corresponding to [2, 2, 2, 5, 1, 6, 2, 4]. The acoustic scores of each phoneme obtained from the decoding result are [1.8, 1.9, 1, 2.4, 2, 2.8, 2.7, 0.8]. The optimal acoustic score for the frame obtained in step one is [3, 2.7, 1, 2.8, 2.5, 2.8, 3, 1]. If frame number normalization is not performed, the weighted acoustic score of the phoneme "d" can be calculated as: 1.8 * 2 = 3.6, and the optimal weighted acoustic score for the 2 frames corresponding to the phoneme "d" is: 3 * 2 = 6.

[0100] Examples of the acoustic score and optimal acoustic score for each phoneme are given below:

[0101] Taking phoneme d as an example, if there are two speech frames corresponding to phoneme d, namely the first frame and the second frame, and the acoustic score of phoneme d is calculated to be 1.0 in the first frame and 0.8 in the second frame, then the acoustic score of phoneme d is 1.0 + 0.8 = 1.8. The optimal acoustic score of phoneme d is the sum of the optimal acoustic scores of the first frame and the second frame. For example, if the optimal acoustic score of the first frame is 1.6 and the optimal acoustic score of the second frame is 1.1.4, then the optimal acoustic score of phoneme d is 1.6 + 1.4 = 3.

[0102] The "weighting" in weighted acoustic score and weighted optimal acoustic score means that the number of frames corresponding to a phoneme is used as the "weight". In other words, the more frames there are, the higher the weight is assigned to that phoneme.

[0103] Therefore, without frame normalization, the weighted acoustic scores for the 8 phonemes are [3.6, 3.8, 2, 12, 2, 16.8, 5.4, 3.2], with a total weighted acoustic score of 48.8. The optimal weighted acoustic scores for each frame are [6, 5.4, 2, 14, 2.5, 16.8, 6, 4], with a total optimal weighted acoustic score of 56. Therefore, the confidence level CM1 = 48.8 / 56 = 0.86.

[0104] S205: Path score confidence CM2 calculation: Obtain the total path cost score corresponding to the optimal path and the total cost score of the second-best path from the decoding results, and calculate the path score confidence CM2.

[0105] Specifically:

[0106] The decoding result can retain multiple possible decoding paths using the Lattice format. Among these paths, the one with the lowest total cost score is the final decoding result. The number of candidate paths may be multiple or only one, depending on the reliability of the decoding. For a correctly pronounced and recognized keyword speech, the decoding result of this audio has high certainty, so there may only be one decoding path, or the total cost score of the optimal path in its Lattice result may be much lower than that of other paths. Conversely, for speech that is misidentified as a keyword, the certainty of the recognition result is very low, so the total cost score of the optimal path may be very close to that of other paths. Based on these characteristics, the difference in path cost scores can be used to evaluate the reliability of the decoding result.

[0107] In the Lattice graph of the decoded results, the two paths with the lowest total cost scores are labeled as the optimal path and the second-best path, respectively, and their respective total cost scores are labeled as S1 and S2. When the second-best path is empty, it indicates that the confidence of the current recognition result is very high, so CM2 = 1. Otherwise, compare the total cost scores of the optimal path and the second-best path:

[0108] CM2 = 1 - S1 / S2.

[0109] As can be seen, the larger CM2 is, the lower the total cost score of the optimal path is relative to the total cost score of the suboptimal path. The greater the differentiation between paths, the higher the credibility of the optimal path.

[0110] It should be noted that the path difference used in this scheme, referred to as CM2, is only described as an example. In practical applications, different basic features can be selected as the CM2 decision criteria based on different tasks, environments, and audio characteristics, such as edit distance, decoding path disorder, and language model backoff probability.

[0111] S206: Confidence Determination: Compare CM1 and CM2 with a preset confidence threshold. If both CM1 and CM2 are greater than the confidence threshold, the recognition result is considered to have high confidence, and the keyword recognition result is output; otherwise, it is considered that noise has been misidentified as a keyword, and the recognition result is discarded.

[0112] Specifically:

[0113] Considering that in most cases, a speech segment correctly identified as a keyword can exhibit high confidence across different decision criteria, but factors such as environmental noise and reverberation can still reduce speech clarity, leading to lower confidence for a particular decision criterion. Therefore, setting both confidence thresholds too high would decrease the keyword recognition rate. To address this, this scheme sets two thresholds for each confidence decision criterion: a high confidence threshold and a basic confidence threshold. A recognition result is considered reliable if one confidence level exceeds the high confidence threshold while the other exceeds the basic confidence threshold.

[0114] For confidence level CM1, A1 is set as the high confidence threshold and A2 as the basic confidence threshold, where A1 > A2. For confidence level CM2, B1 is set as the high confidence threshold and B2 as the basic confidence threshold, where B1 > B2. When the calculation results satisfy either condition 1 (CM1 > A1 and CM2 > B2) or condition 2 (CM1 > A2 and CM2 > B1), the decoding result is considered to have high confidence, and the identified keyword result is output. Otherwise, the decoding result is considered to have low confidence, indicating that background noise has been misidentified as a keyword, and the identification result is discarded.

[0115] Figure 3 The schematic diagram of the speech recognition device provided in this application shows that the device includes:

[0116] The acquisition module 31 is used to acquire the speech signal to be recognized, extract the acoustic feature information of each speech frame in the speech signal, input the acoustic feature information of each speech frame into the trained acoustic model respectively, and obtain the acoustic score of each speech frame corresponding to each first phoneme respectively.

[0117] The determining module 32 is used to input the acoustic score of each first phoneme corresponding to each speech frame into the trained decoding network to obtain the decoding result; determine whether the decoding result contains a preset keyword, and if so, proceed to the following operation: for each second phoneme in the keyword, determine the weighted acoustic score according to the acoustic score of the second phoneme in each speech frame corresponding to the second phoneme; and determine the maximum weighted acoustic score according to the maximum acoustic score of each speech frame in each speech frame.

[0118] The recognition module 33 is used to determine the acoustic score confidence level based on the weighted acoustic score and the maximum weighted acoustic score corresponding to each second phoneme, and to determine the speech recognition result based on the acoustic score confidence level.

[0119] The determining module 32 is further configured to determine the confidence level of at least one feature of the word graph lattice based on the decoding result;

[0120] The recognition module 33 is specifically used to determine the keyword as a speech recognition result if both the acoustic confidence score and the confidence score of the at least one feature meet the confidence score threshold condition; otherwise, it determines the keyword as a misrecognition result.

[0121] The determining module 32 is specifically used to determine the weighted acoustic score based on the sum of the acoustic scores of the second phoneme in each speech frame corresponding to the second phoneme; and to determine the maximum weighted acoustic score based on the sum of the maximum acoustic scores of each speech frame in each speech frame.

[0122] The determining module 32 is specifically used to determine the total weighted acoustic score based on the sum of the weighted acoustic scores corresponding to each of the second phonemes; to determine the total maximum weighted acoustic score based on the sum of the maximum weighted acoustic scores corresponding to each of the second phonemes; and to determine the acoustic score confidence level based on the ratio of the total weighted acoustic score to the total maximum weighted acoustic score.

[0123] The recognition module 33 is specifically configured to determine the keyword as a speech recognition result if the acoustic score confidence level is greater than a preset first acoustic score confidence threshold and the confidence level of the at least one feature is greater than a preset second confidence threshold, or if the acoustic score confidence level is greater than a preset second acoustic score confidence threshold and the confidence level of the at least one feature is greater than a preset first confidence threshold; wherein, the preset first acoustic score confidence threshold is greater than the preset second acoustic score confidence threshold, and the preset first confidence threshold is greater than the preset second confidence threshold.

[0124] The recognition module 33 is specifically used to obtain the total cost score of each path in the word graph lattice of the decoding result, and to determine the confidence level of the total cost score difference based on the ratio of the lowest total cost score to the second lowest total cost score.

[0125] This application also provides an electronic device, such as Figure 4 As shown, it includes: processor 301, communication interface 302, memory 303 and communication bus 304, wherein processor 301, communication interface 302 and memory 303 communicate with each other through communication bus 304;

[0126] The memory 303 stores a computer program, which, when executed by the processor 301, causes the processor 301 to perform any of the above method steps.

[0127] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0128] Communication interface 302 is used for communication between the above-mentioned electronic device and other devices.

[0129] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0130] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0131] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform any of the above method steps.

[0132] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0133] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.< / unk> < / sil>

Claims

1. A voice recognition method, characterized by, The method comprises: acquiring a voice signal to be recognized, extracting acoustic feature information of each voice frame in the voice signal, inputting the acoustic feature information of each voice frame into a trained acoustic model respectively, and obtaining acoustic scores of each first phoneme corresponding to each voice frame respectively; inputting the acoustic scores of each first phoneme corresponding to each voice frame into a trained decoding network to obtain a decoding result; judging whether the decoding result contains a preset keyword, and if so, performing the following operations: obtaining effective voice frames corresponding to the keyword from the decoding result, obtaining the number of voice frames and acoustic scores of each second phoneme in the keyword, for each second phoneme, dividing the sum of the acoustic scores of the second phoneme in each effective voice frame corresponding to the second phoneme by the number of effective voice frames corresponding to the second phoneme to calculate a weighted acoustic score of the keyword, simultaneously obtaining a maximum acoustic score corresponding to the effective voice frames, calculating a maximum weighted acoustic score, and calculating an acoustic score confidence according to the ratio of the weighted acoustic score to the maximum weighted acoustic score; determining a confidence of at least one feature of a word lattice according to the decoding result; if the acoustic score confidence is greater than a preset first acoustic score confidence threshold, the confidence of the at least one feature is greater than a preset second confidence threshold, or the acoustic score confidence is greater than a preset second acoustic score confidence threshold and the confidence of the at least one feature is greater than a preset first confidence threshold, determining that the keyword is a voice recognition result; wherein the preset first acoustic score confidence threshold is greater than the preset second acoustic score confidence threshold, and the preset first confidence threshold is greater than the preset second confidence threshold, otherwise determining that the keyword is a misrecognition result.

2. The method of claim 1, wherein, The at least one feature comprises at least one of a decoding path confusion degree, an edit distance, a total cost score gap, and a language model back-off probability.

3. The method of claim 2, wherein, If the at least one feature comprises a total cost score gap, the confidence of the at least one feature is determined by: obtaining total cost scores of each path in a word lattice of the decoding result, and determining a confidence of the total cost score gap according to the ratio of the lowest total cost score to the second lowest total cost score.

4. A speech recognition apparatus characterized by comprising: The apparatus comprises: an acquisition module configured to acquire a voice signal to be recognized, extract acoustic feature information of each voice frame in the voice signal, input the acoustic feature information of each voice frame into a trained acoustic model respectively, and obtain acoustic scores of each first phoneme corresponding to each voice frame respectively; The determining module is configured to input the acoustic score of each phoneme corresponding to each speech frame into a trained decoding network to obtain a decoding result; determine whether the decoding result contains a preset keyword; if yes, perform the following operations: obtain valid speech frames corresponding to the keyword from the decoding result, obtain the number of speech frames and the acoustic score of each second phoneme in the keyword, for each second phoneme, divide the sum of the acoustic scores of the second phoneme in each valid speech frame corresponding to the second phoneme by the number of valid speech frames corresponding to the second phoneme to calculate a weighted acoustic score of the keyword, obtain the maximum acoustic score corresponding to the valid speech frames, calculate a maximum weighted acoustic score, and calculate an acoustic score confidence according to the ratio of the weighted acoustic score to the maximum weighted acoustic score; and determine a confidence of at least one feature of a word lattice according to the decoding result. The recognition module is configured to determine the keyword as a speech recognition result if the acoustic score confidence is greater than a preset first acoustic score confidence threshold, the confidence of the at least one feature is greater than a preset second confidence threshold, or the acoustic score confidence is greater than a preset second acoustic score confidence threshold and the confidence of the at least one feature is greater than a preset first confidence threshold; otherwise, determine the keyword as a misrecognition result. The preset first acoustic score confidence threshold is greater than the preset second acoustic score confidence threshold, and the preset first confidence threshold is greater than the preset second confidence threshold.

5. An electronic device, comprising: The apparatus includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The memory is configured to store a computer program. The processor is configured to execute the program stored in the memory to implement the method steps in any one of claims 1-3.

6. A computer readable storage medium characterized by, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps in any one of claims 1-3.

Images