An acoustic model performance evaluation method, an acoustic recognition method, and related devices

Abstract

The application discloses an acoustic model performance evaluation method, an acoustic recognition method and related devices. The acoustic model performance evaluation method comprises the following steps: obtaining a speech training set and a language decoding graph, wherein the speech training set comprises at least one piece of speech, and the language decoding graph comprises a pronunciation dictionary, and the pronunciation dictionary comprises a corresponding relationship table between a single character and a phoneme; inputting the speech training set into an acoustic model to extract sound state information; inputting the sound state information into the language decoding graph to extract a recognition result of the speech training set; and performing performance evaluation on the acoustic model by using the recognition result. In the foregoing manner, the acoustic model is not disturbed by a language model, and a more accurate evaluation result can be obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an acoustic model performance evaluation method, an acoustic recognition method and related devices. BACKGROUND

[0002] Technology is constantly progressing, and people expect to control machine devices through voice. Speech recognition technology is applied in smart home, machine control, vehicle-mounted system, electronic products, medical treatment, office conference, etc. The statistical analysis model based on HMM (Hidden Markov Model) in speech recognition technology has achieved great success.

[0003] The speech recognition based on HMM includes feature extraction, acoustic model, language model, pronunciation dictionary, and decoder part. There are countless paths on the decoding graph, and each path represents a recognition result. A score is calculated for the current frame in the recognition process, and the score is composed of two parts, acoustic score and graph inherent score. The sum of the two parts is the score corresponding to the jump. The graph inherent score mainly comes from the language model, and is also affected by the selection of multi-phonetic words in the pronunciation dictionary and the transition probability of the HMM language model. These graph inherent scores are irrelevant to the speech to be recognized, but will affect the decoding result. The evaluation according to the decoding result is the evaluation of the whole speech recognition system, not the independent evaluation of the acoustic model. Therefore, in the prior art, the recognition result is disturbed by the language model. SUMMARY

[0004] The technical problem solved by the present application is to provide an acoustic model performance evaluation method, an acoustic recognition method and related devices, which can make the acoustic model not be disturbed by the language model and obtain more accurate evaluation results.

[0005] To solve the above technical problem, one technical solution adopted by the present application is to provide an acoustic model performance evaluation method, which comprises the following steps: obtaining a speech training set and a language decoding graph, the speech training set comprising at least one speech, the language decoding graph comprising a pronunciation dictionary, the pronunciation dictionary comprising a corresponding relationship table between single words and phonemes; inputting the speech training set into an acoustic model to extract sound state information; inputting the sound state information into the language decoding graph to extract a recognition result of the speech training set; and performing performance evaluation on the acoustic model by using the recognition result.

[0006] In the method, the language decoding graph is obtained by the following steps: performing single word splitting on a corpus sentence in a corpus to obtain a corpus single word set; constructing a corresponding relationship table between the corpus single words and phonemes to obtain the pronunciation dictionary; and combining the pronunciation dictionary with a language model, and making the probability of occurrence of each word in the language model the same to obtain the language decoding graph.

[0007] The corresponding relationship table of the corpus single character and the phoneme is constructed, and a pronunciation dictionary is obtained, comprising: removing repeated corpus single characters; and constructing the pronunciation dictionary by using the corpus single characters, the phonemes of the corpus single characters, the silence words and the phonemes of the silence words, and the out-of-set words and the phonemes of the out-of-set words.

[0008] The language model is trained by using the corpus single character set, wherein the weights of the corpus single characters in the corpus single character set are the same, so that the probabilities of the appearance of each character in the language model are the same.

[0009] The language decoding graph is obtained by further comprising: obtaining a context-related acoustic transcription machine; and combining the acoustic transcription machine, the pronunciation dictionary and the language model to obtain the language decoding graph.

[0010] The pronunciation dictionary and the language model are in the form of a finite state transcription machine.

[0011] The voice state information is extracted by inputting the voice training set into the acoustic model, comprising: performing frame division, windowing and pre-emphasis processing on the voice training set; and extracting acoustic features of the voice training set by using the acoustic model to obtain the voice state information, wherein the voice state information comprises at least one acoustic feature or a fusion feature of multiple acoustic features.

[0012] The recognition result of the voice training set is extracted by inputting the voice state information into the language decoding graph, comprising: obtaining context state information of the voice state information by using the context-related acoustic transcription machine; and converting the context state information of the voice state information into a recognition result of the character information by using the pronunciation dictionary and the language model.

[0013] The performance of the acoustic model is evaluated by using the recognition result, comprising: obtaining the number of multiple recognition characters, the number of missed recognition characters and the number of misrecognized characters in the recognition result; calculating the average character error rate by using the number of multiple recognition characters, the number of missed recognition characters, the number of misrecognized characters and the total number of characters of the voice training set; and the average character error rate and the recognition accuracy of the acoustic model are negatively correlated.

[0014] The number of misrecognized characters is obtained by counting the number of characters in the recognition result which are different from the corresponding characters of the voice training set and the phonemes of which are different from the phonemes of the corresponding voice training set as the number of misrecognized characters.

[0015] To solve the above technical problems, another technical solution adopted by the present application is to provide an acoustic recognition method, comprising: obtaining voice; identifying voice state information of the voice by using an acoustic model; and obtaining character information of the voice state information by using a language decoding graph, wherein the language decoding graph comprises a pronunciation dictionary, and the pronunciation dictionary comprises a corresponding relationship table between single characters and phonemes.

[0016] Wherein, the sound state information of the voice is identified by using the acoustic model, and the method comprises the following steps: evaluating the acoustic model by using the acoustic model performance evaluation method; obtaining misrecognized words in the evaluation process, wherein the misrecognized words comprise multi-recognized words, missed-recognized words and wrongly-recognized words; and adjusting the weight of the language model in the language decoding graph based on the misrecognition rate of the misrecognized words.

[0017] Wherein, the acoustic model is the model with the highest recognition accuracy.

[0018] Wherein, the adjusting the weight of the language model in the language decoding graph based on the misrecognition rate of the misrecognized words comprises the following steps: sorting the misrecognized words, selecting a preset number of words with the highest misrecognition rate; and increasing the weight of the words in the language model.

[0019] To solve the above technical problems, another technical solution adopted by the present application is to provide a data device, and the acoustic model performance evaluation device comprises a processor, and the processor is used to execute to realize the acoustic model performance evaluation method and the acoustic recognition method.

[0020] To solve the above technical problems, another technical solution adopted by the present application is to provide a computer readable storage medium, and the computer readable storage medium is used to store instructions / program data, and the instructions / program data can be executed to realize the acoustic model performance evaluation method and the acoustic recognition method.

[0021] The beneficial effects of the present application are: different from the prior art, the present application constructs a pronunciation dictionary before single words and phonemes, ensures that the probability of occurrence of each word is the same, and does not affect the recognition of other words, in the case of ensuring that the graph inherent of the language decoding graph is consistent, the sound information is converted into text information by using the acoustic model and the language decoding graph, the performance of the acoustic model is indirectly evaluated by using the recognition result of the text information, so that the acoustic model is not disturbed by the language model, and a more accurate evaluation result is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a flowchart of an embodiment of the acoustic model performance evaluation method of the present application;

[0023] Figure 2 is a flowchart of an embodiment of the language decoding graph construction method of the present application;

[0024] Figure 3 is a flowchart of another embodiment of the language decoding graph construction method of the present application;

[0025] Figure 4 is a flowchart of another embodiment of the acoustic model performance evaluation method of the present application;

[0026] Figure 5 is a flowchart of a specific embodiment of the acoustic model performance evaluation method of the present application;

[0027] Figure 6 is a flowchart of an embodiment of the acoustic recognition method of the present application;

[0028] Figure 7 is a structural diagram of an acoustic model performance evaluation device in an embodiment of the present application;

[0029] Figure 8 is a structural diagram of a data device in an embodiment of the present application;

[0030] Figure 9 is a structural diagram of a computer readable storage medium in an embodiment of the present application. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions and effects of the present application clearer and more explicit, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0032] The acoustic model performance evaluation method provided by the present application cannot accurately evaluate the performance of the acoustic model only by using the acoustic model recognition result. Therefore, the acoustic model and the language model are used to convert the sound information into language information, and the performance of the acoustic model is evaluated by using the accuracy of the language information. In the HMM, one phoneme is assumed to have 3 to 5 states, a non-silent phoneme is generally set to have 3 states, and a silent phoneme is set to have 5 states. The states can be transferred between each other with a certain probability. Generally, the language model and the pronunciation dictionary together constitute a language decoding graph. The decoding graph has innumerable paths, each of which represents a recognition result. There is a score on each path, which represents the matching degree of the speech and the recognition result, and the higher the score, the higher the matching degree. The decoding graph is composed of a plurality of nodes and a plurality of jumps. Some jumps correspond to an HMM state, and a score is calculated for the current frame in the recognition process. However, due to the influence of the inherent score of the language decoding graph, the language model has a certain error correction ability for the recognition result. At this time, the recognition result reflects the performance of the entire speech recognition system, i.e., the combined performance of the acoustic model and the language model. Therefore, the present application re-constructs a language decoding graph, uses the relationship between a single word and a phoneme to construct a pronunciation dictionary, so that the inherent score of each path in the language decoding graph is consistent, and the error correction ability of the language model is eliminated. The recognition result can independently reflect the performance of the acoustic model.

[0033] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the acoustic model performance evaluation method of the present application. It should be noted that if there are substantially the same results, the present embodiment does not exclude Figure 1The flow sequence shown is limited. As Figure 1 The embodiment includes:

[0034] S110: Obtain a voice training set and a language decoding graph.

[0035] The voice training set includes at least one voice and corresponding text, and the voice training set is used as a sound material. The language decoding graph is used to convert phonemes into text in advance, wherein the language decoding graph includes a pronunciation dictionary, and the pronunciation dictionary includes a corresponding relationship table between single characters and phonemes.

[0036] S130: Input the voice training set into an acoustic model to extract sound state information.

[0037] The voice information in the voice training set is input into the acoustic model to extract phoneme-level information, i.e., to convert language into sound state information. Different acoustic models can obtain different sound state information. The acoustic model performance evaluation method of the present application is used to evaluate whether the recognized sound state information is accurate, but the sound state information is not convenient for direct evaluation. Therefore, the sound state information is further converted using the language decoding graph.

[0038] S150: Input the sound state information into the language decoding graph to extract the recognition result of the voice training set.

[0039] The sound state information is input into the language decoding graph, and the phoneme-level information is converted into language-level information through a search path to obtain the recognition result of the voice training set. In the language decoding graph, the single character of the pronunciation dictionary corresponds to the phoneme, so that each path on the language decoding graph has consistent graph inherent points, reducing the rectification ability of the language decoding graph to the acoustic model.

[0040] S170: Evaluate the performance of the acoustic model using the recognition result.

[0041] The recognition result is compared with the text in the voice training set, and the performance of the acoustic model is evaluated according to the recognition error rate to obtain the recognition accuracy of the acoustic model.

[0042] In this embodiment, by constructing the pronunciation dictionary before the single character and the phoneme, the probability of each character appearing is ensured to be the same, and the recognition of other characters is not affected. In the case of ensuring that the graph inherent points of the language decoding graph are consistent, the sound information is converted into text information by using the acoustic model and the language decoding graph, the performance of the acoustic model is indirectly evaluated by using the recognition result of the text information, the acoustic model is not disturbed by the language model, and a more accurate evaluation result is obtained.

[0043] Before the performance of the acoustic model is evaluated, a language decoding graph needs to be trained first. Each sound information input will output a corresponding text information. In an embodiment, the language decoding graph includes a pronunciation dictionary and a language model. Please refer to Figure 2 , Figure 2 is a flowchart of an embodiment of the present application for constructing a language decoding graph. It should be noted that the embodiment is not limited to the flow sequence shown in Figure 2 . As shown in Figure 2 , the embodiment includes:

[0044] S210: Constructing a pronunciation dictionary.

[0045] Generally, a corpus is obtained first, and each corpus sentence in the corpus is segmented. In the embodiment of the present application, the corpus sentences in the corpus are split into single characters to obtain a corpus single character set. The pronunciation of each single character is taken as a phoneme, and a corresponding relationship table of the corpus single character and the phoneme is constructed to obtain a pronunciation dictionary. In an embodiment, there are corpus single characters that appear multiple times in the corpus sentences, and therefore, the repeated corpus single characters in the corpus single character set are removed before the pronunciation dictionary is constructed, so that the obtained pronunciation dictionary does not have repeated characters.

[0046] In addition, the pronunciation of the language system includes not only the pronunciation of the corpus single characters split from the corpus, but also the pronunciation of the silence word SIL and the out-of-vocabulary word SPOKEN_NOISE. Therefore, in an embodiment, the pronunciation dictionary is constructed by using the corpus single characters and the phonemes of the corpus single characters, the silence word and the phonemes of the silence word, and the out-of-vocabulary word and the phonemes of the out-of-vocabulary word.

[0047] S230: Training a language model.

[0048] The corpus single character set is used to train an N-gram language model to obtain a language model. Specifically, the historical conditional probability of N-1 words is modeled, i.e.,

[0049] p(w i |w i-(N-1) …w i-2 w i-1 );

[0050] The conditional probability is obtained by collecting a large amount of corpus and counting the number, i.e.,

[0051]

[0052] wherein the weights of the corpus single characters are the same.

[0053] S250: Compounding the pronunciation dictionary and the language model, and making the probability of each character appearing in the language model the same to obtain a language decoding graph.

[0054] After the pronunciation dictionary and the language model, a language decoding graph is obtained, in which the input phonemes are recognized to correspond to the text. Since the pronunciation dictionary is constructed using non-repeated characters, and the weights of the corpus characters in the language model are the same, the probability of recognizing each character in the language model is the same.

[0055] In another embodiment, the language decoding graph includes a context-dependent acoustic transcription machine, a pronunciation dictionary, and a language model. Please refer to Figure 3 , Figure 3 is a flowchart of another embodiment of the present application for constructing a language decoding graph. It should be noted that the present embodiment is not limited to the flow sequence shown in Figure 3 . As shown in Figure 3 , the present embodiment includes:

[0056] S310: Construct a pronunciation dictionary.

[0057] S330: Train a language model.

[0058] S350: Obtain a context-dependent acoustic transcription machine.

[0059] The context-dependent acoustic transcription machine is used to further extract and recognize the sound information recognized by the acoustic model, and obtain sound information with more complex pronunciation habits and rules by combining the direct composite connection relationship of each phoneme and adjacent phonemes.

[0060] S370: Compound the acoustic transcription machine, the pronunciation dictionary, and the language model to obtain a language decoding graph.

[0061] After the pronunciation dictionary and the language model, a language decoding graph is obtained, in which the input phonemes are recognized to correspond to the text. Since the pronunciation dictionary is constructed using non-repeated characters, and the weights of the corpus characters in the language model are the same, the probability of recognizing each character in the language model is the same.

[0062] In an embodiment, the pronunciation dictionary and the language model are both in the form of a finite state transcription machine.

[0063] In the embodiment, the language model is trained, the same weight of each word in the language model is set, and the one-to-one pronunciation dictionary of the character and the phoneme is combined, so that the probability of each word appearing in the language model is consistent, the acoustic model is free from the correction ability of the language model during decoding, and is not affected by the language model. On this basis, the performance and effect of the acoustic model can be independently identified and judged. At the same time, the context-related acoustic transcription machine is added to the language decoding graph, the correlation between the sound information can be learned before the sound information and the character information are converted, and the language decoding graph identification is facilitated.

[0064] Please refer to Figure 4 , Figure 4 is a flowchart of another embodiment of the acoustic model performance evaluation method of the present application. It should be noted that the sequence of the flowchart shown in Figure 4 is not limited if there is substantially the same result. As shown in Figure 4 , the embodiment includes:

[0065] S410: inputting the speech training set into the acoustic model to extract sound state information.

[0066] S430: inputting the sound state information into the language decoding graph to extract the recognition result of the speech training set.

[0067] The context state information of the sound state information is obtained by using the context-related acoustic transcription machine, and the recognition result of the character information is obtained by using the pronunciation dictionary and the language model to convert the context state information of the sound state information.

[0068] S450: obtaining the number of multiple recognition words, the number of missed recognition words, and the number of wrong recognition words in the recognition result.

[0069] The recognition result may have multiple errors, such as the position where the recognized word does not exist a correct answer is recognized as an error; the word exists in the correct answer but is missed; the word is incorrectly recognized as another word, etc. The number of wrong recognition words is counted, wherein the number of wrong recognition words is the number of words that are different from the corresponding speech training set characters and phonemes in the recognition result.

[0070] S470: calculating the average word error rate by using the number of multiple recognition words, the number of missed recognition words, the number of wrong recognition words, and the total number of words of the speech training set.

[0071] The average word error rate is calculated by using the number of wrong recognition words and the total number of words, and the average word error rate and the recognition accuracy of the acoustic model are negatively correlated, that is, the lower the average word error rate, the higher the performance of the acoustic model.

[0072] Please refer to Figure 5 , Figure 5It is a schematic flowchart of a specific implementation manner of the acoustic model performance evaluation method of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited by Figure 5 the process sequence shown. As Figure 5 shown, this embodiment includes:

[0073] S510: Construct a language decoding graph.

[0074] The language decoding graph includes a language model G.fst in the form of a finite state transducer, a pronunciation dictionary L.fst in the form of a finite state transducer, and a context-dependent transducer C.fst.

[0075] First, obtain a corpus, clean each sentence in the corpus, and split it into a corpus single-word set. For example, the sentence "What's the weather like tomorrow" is split into seven words: "tomorrow", "weather", "what", "like", "tomorrow", "is", "it". Among them, there are two repeated "tomorrow" in the seven words. Remove the duplicates of the corpus single words to get six words: "tomorrow", "weather", "what", "like", "is", "it". After splitting all the sentences in the corpus and removing duplicates together, obtain the de-duplicated corpus. And based on the de-duplicated corpus single words and the pronunciation phonemes corresponding to the corpus single words, the silence word SIL and the phonemes of the silence word SIL, the out-of-vocabulary word SPOKEN_NOISE and the phonemes of the out-of-vocabulary word SPOKEN_NOISE, construct a pronunciation dictionary through the G2P (Grapheme to Phoneme) tool of an open-source Chinese dictionary. Tools such as Sequitur G2P and G2P-Seq2Seq can be used to train the pronunciation dictionary generation model.

[0076] At the same time, use the corpus single-word set to train the language model to obtain the language model. In one embodiment, use the n-gram language model to train the language model. Specifically, the 1-gram language model can be used to train the language model. Among them, set the weights of each corpus single word in the corpus single-word set to be the same. The probability of each word appearing in the language model will be 1 / N, where N is the number of entries in the pronunciation dictionary, and set the probability of the silence word to be 0.5.

[0077] Compound the context-dependent transducer C.fst, the pronunciation dictionary L.fst, and the language model G.fst in sequence, and perform graph determinization, minimization, adding self-loops, removing disambiguation symbols, etc., to obtain the final language decoding graph.In such a decoding graph, the scores of each decoding path are the same, and then the acoustic model will not be affected by the language model during decoding.

[0078] S530: Extract voice state information.

[0079] The voice signal of the voice training set is subjected to frame, windowing, pre-emphasis and other processing, acoustic features are extracted, and sound state information is obtained. The sound state information includes at least one acoustic feature or a fusion feature of multiple acoustic features. The acoustic features can be MFCC features, FBank features, PLP features, pitch features, ivector features, or a fusion of the above-mentioned multiple acoustic features. The acoustic features are subjected to an acoustic model to obtain the distribution probability of the phonemes corresponding to the voice frame, i.e., acoustic score.

[0080] S550: Decoding the recognition result by using the language decoding graph.

[0081] The test set is decoded, path searching is performed on the language decoding graph, and the decoding result is determined. Specifically, the context state information of the sound state information is obtained by using the context-dependent acoustic transcription machine, and the context state information of the sound state information is further converted into the recognition result of the text information by using the pronunciation dictionary and the language model.

[0082] S570: Performance evaluation of the acoustic model.

[0083] In this embodiment, the performance of the acoustic model is evaluated by using the most commonly used evaluation index of the voice system, the average word error rate WER. Specifically, the number of multiple recognition words, the number of missed recognition words and the number of wrong recognition words in the recognition result are obtained, and the calculation method of the average word error rate is as follows:

[0084]

[0085] wherein, Insertion represents that the recognized word does not exist in the correct answer position but is incorrectly recognized, i.e., the number of multiple recognition words; Deletion represents that the word exists in the correct answer but is missed, i.e., the number of missed recognition words; Substitute represents that the word is incorrectly recognized as other words, i.e., the number of wrong recognition words; Words represents the total number of words of the voice training set. Among them, because only the acoustic model cannot distinguish homophonic words such as "rain" and "language", therefore, the performance of the language model is reflected in the final result of the recognition error of the homophonic words, and therefore, when counting the number of wrong recognition words, part of the words with the same pronunciation are excluded, because there are many homophonic words, and the same pronunciation means that the acoustic model recognizes correctly. The average word error rate WER of each sentence is calculated, and the WER of the entire test set is randomly calculated, which can reflect the performance of the acoustic model.

[0086] In a specific embodiment, acoustic models iterated to different numbers of times are selected, the customized language model is used, and the performance of the acoustic model is measured by using WER. Table 1 is an acoustic model performance evaluation result.

[0087] Table 1 Acoustic model performance evaluation result

[0088] Acoustic model deletion insertion substitution WER (word error rate) 15-layer tdnnf (ite = 100) 784 50 1651 13.75 15-layer tdnnf (ite = 400) 543 61 1420 11.03 15-layer tdnnf (ite = 519) 453 36 1383 10.14

[0089] The performance of the acoustic model is evaluated by using the above method, the misrecognized result is viewed, and the misrecognized text in the evaluation process is obtained, including multi-recognition text, missed recognition text and wrong recognition text. The weight in the language model in the language decoding graph is adjusted based on the misrecognition rate of the misrecognized text. Specifically, the misrecognized result is viewed, the misrecognized result is sorted, and the preset number of texts with the highest misrecognition rate, i.e. the top N characters in the sorting, are explained. It is indicated that the acoustic model cannot be well distinguished, and the weight of these characters can be appropriately increased in the normal language model training corpus. The acoustic recognition is performed by using the acoustic model and the language model.

[0090] In this embodiment, it is determined that the performance of the acoustic model is insufficient when processing some words, and the weight of these words can be appropriately increased in the normal language model training, which can effectively compensate for the insufficient performance of the acoustic model.

[0091] In another embodiment, a plurality of acoustic models are evaluated by using the above acoustic model evaluation method, and the acoustic model with the highest recognition accuracy, i.e. the lowest average character error rate, is selected for acoustic recognition, and the language model is adjusted at the same time.

[0092] In this embodiment, by completely disassembling the corpus participating in the training into single characters and removing the duplicates, a pronunciation dictionary before single characters and phonemes is constructed, and a language model with the same weight of single characters is trained based thereon, so as to ensure that the occurrence probability of each character is the same and does not affect the recognition of other characters. In the case where the graph inherent consistency of the language decoding graph is ensured and the language model loses the ability to correct the acoustic model, the sound information is converted into text information by using the acoustic model and the language decoding graph, the performance of the acoustic model is indirectly evaluated by using the recognition result of the text information, so that the acoustic model is not disturbed by the language model, and a more accurate evaluation result is obtained. By judging the decoding result of the test set, it is determined that the performance of the acoustic model is insufficient when processing some words. In the normal language model training, the weight of these characters is appropriately increased, which will improve the overall performance of the speech recognition system.

[0093] Please refer to Figure 6 , Figure 6 is a flowchart of an embodiment of the acoustic recognition method of the present application. It should be noted that the order of the flowchart shown in Figure 6 is not limited by the embodiment. As shown in Figure 6 , the embodiment includes:

[0094] S610: Acquire speech.

[0095] S630: Recognize the sound state information of the speech by using the acoustic model.

[0096] S650: obtaining the character information of the sound state information by using the language decoding graph.

[0097] The language decoding graph comprises a pronunciation dictionary, and the pronunciation dictionary comprises a correspondence table between a single character and a phoneme.

[0098] Referring to Figure 7 , Figure 7 is a structural schematic diagram of an acoustic model performance evaluation device in an embodiment of the present application. In the embodiment, the acoustic model performance evaluation device comprises an acquisition module 71, an acoustic module 72, a language module 73 and an evaluation module 74.

[0099] The acquisition module 71 is configured to acquire a speech training set and a language decoding graph, the speech training set comprising at least one piece of speech, and the language decoding graph comprising a pronunciation dictionary, and the pronunciation dictionary comprising a correspondence table between a single character and a phoneme; the acoustic module 72 is configured to input the speech training set into an acoustic model to extract sound state information; the language module 73 is configured to input the sound state information into the language decoding graph to extract a recognition result of the speech training set; and the evaluation module 74 is configured to perform performance evaluation on the acoustic model by using the recognition result. The acoustic model performance evaluation device is configured to construct the pronunciation dictionary between the single character and the phoneme, ensure the same probability of occurrence of each character, and not affect the recognition of other characters, in the case of ensuring the consistency of the graph inherent part of the language decoding graph, convert the sound information into the character information by using the acoustic model and the language decoding graph, indirectly evaluate the performance of the acoustic model by using the recognition result of the character information, so that the acoustic model is not disturbed by the language model, and a more accurate evaluation result is obtained.

[0100] Referring to Figure 8 , Figure 8 is a structural schematic diagram of a data device in an embodiment of the present application. In the embodiment, the data device 81 comprises a processor 82.

[0101] The processor 82 can also be referred to as a CPU (Central Processing Unit). The processor 82 can be an integrated circuit chip having a processing capability of signals. The processor 82 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application-Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 82 can also be any conventional processor.

[0102] The data device 81 can further comprise a memory (not shown in the figure) for storing instructions and data required for the operation of the processor 82.

[0103] The processor 82 is configured to execute instructions to implement the method provided by any embodiment of the acoustic model performance evaluation method and the acoustic recognition method, and any non-conflicting combination thereof.

[0104] Please refer to Figure 9 , Figure 9 is a structural diagram of a computer readable storage medium in the embodiments of the present application. The computer readable storage medium 91 stores instructions / program data 92, which, when executed, implement the method provided by any embodiment of the acoustic model performance evaluation method and the acoustic recognition method, and any non-conflicting combination thereof. The instructions / program data 92 can form a program file and be stored in the storage medium 91 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor executes all or part of the steps of the method of each embodiment of the present application. The storage medium 91 described above includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes, or a computer, a server, a mobile phone, a tablet, and other terminal devices.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of units is only a logical function division. In actual implementation, another division mode can be used, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0106] In addition, each functional unit in the embodiments of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0107] The above description is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method of acoustic model performance evaluation, the method comprising: The method comprises: acquiring a voice training set and a language decoding graph, the voice training set comprising at least one voice, and the language decoding graph comprising a pronunciation dictionary comprising a corresponding relationship table between single words and phonemes; inputting the voice training set into an acoustic model to extract sound state information; inputting the sound state information into the language decoding graph to extract a recognition result of the voice training set; performing performance evaluation on the acoustic model by using the recognition result, comprising: acquiring a number of multi-recognized words, a number of missed-recognized words and a number of mis-recognized words in the recognition result; calculating an average word error rate by using the number of multi-recognized words, the number of missed-recognized words, the number of mis-recognized words and a total number of words of the voice training set, and performing performance evaluation on the acoustic model according to the average word error rate to acquire a recognition accuracy of the acoustic model; the average word error rate and the recognition accuracy of the acoustic model are negatively correlated; wherein acquiring the language decoding graph comprises: performing single word splitting on a corpus sentence in a corpus to obtain a corpus single word set; constructing a corresponding relationship table between corpus single words and phonemes to obtain the pronunciation dictionary; and combining the pronunciation dictionary with a language model, and making the probability of occurrence of each word in the language model the same to obtain the language decoding graph.

2. The acoustic model performance evaluation method of claim 1, wherein, The method comprises: removing duplicate corpus single words in the corpus single word set; constructing the pronunciation dictionary by using the corpus single words, phonemes of the corpus single words, silence words and phonemes of the silence words, out-of-set words and phonemes of the out-of-set words.

3. The acoustic model performance evaluation method of claim 1, wherein, training the language model by using the corpus single word set, wherein the weights of the corpus single words in the corpus single word set are the same, so that the probability of occurrence of each word in the language model is the same.

4. The acoustic model performance evaluation method of claim 1, wherein, The method further comprises: acquiring a context-related acoustic transcription machine; combining the acoustic transcription machine, the pronunciation dictionary and the language model to obtain the language decoding graph.

5. The acoustic model performance evaluation method of claim 1, wherein, The pronunciation dictionary and the language model are both in the form of a finite state transcription machine.

6. The acoustic model performance evaluation method of claim 1, wherein, The method comprises: performing frame splitting, windowing and pre-emphasis processing on the voice training set; extracting acoustic features of the voice training set by using the acoustic model to obtain the sound state information, wherein the sound state information comprises at least one acoustic feature or fused features of multiple acoustic features.

7. The acoustic model performance evaluation method of claim 4, wherein, The method comprises: acquiring context state information of the sound state information by using the context-related acoustic transcription machine; converting the context state information of the sound state information into a recognition result of text information by using the pronunciation dictionary and the language model.

8. The acoustic model performance evaluation method of claim 1, wherein, The method comprises: counting the number of words in the recognition result whose characters are different from corresponding characters of the voice training set and whose phonemes are different from corresponding phonemes of the voice training set as the number of mis-recognized words.

9. An acoustic recognition method, characterized by, The method comprises: acquiring a voice; acoustic model to identify sound state information of the speech; obtain literal information of the sound state information by using a language decoding graph, the language decoding graph comprising a pronunciation dictionary, the pronunciation dictionary comprising a corresponding relationship table between a single word and a phoneme; wherein, before the acoustic model is used to identify the sound state information of the speech, the acoustic model performance evaluation method of any one of claims 1-8 is used to evaluate the acoustic model; misrecognized literals in the evaluation process are obtained, the misrecognized literals comprising multi-recognition literals, missed recognition literals and wrong recognition literals; weights in a language model in the language decoding graph are adjusted based on a misrecognition rate of the misrecognized literals; and the speech is recognized by using the evaluated acoustic model and the adjusted language decoding graph.

10. The acoustic identification method of claim 9, wherein, The acoustic model is a model with the highest recognition accuracy.

11. The acoustic identification method of claim 9, wherein, The adjusting the weights in the language model in the language decoding graph based on the misrecognition rate of the misrecognized literals comprises: sorting the misrecognized literals and selecting a preset number of literals with the highest misrecognition rate; increasing the weights of the literals in the language model.

12. A data device, comprising: The processor is configured to execute instructions to implement the acoustic model performance evaluation method of any one of claims 1-8 or the acoustic recognition method of any one of claims 9-11.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store instructions / program data capable of being executed to implement the acoustic model performance evaluation method of any one of claims 1-8 or the acoustic recognition method of any one of claims 9-11.

Images