Audio evaluation method and device, and non-transient storage medium

Abstract

An audio evaluation method and device, and a non-transient storage medium. The audio evaluation method includes: acquiring audio data and a reference text including a first text; performing a first decoding operation on the audio data based on a first pronunciation dictionary model and a first language model to obtain a first decoded text; determining a first score based on the first decoded text and the reference text; determining a second pronunciation dictionary model based on the reference text and the first pronunciation dictionary model; determining a second language model based on the reference text, wherein the second language model is obtained by training based on the reference text; performing a second decoding operation on the audio data based on the second pronunciation dictionary model and the second language model to obtain a second decoded text and a corresponding relationship between the audio data and the second decoded text; determining a second score based on the first text, the second decoded text, and the above-mentioned corresponding relationship; and determining a final score of the audio data based on the first score and the second score.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an audio evaluation method, an audio evaluation device, and a non-transitory storage medium. Background Art

[0002] Oral communication is an important form of interpersonal communication and plays an important role in people's real life. With the continuous development of social economy, people have put forward higher and higher requirements for the efficiency of oral learning and the objectivity, fairness and large-scale testing of oral evaluation. Traditional manual oral proficiency assessment methods often cannot guarantee the uniformity of scoring standards due to individual differences among evaluators; in addition, manual assessment methods are not suitable for large-scale oral tests because they require a lot of manpower, material and financial support.

[0003] As speech technology continues to mature, its application in various fields is becoming more and more extensive. Oral assessment is one of the earliest application fields of speech technology. More and more oral teachers and users are using this oral assessment technology to teach and learn oral English. Summary of the invention

[0004] At least one embodiment of the present disclosure provides an audio evaluation method, comprising: acquiring audio data and a reference text, wherein the reference text comprises a first text; performing a first decoding operation on the audio data based on a first pronunciation dictionary model and a first language model to obtain a first decoded text; determining a first score according to the first decoded text and the reference text; determining a second pronunciation dictionary model according to the reference text and the first pronunciation dictionary model; determining a second language model according to the reference text, wherein the second language model is obtained by training based on the reference text; performing a second decoding operation on the audio data based on the second pronunciation dictionary model and the second language model to obtain a second decoded text and a corresponding relationship between the audio data and the second decoded text; determining a second score according to the first text, the second decoded text and the corresponding relationship between the audio data and the second decoded text; and determining a final score of the audio data according to the first score and the second score.

[0005] For example, in the audio evaluation method provided in some embodiments of the present disclosure, the second pronunciation dictionary model is determined according to the reference text and the first pronunciation dictionary model, including: in response to any word in the reference text not appearing in the first pronunciation dictionary model, generating the pronunciation of the any word based on the any word, and adding the any word and the pronunciation of the any word to the first pronunciation dictionary model to obtain the second pronunciation dictionary model; and in response to all words in the reference text appearing in the first pronunciation dictionary model, using the first pronunciation dictionary model as the second pronunciation dictionary model.

[0006] For example, in the audio evaluation method provided in some embodiments of the present disclosure, generating the pronunciation of any word based on any word includes: processing any word with a grapheme-to-phoneme conversion model to generate the pronunciation of any word.

[0007] For example, in the audio evaluation method provided in some embodiments of the present disclosure, based on the first pronunciation dictionary model and the first language model, the first decoding operation is performed on the audio data to obtain the first decoded text, including: constructing a first weighted finite state converter decoding graph based on the acoustic model, the context-related phonetic model, the first pronunciation dictionary model and the first language model; and based on the first weighted finite state converter decoding graph, using the Viterbi algorithm to perform the first decoding operation on the audio data to obtain the first decoded text.

[0008] For example, in the audio evaluation method provided in some embodiments of the present disclosure, based on the second pronunciation dictionary model and the second language model, the second decoding operation is performed on the audio data to obtain the second decoded text and the corresponding relationship between the audio data and the second decoded text, including: constructing a second weighted finite state converter decoding graph based on the acoustic model, the context-related phonetic model, the second pronunciation dictionary model and the second language model; and based on the second weighted finite state converter decoding graph, using the Viterbi algorithm to perform the second decoding operation on the audio data to obtain the second decoded text and the corresponding relationship between the audio data and the second decoded text.

[0009] For example, in the audio evaluation method provided by some embodiments of the present disclosure, the acoustic model includes a chain model based on a time-delay neural network or a Gaussian mixture model-a hidden Markov model.

[0010] For example, in the audio evaluation method provided by some embodiments of the present disclosure, the second language model includes a unigram language model.

[0011] For example, in the audio evaluation method provided in some embodiments of the present disclosure, the first score is determined based on the first decoded text and the reference text, including: determining the overlap and the longest common subsequence between the first decoded text and the reference text; and obtaining the first score based on the overlap and the longest common subsequence.

[0012] For example, in the audio evaluation method provided in some embodiments of the present disclosure, the second score is determined according to the first text, the second decoded text, and the correspondence between the audio data and the second decoded text, including: determining a second text in the second decoded text corresponding to the first text; determining an audio segment in the audio data corresponding to the second text based on the correspondence between the audio data and the second decoded text; and determining the second score based on the first text and the audio segment corresponding to the second text.

[0013] For example, in the audio evaluation method provided in some embodiments of the present disclosure, the first text includes at least one text segment, and the second score is determined based on the first text and the audio segment corresponding to the second text, including: determining the audio sub-segment corresponding to each word in each text segment in the at least one text segment based on the first text and the audio segment corresponding to the second text; determining the word score of each word in each text segment according to the audio sub-segment corresponding to each word in the each text segment based on a pronunciation accuracy algorithm, and taking the average of the word scores of all words in each text segment as the segment score of each text segment; and determining the second score according to the segment score of the at least one text segment.

[0014] For example, in the audio evaluation method provided in some embodiments of the present disclosure, determining the final score of the audio data according to the first score and the second score includes: obtaining a first weight corresponding to the first score and a second weight corresponding to the second score; and determining the final score according to the first score, the first weight, the second score, and the second weight, wherein the final score is expressed as:

[0015] Score_Final=W1*Score_1+W2*Score2,

[0016] Among them, Score_Final represents the final score, Score_1 represents the first score, Score2 represents the second score, W1 represents the first weight, W2 represents the second weight, and W1+W2=1.

[0017] For example, in the audio evaluation method provided in some embodiments of the present disclosure, the value range of the first weight W1 is [0.3, 0.5].

[0018] For example, in the audio evaluation method provided by some embodiments of the present disclosure, the first text includes at least one of a number, a symbol unit, and a foreign word.

[0019] For example, in the audio evaluation method provided in some embodiments of the present disclosure, the audio data includes voice data for answering a test question, the reference text includes at least one reference answer text corresponding to the test question, and each reference answer text includes the first text.

[0020] At least one embodiment of the present disclosure further provides an audio evaluation device, comprising: a memory for non-temporarily storing computer-readable instructions; and a processor for executing the computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the audio evaluation method provided by any embodiment of the present disclosure is executed.

[0021] At least one embodiment of the present disclosure further provides a non-transitory storage medium that non-temporarily stores computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the audio evaluation method provided by any embodiment of the present disclosure are executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0023] Figure 1 A flowchart of an audio evaluation method provided for at least one embodiment of the present disclosure;

[0024] Figure 2 A schematic block diagram of an audio evaluation device provided for at least one embodiment of the present disclosure; and

[0025] Figure 3 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0027] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0028] The present disclosure is described below through several specific embodiments. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of known functions and known components. When any component of the embodiments of the present disclosure appears in more than one figure, the component is represented by the same or similar reference numeral in each figure.

[0029] Semi-open questions are a common type of question in oral assessment. This type of semi-open question refers to a test item in which the test system plays prompts such as images, videos or short texts, and requires users to answer relevant questions or orally repeat the played content based on these prompts. For example, in the oral assessment of semi-open questions, the user's voice is converted into text by a speech recognition engine based on automatic speech recognition (ASR) technology, and then the text is semantically matched with the pre-set reference answer to obtain the final score of the user's voice.

[0030] For example, a general speech recognition engine may include a general acoustic model, a general context-dependent phone model, a general pronunciation lexicon model, and a general language model. For example, the acoustic model may be used to recognize a sound signal (e.g., audio data) into acoustic units (e.g., context-dependent phonemes, such as triphones). For example, the speech recognition engine may first frame the audio data, extract feature information of each frame, and then input the extracted feature information into the acoustic model. For example, the acoustic model may process the above feature information (including determining the Hidden Markov Model (HMM) state corresponding to each frame, determining the transition process, etc.) to obtain a sequence of context-dependent phonemes (e.g., a triphone sequence). For example, the context-dependent phone model may process the above sequence of context-dependent phonemes to obtain a sequence of phonemes (e.g., a monophone). For example, the pronunciation lexicon model includes a vocabulary set (referred to as a word list) and its pronunciation that the speech recognition engine can process, and may be used to convert the above phoneme sequence into a word sequence. For example, a language model can be used to screen the above word sequence to obtain a grammatical sentence (eg, a string of words consisting of multiple words).

[0031] In the evaluation process of semi-open questions, when there are special words in the reference answer, even if the user correctly answers the special word, the general speech recognition engine may not be able to recognize it, resulting in a low score (especially when the special word is a keyword in the reference answer). For example, special words include but are not limited to unregistered words (OutOf Vocabulary, OOV, also known as "out-of-vocabulary words") and uncommon words. For example, unregistered words refer to words that do not appear in the vocabulary used by the speech recognition engine, and uncommon words refer to words that appear very rarely in the corpus used to build the language model.

[0032] Take the test question "What do people usually eat on Dragon Boat Festival?" and the reference answer "Zongzi" or "people usually eat zongzi" (where the keyword is "zongzi") as an example. On the one hand, since the vocabulary in the general speech recognition engine cannot cover all words, the word "zongzi" (relative to English, it is a foreign word) is likely to be an unregistered word (i.e., it does not appear in the vocabulary), so the general speech recognition engine cannot recognize it (for example, it may be mistakenly recognized as "zone z"); on the other hand, even if the word "zongzi" exists in the vocabulary, due to the influence of factors such as the general language model (for example, the word "zongzi" is likely to be an uncommon word), the general speech recognition engine may not be able to recognize it. Therefore, in the oral evaluation process of semi-open questions based on the general speech recognition engine, if the reference answer includes a keyword that the general speech recognition engine cannot recognize, even if the user correctly answers the keyword, the score may still be low.

[0033] At least one embodiment of the present disclosure provides an audio evaluation method. The audio evaluation method includes: obtaining audio data and a reference text, wherein the reference text includes a first text; performing a first decoding operation on the audio data based on a first pronunciation dictionary model and a first language model to obtain a first decoded text; determining a first score based on the first decoded text and the reference text; determining a second pronunciation dictionary model based on the reference text and the first pronunciation dictionary model; determining a second language model based on the reference text, wherein the second language model is obtained by training based on the reference text; performing a second decoding operation on the audio data based on the second pronunciation dictionary model and the second language model to obtain a second decoded text and a corresponding relationship between the audio data and the second decoded text; determining a second score based on the first text, the second decoded text, and the corresponding relationship between the audio data and the second decoded text; and determining a final score of the audio data based on the first score and the second score.

[0034] Some embodiments of the present disclosure also provide an audio evaluation device and a non-transitory storage medium corresponding to the above-mentioned audio evaluation method.

[0035] The audio evaluation method provided by the embodiments of the present disclosure can determine a first score of the audio data based on the first pronunciation dictionary model and the first language model in combination with a reference text, and at the same time determine a second pronunciation dictionary model and a second language model based on the reference text (for example, a reference answer text), and on this basis determine a second score of the audio data based on the first text in the reference text (for example, keywords in the reference answer text), so that the final score of the audio data can be determined in combination with the first score and the second score, thereby avoiding the problem of directly using the first score as the final score of the audio data, which may result in a low final score (because there may be a problem of not being able to recognize the first text in the process of determining the first score of the audio data), and can provide a more objective, more reasonable, and more accurate evaluation result for the evaluation of audio data (for example, oral evaluation of semi-open questions), and has high practicality.

[0036] Some embodiments and examples of the present disclosure are described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementations described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure.

[0037] Figure 1 A flowchart of an audio evaluation method provided for at least one embodiment of the present disclosure. For example, the audio evaluation method can be applied to a computing device, and the computing device includes any electronic device with a computing function, such as a smart phone, a laptop computer, a tablet computer, a desktop computer, a server, etc., and the embodiments of the present disclosure are not limited to this. For example, the computing device has a central processing unit (CPU) or a graphics processing unit (GPU), and the computing device also includes a memory. The memory is, for example, a non-volatile memory (such as a read-only memory (ROM)), on which the code of the operating system is stored. For example, the memory also stores codes or instructions, and by running these codes or instructions, the audio evaluation method provided by the embodiments of the present disclosure can be implemented.

[0038] For example, Figure 1 As shown, the audio evaluation method includes steps S10 to S80.

[0039] Step S10: Acquire audio data and reference text, wherein the reference text includes a first text.

[0040] For example, in some embodiments, Figure 1 The audio evaluation method shown can be applied to oral evaluation of semi-open questions, but is not limited thereto. Figure 1The audio evaluation method shown is applied to oral evaluation of semi-open questions for illustration, but should not be regarded as a limitation of the present disclosure.

[0041] For example, the audio data may include voice data of answering a test question, for example, audio data of a user answering a semi-open test question; for example, the reference text may include at least one reference answer text corresponding to the test question, for example, each reference answer text may include a first text. For example, the first text may include one or more keywords (or key phrases) in the reference answer text.

[0042] For example, in a specific example, the test question is “What do people usually eat on DragonBoat Festival?”, the audio data may include voice data of the user answering the test question, the reference text may include the reference answer text “Zongzi” or / and the reference answer text “people usually eat zongzi”, and the first text may include the keyword “zongzi”.

[0043] For example, in some embodiments, Figure 1 The audio evaluation method shown can be executed locally by, for example, a client. In this case, the reference text can be a reference answer text pre-stored in the client, or a reference answer text received by the client from the server, and the embodiments of the present disclosure are not limited to this; the audio data can include speech collected by an audio collection device of the client, but is not limited thereto. For example, the audio data and the reference text can also be obtained by the client from the network.

[0044] For example, the client includes but is not limited to a smart phone, a tablet computer, a personal computer, a personal digital assistant (PDA), a wearable device, a head-mounted display device, etc. For example, the audio acquisition device includes but is not limited to a built-in or external microphone of the client. For example, the audio data can be recorded in real time or in advance, and the embodiments of the present disclosure are not limited to this.

[0045] For example, in some other embodiments, Figure 1 The audio evaluation method shown can also be performed remotely, for example, by a server. In this case, the server can receive the audio data uploaded by the user through the client, and then perform the audio evaluation process based on the reference text pre-stored in the server, and return the evaluation result to the client for the user's reference; of course, the reference text may not be pre-stored in the server, but uploaded to the server by the user through the client together with the audio data.

[0046] For example, the language of the reference text may be one of English, French, German, Spanish, Chinese, Japanese, Korean, etc., and embodiments of the present disclosure include but are not limited to this. For example, when the language of the reference text is Chinese, Japanese, Korean, etc., "a word" in the embodiments of the present disclosure may be correspondingly understood as "a character" (e.g., Chinese characters in Chinese, etc.). For example, the language used in the audio data is generally consistent with the language of the reference text, and embodiments of the present disclosure include but are not limited to this.

[0047] For example, the reference text is usually a sentence or a phrase, and each sentence or phrase includes several words. The embodiments of the present disclosure include but are not limited to the above.

[0048] Step S20: Based on the first pronunciation dictionary model and the first language model, a first decoding operation is performed on the audio data to obtain a first decoded text.

[0049] For example, in some embodiments, step S20 may include the following steps S21 to S22.

[0050] Step S21: constructing a first weighted finite state converter decoding graph based on the acoustic model, the context-dependent phonetic model, the first pronunciation dictionary model and the first language model;

[0051] Step S22: Based on the first weighted finite state converter decoding graph, use the Viterbi algorithm to perform a first decoding operation on the audio data to obtain a first decoded text.

[0052] For example, the acoustic model, context-dependent phonetic model, first pronunciation dictionary model and first language model in step S21 can respectively adopt a general acoustic model, a general context-dependent phonetic model, a general pronunciation dictionary model and a general language model (that is, a general speech recognition engine can be used to perform the first decoding operation in step S20), and the embodiments of the present disclosure are not limited to this. The specific details of each of the above four general models can refer to the above-mentioned related descriptions and related technologies in the field of natural language processing, and will not be repeated here.

[0053] For example, the acoustic model can usually be modeled using a Gaussian Mixture Model (GMM)-Hidden Markov Model (HMM) architecture (GMM-HMM), a Deep Neural Network (DNN)-Hidden Markov Model architecture (DNN-HMM), or a Chain model architecture based on a Time-Delay Neural Network (DNN). It should be noted that the embodiments of the present disclosure include but are not limited to this.

[0054] For example, the context-dependent phonetic model can be constructed using a clustering algorithm (e.g., a data-driven clustering algorithm or a decision tree-based clustering algorithm, etc.), and the embodiments of the present disclosure include but are not limited to this. For example, the specific technical details of using a clustering algorithm to construct a context-dependent phonetic model can refer to the relevant technology in the field of natural language processing, and will not be repeated here.

[0055] For example, the first pronunciation dictionary model (e.g., a general pronunciation dictionary model) may include a vocabulary set and its pronunciation that can be processed by a general speech recognition engine. For example, the specific technical details of constructing the first pronunciation dictionary model may refer to the relevant technology in the field of natural language processing, and will not be repeated here.

[0056] For example, the first language model (e.g., a general language model) may be a language model based on statistics or a language model based on a neural network, etc., and the embodiments of the present disclosure include but are not limited to the same. For example, the language model based on statistics includes an N-gram language model, such as an N-gram language model including but not limited to the commonly used ternary (N=3) language model and bigram (N=2) language model, etc. For example, the N-gram language model is a probability-based discriminant model, the input of the N-gram language model is a sequential sequence of N words, and the output of the N-gram language model is the joint probability of these N words. For example, the specific technical details of the N-gram language model can refer to the relevant technologies in the field of natural language processing, which will not be repeated here.

[0057] For example, in a language model based on a neural network, the neural network includes but is not limited to a recursive neural network (RNN), a long short-term memory network (LSTM), or a bidirectional long short-term memory network (Bi-directional Long-Short Term Memory, Bi-LSTM), etc. For example, the specific technical details of the language model based on a neural network can refer to the relevant technology in the field of natural language processing, and will not be repeated here.

[0058] For example, a weighted finite state transducer (WFST) provides a unified form to represent different knowledge sources of the current large-scale continuous speech recognition (LVCSR) system in the industry, and multiple WFSTs representing different knowledge sources can be integrated into one WFST through compound operations. For example, in some embodiments, the above-mentioned acoustic model, context-dependent phonetic model, first pronunciation dictionary model and first language model can all be compiled into WFSTs, which are respectively recorded as H.FST, C.FST, L.FST (for example, to distinguish from the WFST corresponding to the subsequent second pronunciation dictionary model, it can be recorded as L.FST_1) and G.FST (for example, to distinguish from the WFST corresponding to the subsequent second language model, it can be recorded as G.FST_1), and then can be integrated into a WFST in HCLG format (i.e., including H.FST, C.FST, L.FST_1 and G.FST_1) through compound operations, referred to as the "first WFST". For example, the decoding graph represented by the first WFST (i.e., the first weighted finite state converter decoding graph) can be transformed into an equivalent but more compact and efficient WFST decoding graph by removing redundant parts through various optimization operations to accelerate the decoding process. For example, the specific technical details of compiling and integrating the above-mentioned acoustic model, context-dependent phonetic model, first pronunciation dictionary model, and first language model into a WFST in HCLG format can refer to the relevant technologies in the field of natural language processing, and will not be repeated here.

[0059] For example, the Viterbi algorithm is based on the idea of ​​Token Passing for decoding. For example, based on the Viterbi algorithm, the optimal decoding path can be found in the first weighted finite state converter decoding graph to determine each word in the first decoded text, that is, to obtain the first decoded text. It should be understood that the time boundary of each word in the first decoded text can be obtained in the process of backtracking after the end of the Viterbi algorithm, so that according to the time boundary of each word in the first decoded text (for example, including the start time and the end time), the audio sub-segment corresponding to each word in the first decoded text can be determined. For example, the specific technical details of the Viterbi algorithm can refer to the relevant technology in the field of natural language processing, which will not be repeated here.

[0060] Step S30: Determine a first score according to the first decoded text and the reference text.

[0061] For example, in some embodiments, a common text evaluation method may be used to evaluate the first decoded text based on a reference text to determine a first score. For example, common text evaluation methods include but are not limited to bilingual evaluation understudy (BLEU) and recall-oriented understudy for gisting evaluation (ROUGE).

[0062] For example, BLEU can determine the degree of conformity of the first decoded text with the reference text by calculating the overlap between the first decoded text and the reference text (e.g., N-gram overlap). For example, in some embodiments, the overlap between the first decoded text and the reference text may include uni-gram (N=1) overlap, bi-gram (N=2) overlap, and tri-gram (N=3) overlap, etc. For example, the overlap between the first decoded text and the reference text can be obtained by weighted averaging multiple N-gram overlaps (e.g., N=1, 2, 3, etc.), and then, the first score can be determined based on the overlap between the first decoded text and the reference text.

[0063] For example, ROUGE may include ROUGE-L (where L refers to the longest common subsequence, LCS), etc. For example, ROUGE-L may determine the longest common subsequence between the first decoded text and the reference text to obtain a ROUGE-L score, and then, the ROUGE-L score may be converted into a first score.

[0064] For example, in some embodiments, a first score can be determined by combining multiple text evaluation methods and comprehensively considering multiple text evaluation indicators (e.g., the overlap between the first decoded text and the reference text and the ROUGE-L score, etc.). For example, the score determined according to the overlap between the first decoded text and the reference text and the score determined according to the ROUGE-L score can be weighted averaged to determine the first score. For example, the specific technical details of various text evaluation methods and text evaluation indicators can refer to the relevant technologies in the field of natural language processing (e.g., the relevant technologies of text generation evaluation indicators), which will not be repeated here.

[0065] It should be understood that the embodiments of the present disclosure do not limit the method for determining the first score; in practical applications, a commonly used scoring method or any reasonable scoring method may be used to perform the operation of determining the first score in step S30.

[0066] Step S40: Determine a second pronunciation dictionary model according to the reference text and the first pronunciation dictionary model.

[0067] For example, in some embodiments, step S40 may include the following steps S41 to S42.

[0068] Step S41: in response to any word in the reference text not appearing in the first pronunciation dictionary model, generating a pronunciation of the any word based on the any word, and adding the any word and the pronunciation of the any word to the first pronunciation dictionary model to obtain a second pronunciation dictionary model;

[0069] Step S42: In response to all words in the reference text appearing in the first pronunciation dictionary model, the first pronunciation dictionary model is used as the second pronunciation dictionary model.

[0070] It should be noted that since the vocabulary in the first pronunciation dictionary model (for example, a general pronunciation dictionary model) cannot include all numbers (for example, numbers representing time, years, telephone numbers, age, etc.) and combinations of numbers and words, it is also impossible to include all foreign words (especially newly emerging foreign words), and usually does not include all symbolic units (for example, currency units, temperature units, etc.), therefore, any word that does not appear in the first pronunciation dictionary model (i.e., unregistered words) is usually a number, a foreign word, a symbolic unit, etc.

[0071] For example, in some embodiments, the first text in the reference text (for example, keywords in the reference answer text) may include at least one of a number, a symbolic unit, and a foreign word. In this case, if the first score obtained based on steps S20 to S30 is directly used as the final score of the audio data, the final score may be low due to the problem that the first text may not be recognized in the process of determining the first score. Therefore, the audio evaluation method provided by the embodiment of the present disclosure also includes steps S40 to S70 specifically for scoring the first text in the reference text and step S80 for determining the final score in combination with the first score and the second score (for specific details, refer to the corresponding description of steps S40 to S80 in the present disclosure) to evaluate the audio data more objectively, more reasonably, and more accurately.

[0072] It should be understood that in the embodiments of the present disclosure, the unregistered words are relative to the first pronunciation dictionary model. It should also be understood that in the embodiments of the present disclosure, the foreign words are relative to the language of the reference text. For example, when the language of the reference text is English, "zongzi" (zongzi), "baozi" (baozi), etc. are foreign words.

[0073] For example, in some embodiments, a grapheme-to-phoneme (G2P) conversion model may be used to process any word (i.e., an unregistered word) to generate the pronunciation of any word. For example, in some examples, a G2P conversion model may be implemented using RNN and LSTM, and the embodiments of the present disclosure include but are not limited to this. For example, the specific technical details of the G2P conversion model may refer to the relevant technologies in the field of natural language processing, and will not be repeated here.

[0074] For example, in a specific example, taking the test question "What do people usually eat on Dragon Boat Festival?" and the reference texts "Zongzi" and "people usually eat zongzi" (the reference text includes two reference answer texts) as an example, if the word "zongzi" does not appear in the first pronunciation dictionary (that is, there are unregistered words), the pronunciation of the word "zongzi" can be generated through the G2P conversion model, and the word "Zongzi" and its pronunciation are added to the first pronunciation dictionary model to obtain the second pronunciation dictionary model; if all the words "zongzi", "people", "usually", and "eat" in the reference text appear in the first pronunciation dictionary model (that is, there are no unregistered words), the first pronunciation dictionary model can be directly used as the second pronunciation dictionary model.

[0075] It should be understood that when the second pronunciation dictionary model is determined based on step S40, step S41 and step S42 are performed alternatively according to the judgment condition.

[0076] It should be understood that step S40 can ensure that the second pronunciation dictionary model can be used to process unregistered words.

[0077] Step S50: determining a second language model according to the reference text, wherein the second language model is obtained by training based on the reference text.

[0078] For example, in some embodiments, a reference text may be used as a training corpus to train an N-gram language model, and the N-gram language model may be used as the second language model. For example, the N-gram language model includes but is not limited to commonly used unigram (N=1) language models, bigram (N=2) language models, and trigram (N=3) language models. Therefore, in some embodiments, the second language model may be a unigram language model. For example, in Figure 1When the audio evaluation method shown is applied to oral evaluation of semi-open questions, since the reference text (e.g., reference answer text) includes relatively few words, training the second language model as a unary language model can improve the accuracy of the second decoding operation in the subsequent step S60.

[0079] It should be noted that, unlike the first language model (e.g., a universal language model), the second language model corresponds to the reference text, that is, different reference texts correspond to different second language models. In the embodiment of the present disclosure, the reference text can be any text (e.g., it changes with the test questions). Therefore, in each execution of the audio evaluation method, it is usually necessary to train the second language model corresponding to the reference text obtained each time (i.e., the information of the previous reference text will not affect the language model of this time), thereby ensuring that each word in the second decoded text obtained by the subsequent second decoding operation comes from the reference text. It should be understood that step S50 can ensure that the second language model can be used to process unregistered words and uncommon words.

[0080] For example, in some embodiments, when executing Figure 1 Before the audio evaluation method shown in the figure, the reference text is known; in this case, the reference text can be used in advance (i.e., before executing Figure 1 The audio evaluation method shown in the figure is used before) to train the second language model, so when performing the operation of determining the second language model in step S50, it is only necessary to directly select the second language model corresponding to the reference text.

[0081] For example, in some other embodiments, when executing Figure 1 Before the audio evaluation method shown, the reference text is unknown; in this case, when the operation of determining the second language model in step S50 is performed, the second language model can be obtained by real-time training based on the reference text.

[0082] Step S60: Based on the second pronunciation dictionary model and the second language model, a second decoding operation is performed on the audio data to obtain a second decoded text and a correspondence between the audio data and the second decoded text.

[0083] For example, in some embodiments, step S60 may include the following steps S61 to S62.

[0084] Step S61: constructing a second weighted finite state transducer decoding graph based on the acoustic model, the context-dependent phonetic model, the second pronunciation dictionary model and the second language model;

[0085] Step S62: Based on the second weighted finite state converter decoding graph, use the Viterbi algorithm to perform a second decoding operation on the audio data to obtain a second decoded text and a corresponding relationship between the audio data and the second decoded text.

[0086] For example, in some embodiments, the acoustic model in step S61 is generally the same model as the acoustic model in step S21; the context-dependent phonetic model in step S61 is generally the same model as the context-dependent phonetic model in step S21; the second pronunciation dictionary model in step S61 may be the same as or different from the first pronunciation dictionary model in step S21, for example, refer to the relevant description of the aforementioned step S40; the second language model in step S61 is generally different from the first language model in step S21, for example, refer to the relevant description of the aforementioned step S50.

[0087] For example, in some embodiments, the acoustic model, context-dependent phonetic model, second pronunciation dictionary model and second language model can all be compiled into WFST, which are respectively recorded as H.FST, C.FST, L.FST (for example, to distinguish from the WFST corresponding to the aforementioned first pronunciation dictionary model, it can be recorded as L.FST_2) and G.FST (for example, to distinguish from the WFST corresponding to the aforementioned first language model, it can be recorded as G.FST_2), and then can be integrated into a WFST in HCLG format (i.e., including H.FST, C.FST, L.FST_2 and G.FST_2) through compound operations, referred to as "second WFST". The decoding graph represented by the second WFST (i.e., the second weighted finite state converter decoding graph) can be transformed into an equivalent but more compact and efficient WFST decoding graph through various optimization operations to remove the redundant parts therein to accelerate the decoding process. For example, the specific technical details of compiling and integrating the above-mentioned acoustic model, context-dependent phonetic model, second pronunciation dictionary model, and second language model into a WFST in HCLG format can refer to the relevant technologies in the field of natural language processing, which will not be repeated here.

[0088] For example, based on the Viterbi algorithm, the optimal decoding path can be found in the second weighted finite state converter decoding graph, so as to determine each word in the second decoded text, that is, to obtain the second decoded text; for another example, the time boundary of each word in the second decoded text can be obtained in the process of backtracking after the end of the Viterbi algorithm, so that according to the time boundary of each word in the second decoded text (for example, including the start time and the end time), the audio sub-segment corresponding to each word in the second decoded text can be determined, that is, the corresponding relationship between the audio data and the second decoded text is obtained. For example, the specific technical details of the Viterbi algorithm can refer to the relevant technology in the field of natural language processing, which will not be repeated here.

[0089] Step S70: Determine a second score based on the first text, the second decoded text, and the correspondence between the audio data and the second decoded text.

[0090] For example, in some embodiments, step S70 may include the following steps S71 to S73.

[0091] Step S71: Determine a second text in the second decoded text that corresponds to the first text.

[0092] For example, in some embodiments, each word in the second decoded text comes from the reference text. In other words, if a set is constructed with each word in the reference text as an element, each word in the second decoded text is an element in the set.

[0093] For example, in some embodiments, the first text may include at least one text segment. For example, when the first text includes a text segment, the portion of the second decoded text that is identical to the text segment may be extracted as the second text; for another example, when the first text includes multiple text segments, the portions of the second decoded text that are identical to each text segment may be extracted as the second text. For example, in some embodiments, the second text generally includes at least part of the text segment in the first text. For example, in some embodiments, the second text may include all the text segments in the first text, that is, the second text is identical to the first text.

[0094] Step S72: Based on the correspondence between the audio data and the second decoded text, determine the audio segment in the audio data corresponding to the second text.

[0095] For example, according to the correspondence between the audio data obtained in step S60 and the second decoded text, the audio segment corresponding to each word in the second decoded text can be determined. For example, in some embodiments, the audio segment corresponding to the second text can be determined according to the words included in the second text and the audio segment corresponding to each word in the second decoded text.

[0096] Step S73: Determine a second score based on the first text and the audio segment corresponding to the second text.

[0097] For example, in some embodiments, the first text may include at least one text segment, in which case step S73 may include: determining an audio sub-segment corresponding to each word in each text segment in the at least one text segment based on the first text and an audio segment corresponding to the second text; determining a word score for each word based on the audio sub-segment corresponding to each word in each text segment based on a Goodness Of Pronunciation (GOP) algorithm, and taking the average of the word scores of all words in each text segment as the segment score of each text segment; and determining a second score based on the segment score of the at least one text segment.

[0098] For example, in some examples, the pronunciation accuracy algorithm may include: first extracting feature information corresponding to the audio sub-segment of each word, for example, the feature information includes but is not limited to Mel-scale Frequency Cepstral Coefficients (MFCC), etc.; then inputting the feature information into a pre-trained phoneme evaluation model for phoneme evaluation to obtain the GOP value of each phoneme in the above audio sub-segment; finally, based on the GOP value of each phoneme, determining the phoneme score of each phoneme in the above audio sub-segment, and then determining the score of each word. On this basis, the segment score of each text segment can be further determined, and then the second score can be determined. For example, the specific technical details of the pronunciation accuracy algorithm can refer to the relevant technology in the field of natural language processing, which will not be repeated here.

[0099] For example, determining the second score based on the fragment score of at least one text fragment may include: when the first text includes one text fragment, using the fragment score of the text fragment as the second score; when the first text includes multiple text fragments, using the average (for example, the arithmetic mean) or the weighted average of the fragment scores of the multiple text fragments as the second score.

[0100] Step S80: Determine a final score of the audio data according to the first score and the second score.

[0101] For example, in some embodiments, step S80 may include: obtaining a first weight corresponding to the first score and a second weight corresponding to the second score; and determining a final score according to the first score, the first weight, the second score, and the second weight. For example, in some examples, the final score may be expressed as:

[0102] Score_Final=W1*Score_1+W2*Score2,

[0103] Among them, Score_Final represents the final score, Score_1 represents the first score, Score2 represents the second score, W1 represents the first weight, W2 represents the second weight, and W1+W2=1.

[0104] For example, in some examples, the first score Score_1 obtained in step S30 and the second score Score_2 obtained in step S70 have the same value range; for example, the first score Score_1 and the second score Score_2 can both have a value range of [0, 100], and accordingly, the final score Score_Final also has a value range of [0, 100]. It should be noted that the embodiments of the present disclosure include but are not limited to this.

[0105] For example, in some examples, the value range of the first weight W1 may be [0.3, 0.5], and the embodiments of the present disclosure include but are not limited to this. For example, the value of the first weight W1 may be set to 0.3, 0.35, 0.4, 0.45, 0.5, etc. For example, the value of the first weight W1 may also be set according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0106] It should be understood that Figure 1 When the audio evaluation method shown is applied to oral evaluation of semi-open questions, if there are unknown words and / or uncommon words in the reference text (e.g., the reference answer text), especially when the unknown words and / or uncommon words are the first text in the reference text (e.g., the keywords in the reference answer text), the first score (which can be obtained based on steps S20 to S30) cannot usually be directly used as the final score, because there may be a problem of not being able to recognize the first text in the process of determining the first score. Therefore, if the first score is directly used as the final score, the final score may be too low. In order to deal with the above problems, Figure 1 The audio evaluation method shown also specifically scores the first text in the reference text (i.e., the keywords in the reference answer text) (refer to steps S40 to S70) to obtain a second score, and combines the first score and the second score to determine the final score of the audio data (refer to step S80), thereby providing a more objective, more reasonable, and more accurate evaluation result for the audio data, which has higher practicality.

[0107] It should be noted that although the embodiments of the present disclosure are described using English text as an example, this should not be regarded as a limitation on the audio evaluation method provided by the embodiments of the present disclosure. The audio evaluation method provided by the embodiments of the present disclosure can be applied to the evaluation of semi-open questions in various languages ​​such as English, French, German, Spanish, Chinese, Japanese, and Korean.

[0108] It should be noted that, in the embodiments of the present disclosure, the process of the above-mentioned audio evaluation method may include more or fewer operations, and these operations may be performed sequentially or in parallel. Although the process of the audio evaluation method described above includes multiple operations that appear in a specific order, it should be clearly understood that the order of the multiple operations is not limited. The audio evaluation method described above can be performed once or multiple times according to predetermined conditions.

[0109] At least one embodiment of the present disclosure further provides an audio evaluation device. Figure 2 A schematic block diagram of an audio evaluation device provided by at least one embodiment of the present disclosure. Figure 2 As shown, the audio evaluation device 100 includes a memory 110 and a processor 120 .

[0110] For example, the memory 110 is used to non-temporarily store computer-readable instructions, and the processor 120 is used to execute the computer-readable instructions. When the computer-readable instructions are executed by the processor 120, the audio evaluation method provided by any embodiment of the present disclosure is executed.

[0111] For example, the memory 110 and the processor 120 may communicate with each other directly or indirectly. Figure 2 As shown, the audio evaluation device 100 may also include a system bus 130, and the memory 110 and the processor 120 may communicate with each other through the system bus 130. For example, the processor 120 may access the memory 110 through the system bus 130. For example, in other examples, the memory 110 and the processor 120 and other components may communicate through a network connection. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The network may include a local area network, the Internet, a telecommunication network, an Internet of Things based on the Internet and / or a telecommunication network, and / or any combination of the above networks. For example, a wired network may communicate by means of twisted pair, coaxial cable or optical fiber transmission, and for example, a wireless network may communicate by means of 3G / 4G / 5G mobile communication network, Bluetooth, Zigbee or WiFi. The present disclosure does not limit the type and function of the network.

[0112] For example, the processor 120 can control other components in the audio evaluation device to perform desired functions. The processor 120 can be a device with data processing capabilities and / or program execution capabilities such as a central processing unit (CPU), a tensor processing unit (TPU) or a graphics processing unit GPU. The central processing unit (CPU) can be an X86 or ARM architecture, etc. The GPU can be directly integrated into the motherboard separately, or built into the north bridge chip of the motherboard. The GPU can also be built into the central processing unit (CPU).

[0113] For example, the memory 110 may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc.

[0114] For example, one or more computer instructions may be stored on the memory 110, and the processor 120 may execute the computer instructions to implement various functions. Various applications and various data may also be stored in the computer-readable storage medium, such as audio data, reference text, first decoded text, second decoded text, first score, second score, final score, and various data used and / or generated by the application.

[0115] For example, when some computer instructions stored in the memory 110 are executed by the processor 120 , one or more steps in the audio evaluation method described above can be performed.

[0116] For example, Figure 2 As shown, the audio evaluation apparatus 100 may further include an input interface 140 that allows an external device to communicate with the audio evaluation apparatus 100. For example, the input interface 140 may be used to receive instructions from an external computer device, from a user, etc. The audio evaluation apparatus 100 may further include an output interface 150 that interconnects the audio evaluation apparatus 100 and one or more external devices. For example, the audio evaluation apparatus 100 may output an audio evaluation result, etc., through the output interface 150. The external device that communicates with the audio evaluation apparatus 100 through the input interface 140 and the output interface 150 may be included in an environment that provides any type of user interface with which a user may interact. Examples of user interface types include a graphical user interface, a natural user interface, etc. For example, a graphical user interface may accept input from a user using an input device such as a keyboard, a mouse, a remote controller, etc., and provide output on an output device such as a display. In addition, a natural user interface may enable a user to interact with the audio evaluation apparatus 100 in a manner that is not subject to constraints imposed by input devices such as a keyboard, a mouse, a remote controller, etc. In contrast, natural user interfaces may rely on speech recognition, touch and stylus recognition, gesture recognition on and near the screen, mid-air gestures, head and eye tracking, voice and semantics, vision, touch, gestures, and machine intelligence, among others.

[0117] For example, in some embodiments, the audio evaluation apparatus 100 further includes the audio acquisition device described in the embodiment of the audio evaluation method.

[0118] In addition, although the audio evaluation device 100 Figure 2 Although shown as a single system, it is understood that the audio evaluation device 100 may also be a distributed system and may also be arranged as a cloud facility (including a public cloud or a private cloud). Thus, for example, several devices may communicate via a network connection and may jointly perform the tasks described as being performed by the audio evaluation device 100. For example, in some embodiments, a test question of a semi-open question type may be obtained through a client, and then audio data of a user answering the test question may be collected and uploaded to a server; the server performs an audio evaluation process based on the received audio data and a reference text pre-stored in the server and returns the evaluation result (e.g., a final score) to the client to provide to the user.

[0119] For example, for a detailed description of the implementation process of the audio evaluation method, reference may be made to the relevant description in the above-mentioned embodiment of the audio evaluation method, and the repeated parts will not be repeated here.

[0120] For example, in some examples, the audio evaluation device may include but is not limited to a smart phone, a tablet computer, a personal computer, a personal digital assistant (PDA), a wearable device, a head-mounted display device, a server, etc.

[0121] It should be noted that the audio evaluation device provided in the embodiments of the present disclosure is exemplary rather than restrictive. According to actual application requirements, the audio evaluation device may also include other conventional components or structures. For example, to realize the necessary functions of the audio evaluation device, those skilled in the art may set other conventional components or structures according to specific application scenarios, and the embodiments of the present disclosure are not limited to this.

[0122] The technical effects of the audio evaluation device provided by the embodiments of the present disclosure can be referred to the corresponding description of the audio evaluation method in the above embodiments, which will not be repeated here.

[0123] At least one embodiment of the present disclosure also provides a non-transitory storage medium. Figure 3 A schematic diagram of a non-transient storage medium provided by an embodiment of the present disclosure. Figure 3 As shown, the non-transitory storage medium 200 non-transitory stores computer-readable instructions 201 , and when the non-transitory computer-readable instructions 201 are executed by a computer (including a processor), the instructions of the audio evaluation method provided by any embodiment of the present disclosure can be executed.

[0124] For example, one or more computer instructions may be stored on the non-transitory storage medium 200. Some of the computer instructions stored on the non-transitory storage medium 200 may be, for example, instructions for implementing one or more steps in the above-mentioned audio evaluation method.

[0125] For example, the non-transitory storage medium may include a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a compact disk read-only memory (CD-ROM), a flash memory, or any combination of the above storage media, or other applicable storage media.

[0126] The technical effects of the non-transitory storage medium provided by the embodiments of the present disclosure can be referred to the corresponding description of the audio evaluation method in the above embodiments, which will not be repeated here.

[0127] There are a few points to note about this disclosure:

[0128] (1) In the drawings of the embodiments of the present disclosure, only the structures related to the embodiments of the present disclosure are involved, and other structures can refer to the general design.

[0129] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.

[0130] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An audio evaluation method, comprising: Acquire audio data and reference text, wherein the reference text includes a first text; Based on the first pronunciation dictionary model and the first language model, performing a first decoding operation on the audio data to obtain a first decoded text; Determining a first score according to the first decoded text and the reference text; Determining a second pronunciation dictionary model according to the reference text and the first pronunciation dictionary model; Determining a second language model according to the reference text, wherein the second language model is trained based on the reference text; Based on the second pronunciation dictionary model and the second language model, performing a second decoding operation on the audio data to obtain a second decoded text and a corresponding relationship between the audio data and the second decoded text; Determining a second score according to the first text, the second decoded text, and the correspondence between the audio data and the second decoded text; and Determining a final score of the audio data according to the first score and the second score; The step of determining the second pronunciation dictionary model according to the reference text and the first pronunciation dictionary model includes: In response to any word in the reference text not appearing in the first pronunciation dictionary model, generating a pronunciation of the any word based on the any word, and adding the any word and the pronunciation of the any word to the first pronunciation dictionary model to obtain the second pronunciation dictionary model.

2. The audio evaluation method according to claim 1, wherein: Determining the second pronunciation dictionary model according to the reference text and the first pronunciation dictionary model includes: In response to all words in the reference text appearing in the first pronunciation dictionary model, the first pronunciation dictionary model is used as the second pronunciation dictionary model.

3. The audio evaluation method according to claim 1, wherein: Generating a pronunciation of any word based on any word comprises: The any word is processed using a grapheme-to-phoneme conversion model to generate a pronunciation of the any word.

4. The audio evaluation method according to any one of claims 1 to 3, wherein: Based on the first pronunciation dictionary model and the first language model, performing the first decoding operation on the audio data to obtain the first decoded text includes: constructing a first weighted finite state transducer decoding graph based on the acoustic model, the context-dependent phonetic submodel, the first pronunciation dictionary model and the first language model; and Based on the first weighted finite state converter decoding graph, the first decoding operation is performed on the audio data using the Viterbi algorithm to obtain the first decoded text.

5. The audio evaluation method according to claim 4, wherein: Based on the second pronunciation dictionary model and the second language model, performing the second decoding operation on the audio data to obtain the second decoded text and a corresponding relationship between the audio data and the second decoded text includes: constructing a second weighted finite state transducer decoding graph based on the acoustic model, the context-dependent phonetic submodel, the second pronunciation dictionary model and the second language model; and Based on the second weighted finite state converter decoding graph, the second decoding operation is performed on the audio data using the Viterbi algorithm to obtain the second decoded text and the corresponding relationship between the audio data and the second decoded text.

6. The audio evaluation method according to claim 4, wherein: The acoustic model includes a chain model based on a time-delay neural network or a Gaussian mixture model-a hidden Markov model.

7. The audio evaluation method according to any one of claims 1 to 3, wherein: The second language model comprises a unigram language model.

8. The audio evaluation method according to any one of claims 1 to 3, wherein: Determining the first score according to the first decoded text and the reference text includes: Determining the overlap and the longest common subsequence between the first decoded text and the reference text; and The first score is obtained based on the overlap degree and the longest common subsequence.

9. The audio evaluation method according to any one of claims 1 to 3, wherein: Determining the second score according to the first text, the second decoded text, and the correspondence between the audio data and the second decoded text includes: Determining a second text in the second decoded text corresponding to the first text; Based on the correspondence between the audio data and the second decoded text, determining an audio segment in the audio data corresponding to the second text; and The second score is determined based on the first text and an audio segment corresponding to the second text.

10. The audio evaluation method according to claim 9, wherein: The first text includes at least one text segment, Determining the second score based on the first text and an audio segment corresponding to the second text includes: determining, based on the first text and the audio segment corresponding to the second text, an audio sub-segment corresponding to each word in each text segment of the at least one text segment; Based on the pronunciation accuracy algorithm, determining a word score for each word in each text segment according to the audio subsegment corresponding to each word in each text segment, and taking an average of the word scores of all words in each text segment as the segment score of each text segment; and The second score is determined according to an average value of the segment scores of the at least one text segment.

11. The audio evaluation method according to any one of claims 1 to 3, wherein: Determining the final score of the audio data according to the first score and the second score includes: Obtaining a first weight corresponding to the first score and a second weight corresponding to the second score; and determining the final score according to the first score, the first weight, the second score and the second weight, The final score is expressed as: Score_Final=W1*Score_1+W2*Score2, Among them, Score_Final represents the final score, Score_1 represents the first score, Score2 represents the second score, W1 represents the first weight, W2 represents the second weight, and W1+W2=1.

12. The audio evaluation method according to claim 11, wherein: The value range of the first weight W1 is [0.3, 0.5].

13. The audio evaluation method according to any one of claims 1 to 3, wherein: The first text includes at least one of a number, a symbol unit, and a foreign word.

14. The audio evaluation method according to any one of claims 1 to 3, wherein: The audio data includes voice data for answering a test question, and the reference text includes at least one reference answer text corresponding to the test question, and each of the reference answer texts includes the first text.

15. An audio evaluation device, comprising: a memory for non-transitory storage of computer readable instructions; as well as A processor is used to execute the computer-readable instructions, wherein the computer-readable instructions are executed by the processor to perform the audio evaluation method according to any one of claims 1 to 14.

16. A non-transitory storage medium that non-temporarily stores computer-readable instructions, wherein: When the non-transitory computer-readable instructions are executed by a computer, the instructions of the audio evaluation method according to any one of claims 1 to 14 are executed.

Images