Method and apparatus for confidence evaluation of automatic speech recognition result, and electronic device

Abstract

This application relates to the field of artificial intelligence, and provides a method, apparatus, and electronic device for evaluating the confidence of automatic speech recognition results. The method includes: acquiring a first candidate translated text of an audio sample; constructing a target reward function based on the word error rate and a first confidence score of the first candidate translated text; wherein the first confidence score is generated using an initial confidence evaluation model; based on the target reward function, iteratively updating the network parameters of the initial confidence evaluation model using a reinforcement learning algorithm until the training termination condition is met, thereby obtaining a confidence evaluation model; and using the confidence evaluation model to determine a second confidence score of a second candidate translated text of the audio to be processed. The method, apparatus, and electronic device for evaluating the confidence of automatic speech recognition results provided in this application can accurately evaluate the results of automatic speech recognition.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and electronic device for evaluating the confidence level of automatic speech recognition results. Background Technology

[0002] In today's era of rapid digital information development, Automatic Speech Recognition (ASR) technology plays an increasingly important role as a key component of human-computer interaction and information processing. ASR technology aims to convert human speech into readable text. Due to its convenience and efficiency, it is widely used in numerous scenarios such as conferencing systems and human-computer interaction, greatly improving the efficiency of information acquisition and interaction. Currently, end-to-end technology frameworks have become the mainstream system in the field of speech recognition. This framework simplifies the complex process of multiple independent modules in traditional speech recognition systems, directly mapping the input speech signal to the output text. It demonstrates significant advantages in improving recognition efficiency and accuracy, driving the rapid development of speech recognition technology and resulting in substantial performance improvements.

[0003] However, in practical applications, existing speech recognition technologies still face many challenges and problems. Regarding confidence assessment, traditional methods primarily rely on the word-level softmax posterior probability output by the end-to-end Automatic Speech Recognition (ASR) model during the decoding phase, or employ sequence-level scoring functions, such as the Connectionist Temporal Classification prefix-score (CTC prefix-score) or the attention decoder beam-score, as confidence levels. However, the softmax result output by the end-to-end ASR model decoder suffers from overconfidence; a high confidence score may correspond to a high word error rate (WER), and the softmax probability does not match the true WER. Directly using the decoder output probability as the confidence assessment standard is not feasible. Furthermore, the training phase typically uses a cross-entropy objective function, while the inference phase uses WER as the evaluation metric. This mismatch between training and inference metrics further affects the accuracy and effectiveness of confidence assessment.

[0004] Therefore, how to accurately evaluate the results of automatic speech recognition is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for evaluating the confidence level of automatic speech recognition results, so as to accurately evaluate the results of automatic speech recognition.

[0006] In a first aspect, embodiments of this application provide a method for evaluating the confidence level of automatic speech recognition results, including: Obtain the first candidate translated text of the audio sample; construct a target reward function based on the word error rate and first confidence score of the first candidate translated text; wherein the first confidence score is generated using an initial confidence evaluation model; based on the target reward function, use a reinforcement learning algorithm to iteratively update the network parameters of the initial confidence evaluation model until the training termination condition is met to obtain a confidence evaluation model; use the confidence evaluation model to determine the second confidence score of the second candidate translated text of the audio to be processed.

[0007] In one embodiment, the target reward function includes a confidence alignment reward sub-item; constructing the target reward function based on the word error rate and the first confidence score of the first candidate translated text includes: obtaining a word error rate reward value based on the word error rate of the first candidate translated text; and constructing the confidence alignment reward sub-item based on the difference between the first confidence score of the first candidate translated text and the word error rate reward value.

[0008] In one embodiment, constructing the confidence alignment reward sub-item based on the difference between the first confidence score and the word error rate reward value of the first candidate translated text includes: Determine the first difference between the first confidence score and the word error rate reward value of the first candidate translated text; obtain the confidence alignment reward sub-item based on the value 1 and the first difference.

[0009] In one embodiment, obtaining the word error rate reward value based on the word error rate of the first candidate translated text includes: Based on the word error rate of the first candidate translated text, a word error rate reward sub-item is obtained; based on the keyword weight table and the indicator function of each word element in the first candidate translated text, a weighted word error rate reward sub-item is determined; wherein, the indicator function is a binary function used to indicate whether a word element has a translation error, and the keyword weight table records the keyword weight of the word element; based on the word error rate reward sub-item and / or the weighted word error rate reward sub-item, the word error rate reward value is obtained.

[0010] In one embodiment, obtaining the word error rate reward item based on the word error rate of the first candidate translated text includes: obtaining the word error rate reward item based on the difference between the value 1 and the word error rate of the first candidate translated text.

[0011] In one embodiment, determining the weighted word error rate reward item for the first candidate translated text based on the keyword weight table and the indicator function of each word element in the first candidate translated text includes: determining the keyword weight of each word element in the first candidate translated text based on the keyword weight table; performing a weighted summation of the indicator functions of each word element based on the keyword weight of each word element in the first candidate translated text to obtain a weighted sum of word error rates; normalizing the weighted sum of word error rates based on the sum of the keyword weights of each word element in the first candidate translated text to obtain a weighted word error rate; and obtaining the weighted word error rate reward item based on the weighted word error rate.

[0012] In one embodiment, obtaining the weighted word error rate reward item based on the weighted word error rate includes: obtaining the weighted word error rate reward item based on the difference between the value 1 and the weighted word error rate.

[0013] In one embodiment, the indicator function for each lexical unit is determined in the following manner: Based on the labeled text of the audio sample, determine whether each word in the first candidate translated text has a translation error; for each word in the first candidate translated text, if it is determined that the word has a translation error, determine the indicator function of the word as a first value; or, if it is determined that the word is translated correctly, determine the indicator function of the word as a second value.

[0014] In one embodiment, the target reward function further includes the word error rate reward sub-item and the weighted word error rate reward sub-item.

[0015] In one embodiment, obtaining the first candidate translated text of the audio sample includes: The audio sample is processed using an acoustic encoder to obtain frame-level acoustic features of the audio sample; wherein the frame-level acoustic features include feature vectors of each audio frame of the audio sample; and the first candidate translated text is obtained using an acoustic decoder based on the frame-level acoustic features.

[0016] In one embodiment, obtaining the first candidate translated text based on the frame-level acoustic features using an acoustic decoder includes: processing the frame-level acoustic features using an acoustic decoder to obtain the posterior probability space of the translated text; and performing a beam search on the posterior probability space of the translated text to obtain the first candidate translated text.

[0017] In one embodiment, the first confidence score of the first candidate translated text is obtained by: using the initial confidence evaluation model, generating the first confidence score of the first candidate translated text based on the frame-level acoustic features of the audio sample and the first candidate translated text.

[0018] In one embodiment, generating a first confidence score for the first candidate translated text based on the frame-level acoustic features of the audio sample and the first candidate translated text using the initial confidence evaluation model includes: obtaining a first confidence score for the first candidate translated text based on the frame-level acoustic features of the training audio, the first candidate translated text, and a keyword weight table using the initial confidence evaluation model; wherein the keyword weight table records the keyword weights of keyword elements.

[0019] In one embodiment, the audio sample relates to a target technical field, and the keyword elements include specialized terms within that target technical field.

[0020] In one embodiment, the keyword element further includes high-frequency word elements; wherein the high-frequency word elements are extracted from the corpus based on word frequency and inverse document frequency.

[0021] Secondly, this application provides a method for evaluating the confidence level of automatic speech recognition results, including: The process involves: acquiring the audio to be processed; using a speech recognition model to obtain frame-level acoustic features and a second candidate translated text for the audio to be processed; wherein the frame-level acoustic features include feature vectors for each audio frame of the audio to be processed; using a confidence evaluation model to obtain a second confidence score for the second candidate translated text based on the frame-level acoustic features and the second candidate translated text; wherein the confidence evaluation model is obtained by reinforcement learning of an initial confidence evaluation model based on a target reward function, the target reward function being constructed based on the word error rate and first confidence score of the first candidate translated text of the audio sample, and the first confidence score of the audio sample being generated using the initial confidence evaluation model.

[0022] In one embodiment, the speech recognition model includes an acoustic encoder and an acoustic decoder; the step of using the speech recognition model to obtain frame-level acoustic features of the audio to be processed and a second candidate translated text based on the audio to be processed includes: using the acoustic encoder to process the audio to be processed to obtain frame-level acoustic features of the audio to be processed; and using the acoustic decoder to process the frame-level acoustic features to obtain the second candidate translated text.

[0023] In one embodiment, processing the frame-level acoustic features using the acoustic decoder to obtain the second candidate translated text includes: processing the frame-level acoustic features using the acoustic decoder to obtain the posterior probability space of the translated text; and performing a beam search on the posterior probability space of the translated text to obtain the second candidate translated text.

[0024] In one embodiment, obtaining a second confidence score of the second candidate translated text using a confidence assessment model based on the frame-level acoustic features and the second candidate translated text includes: obtaining a second confidence score of the second candidate translated text using the confidence assessment model based on the frame-level acoustic features, the second candidate translated text, and a keyword weight table; wherein the keyword weight table records the keyword weights of keyword elements.

[0025] In one embodiment, the audio to be processed relates to a target technical field, and the keyword elements include specialized terms within that target technical field.

[0026] In one embodiment, the keyword elements include high-frequency word elements; wherein, the high-frequency word elements are extracted from the corpus based on word frequency and inverse document frequency.

[0027] Thirdly, this application provides a confidence assessment device for automatic speech recognition results, comprising: The acquisition module is used to: acquire the first candidate translated text of the audio sample; the construction module is used to: construct a target reward function based on the word error rate and the first confidence score of the first candidate translated text; wherein the first confidence score is generated using an initial confidence evaluation model; the training module is used to: iteratively update the network parameters of the initial confidence evaluation model using a reinforcement learning algorithm based on the target reward function until the training termination condition is met, thereby obtaining a confidence evaluation model; the determination module is used to: determine the second confidence score of the second candidate translated text of the audio to be processed using the confidence evaluation model.

[0028] Fourthly, this application provides a confidence assessment device for automatic speech recognition results, comprising: The first acquisition module is used to: acquire the audio to be processed; the second acquisition module is used to: use a speech recognition model to obtain the frame-level acoustic features of the audio to be processed and a second candidate translated text based on the audio to be processed; wherein, the frame-level acoustic features include the feature vectors of each audio frame of the audio to be processed; the third acquisition module is used to: use a confidence evaluation model to obtain a second confidence score of the second candidate translated text based on the frame-level acoustic features and the second candidate translated text; wherein, the confidence evaluation model is obtained by reinforcement learning of an initial confidence evaluation model based on a target reward function, the target reward function is constructed based on the word error rate and the first confidence score of the first candidate translated text of the audio sample, and the first confidence score of the audio sample is generated using the initial confidence evaluation model.

[0029] Fifthly, embodiments of this application provide an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the steps of the confidence assessment method for the automatic speech recognition results described in the first or second aspect.

[0030] In a sixth aspect, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the confidence assessment method for the automatic speech recognition results described in the first or second aspect.

[0031] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the confidence assessment method for the automatic speech recognition results described in the first or second aspect.

[0032] The confidence evaluation method, apparatus, and electronic device for automatic speech recognition results provided in this application construct a target reward function based on the word error rate and first confidence score of the first candidate translated text, and use it for fine-tuning training in the reinforcement learning stage. The word error rate is a core indicator for evaluating the performance of a speech recognition model. By explicitly introducing the evaluation indicator (word error rate) from the inference stage into the training process, and based on this indicator, optimizing the network parameters of the initial confidence evaluation model using a reinforcement learning algorithm, the parameter update direction of the confidence evaluation model is highly aligned with the target of reducing the word error rate. This ensures the consistency between the training objective and the evaluation indicator, eliminating the disconnect between confidence and the true error rate in traditional methods. Therefore, the trained confidence evaluation model can be used to accurately evaluate the automatic speech recognition results. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is one of the flowcharts illustrating the confidence assessment method for automatic speech recognition results provided in this application embodiment; Figure 2 This is a flowchart illustrating the method for constructing the target reward function provided in an embodiment of this application; Figure 3 This is a second flowchart illustrating the confidence assessment method for automatic speech recognition results provided in this application embodiment; Figure 4 This is a schematic diagram illustrating the process of obtaining candidate translated text provided in an embodiment of this application; Figure 5 This is a schematic diagram illustrating the process of obtaining the confidence score provided in an embodiment of this application; Figure 6 This is one of the structural schematic diagrams of the confidence assessment device for automatic speech recognition results provided in the embodiments of this application; Figure 7 This is a second schematic diagram of the structure of the confidence assessment device for automatic speech recognition results provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0036] Figure 1 This is one of the flowcharts illustrating the confidence assessment method for automatic speech recognition results provided in this application embodiment. (Refer to...) Figure 1 This application provides a method for evaluating the confidence level of automatic speech recognition results, which may include: Step 101: Obtain the first candidate translated text of the audio sample.

[0037] Audio samples are digital audio files used during the training phase of the confidence assessment model, such as a recording of a conversation.

[0038] The first candidate translated text is one or more text results generated by the speech recognition model after processing the input audio sample. For example, if the audio sample is "The weather is very nice today", the first candidate translated text may include "The sky is clear and the weather is very nice today" or "The weather is very nice today".

[0039] In practice, any pre-trained speech recognition model (e.g., Transformer, Conformer, Wav2Vec 2.0, Hubert, etc.) can be used to process audio samples to obtain the first candidate translated text, without being limited by the description in this specification. A speech recognition model is an algorithmic system trained by machine learning algorithms that can convert audio signals into corresponding text.

[0040] In some embodiments, the speech recognition model includes an acoustic encoder and an acoustic decoder. The acoustic encoder can be used to process audio samples to obtain frame-level acoustic features of the audio samples. The frame-level acoustic features include feature vectors of each audio frame of the audio sample. The acoustic decoder is used to obtain a first candidate translated text based on the frame-level acoustic features.

[0041] Frame-level acoustic features are structural data such as spectrum and energy extracted from each frame of audio signal after segmenting the audio into fixed time windows in speech signal processing. These features are used to characterize local speech characteristics, such as the Mel frequency cepstral coefficient vector of a single frame of audio.

[0042] An acoustic encoder is used to extract and encode features from the input audio, and output frame-level acoustic features, which can be represented as a T*D dimensional feature matrix.

[0043] An acoustic decoder is used to decode the frame-level acoustic features output by an acoustic encoder into the corresponding translated text.

[0044] In the specific implementation process, an acoustic decoder can be used to process frame-level acoustic features to obtain the posterior probability space of the translated text; a beam search can be performed on the posterior probability space of the translated text to obtain the first candidate translated text.

[0045] Beam search is a heuristic graph search algorithm used to efficiently search for near-optimal solutions in a large candidate space of speech sequences. At each time step, the algorithm retains only the top N candidate sequences with the highest probabilities, gradually expanding until a termination condition is met (such as generating an end-of-sequence symbol or reaching the maximum length), thus balancing search efficiency and result quality. By performing a beam search on the posterior probability space output by the acoustic decoder, the top N candidate sequences can be output as the first candidate translated text, where the value N can be set experimentally or empirically.

[0046] Step 102: Construct a target reward function based on the word error rate and the first confidence score of the first candidate translated text; wherein the first confidence score is generated using the initial confidence evaluation model.

[0047] In the specific implementation process, the first confidence score of the first candidate translated text can be obtained in the following way: using the initial confidence evaluation model, the first confidence score of the first candidate translated text is generated based on the frame-level acoustic features of the audio sample and the first candidate translated text.

[0048] The first confidence score is a quantitative score given by the initial confidence assessment model to the credibility of the first candidate translated text. For example, if the initial confidence assessment model outputs "confidence 0.92", it means that the corresponding candidate translated text has a 92% probability of being judged as correct.

[0049] A confidence assessment model is a specialized model built using machine learning algorithms to measure the reliability of speech recognition results. The initial confidence assessment model is an untrained confidence assessment model.

[0050] In practice, the initial confidence assessment model can be constructed in various ways, without being limited by the description in this manual. For example, an attention layer + multilayer perceptron (MLP) + sigmoid output layer can be used to construct the initial confidence assessment model.

[0051] In some embodiments, an initial confidence evaluation model can be used to obtain a first confidence score for the first candidate translated text based on the frame-level acoustic features of the training audio, the first candidate translated text, and a keyword weight table; wherein, the keyword weight table records the keyword weights of keyword elements.

[0052] In some embodiments, the audio sample relates to a target technical field, and the keyword elements include specialized terms within that field.

[0053] In some embodiments, the keyword element also includes high-frequency word elements; wherein, the high-frequency word elements are extracted from the corpus based on word frequency and inverse document frequency.

[0054] As an example only, the initial confidence assessment model receives frame-level acoustic features. The word embedding vector of the first candidate translated text and keyword weight table As input, the output is a confidence score between 0 and 1. The keyword weight table is stored in dictionary format and can be divided into two modes: static vocabulary configuration and dynamic analysis. The static vocabulary configuration mode allows users to directly import domain-specific vocabulary (e.g., medical drug names). The dynamic analysis mode automatically extracts keywords by analyzing word frequency and distribution features in the corpus (based on word frequency and inverse document frequency). Taking a medical scenario as an example, in the sentence "Take 2 mg of glimepiride before each meal," errors in "glimepiride" and "2" could have serious consequences. Therefore, a keyword weight table can be designed to differentiate the error costs of keywords and non-keywords, assigning higher weights to keywords. For example: {"glimepiride": 5.0, "2": 3.0}.

[0055] For an example of constructing the target reward function based on the word error rate and first confidence score of the first candidate translated text, see [link to example]. Figure 2 The relevant content will not be repeated here.

[0056] Step 103: Based on the target reward function, use a reinforcement learning algorithm to iteratively update the network parameters of the initial confidence evaluation model until the training termination condition is met, thus obtaining the confidence evaluation model.

[0057] The speech recognition model (including the acoustic encoder and acoustic decoder) is pre-trained. When training the confidence evaluation model, the speech recognition model is frozen, and only the network parameters of the confidence evaluation model are trained.

[0058] For each batch of audio samples, a speech recognition model is used to generate frame-level acoustic features and N-best first-choice translated texts. Training data is constructed based on the frame-level acoustic features of the audio samples, the word embedding vectors of the first-choice translated texts, and the keyword weight table. The training data is input into an initial confidence evaluation model, which generates confidence scores for each first-choice translated text based on the current network parameters. The reinforcement learning algorithm adjusts the network parameters of the confidence evaluation model by calculating the gradient information fed back from the target reward function to maximize the long-term cumulative reward. This process is iterated until the reward value of the confidence evaluation model on the validation set converges or reaches the preset number of training rounds, ultimately yielding the optimized confidence evaluation model.

[0059] Step 104: Using the confidence assessment model, determine the second confidence score of the second candidate translated text of the audio to be processed.

[0060] For a detailed description of how to determine the second confidence score of the second candidate translated text of the audio to be processed using the confidence assessment model, please refer to the relevant content in step 303, which will not be repeated here.

[0061] Figure 2This is a flowchart illustrating the method for constructing the target reward function provided in an embodiment of this application. (Refer to...) Figure 2 This application provides a method for constructing a target reward function, which may include: Step 201: Based on the word error rate of the first candidate translated text, obtain the word error rate reward sub-item.

[0062] In the specific implementation process, the word error rate of the first candidate translated text It can be calculated using the edit distance algorithm, and the specific formula is as follows: ; in, S This indicates a replacement error. D This indicates a deletion error. I This indicates an insertion error. N This represents the total number of characters in the annotated text.

[0063] The word error rate reward sub-item can be calculated based on the MWER (Minimum Word Error Rate) metric, which is calculated based on minimizing the word error rate.

[0064] The word error rate reward sub-item is obtained based on the difference between the value 1 and the word error rate of the first candidate translated text. The formula is as follows: ; in, Representing the The first candidate translated text, The annotation text for the audio samples.

[0065] Step 202: Determine the weighted word error rate reward item based on the keyword weight table and the indicator function of each word in the first candidate translated text.

[0066] The indicator function is a binary function used to indicate whether a token has a translation error, and the keyword weight table records the keyword weight of the token.

[0067] In some embodiments, the indicator function for each lexical unit can be determined in the following manner: Based on the annotated text of the audio samples, determine whether each word in the first candidate translated text has a translation error. For each word in the first candidate translated text, if a translation error is determined, the indicator function for that word is assigned a first value; or, if the word is determined to be translated correctly, the indicator function for that word is assigned a second value. The first and second values ​​are different numerical values, with the first value being less than the second value. For example, the first value is 0, and the second value is 1. The indicator function is expressed by the following formula: ; Among them, For the first i The first candidate translated text j The indicator function corresponding to the lexical unit. If the lexical unit of the tagged text of the audio sample... j Each word in the candidate translated text If an insertion, deletion, or replacement error occurs, then the word / symbol translation is determined to be incorrect. If the first is 1; j Each word in the candidate translated text If it is correctly identified, then It is 0.

[0068] The weighted word error rate reward sub-item is calculated based on the weighted word error rate (WWER). WWER is based on the traditional word error rate (WER), but introduces keyword weights to each word element. First, the keyword weight of each word is obtained through the keyword weight table. When a keyword (a word with a higher keyword weight) is misidentified, it is penalized more severely, while non-keywords (words with lower keyword weights) are penalized less severely. The weight allocation supports a dynamic adjustment strategy. For key information such as drug names in medical scenarios and monetary amounts in financial scenarios, a significantly higher weight value can be set than for ordinary words.

[0069] In the specific implementation process, the keyword weight of each word element in the first candidate translated text can be determined according to the keyword weight table. Based on the keyword weight of each word element in the first candidate translated text, the indicator function of each word element is weighted and summed to obtain the weighted sum of word error rates. Based on the sum of the keyword weights of each word element in the first candidate translated text, the weighted sum of word error rates is normalized to obtain the weighted word error rate. Then, based on the weighted word error rate, a weighted word error rate reward item is obtained. For example, the weighted word error rate reward item can be obtained based on the difference between the value 1 and the weighted word error rate.

[0070] Weighted word error rate reward sub-item The calculation formula is as follows: ; in, For the first i The weighted word error rate reward item for each candidate translated text. For the first j Keyword weight for each word element.

[0071] If a terminology with a high keyword weight is translated incorrectly, the product of the terminology's keyword weight and the indicator function will be used to determine the translation error. This will result in a larger penalty for the weighted word error rate reward item, making the reinforcement learning of the confidence assessment model pay more attention to the translation errors of words with higher keyword weights.

[0072] Step 203: Obtain the word error rate reward value based on the word error rate reward sub-item and / or the weighted word error rate reward sub-item.

[0073] The word error rate reward value is a reward signal calculated under the reinforcement learning framework based on the word error rate and / or weighted word error rate of the candidate translated text. Its value is negatively correlated with the word error rate, that is, the lower the word error rate, the higher the word error rate reward value.

[0074] In some embodiments, the word error rate reward value may include only the word error rate reward sub-item or the weighted word error rate reward sub-item.

[0075] In some embodiments, the word error rate reward value can be a weighted sum of the word error rate reward sub-item and the weighted word error rate reward sub-item, and the calculation formula is shown below.

[0076] ; in, , This is a weighting coefficient, which can be adjusted according to the needs of different application scenarios.

[0077] The confidence alignment reward sub-item is used to measure the consistency between the confidence score of the candidate translated text output by the confidence evaluation model and the word error rate of the candidate translated text.

[0078] Step 204: Based on the difference between the first confidence score and the word error rate reward value of the first candidate translated text, construct a confidence alignment reward sub-item.

[0079] In some embodiments, the target reward function includes a confidence alignment reward sub-item. The confidence alignment reward sub-item is used to align the confidence score of the candidate translated text output by the confidence evaluation model with the word error rate of the candidate translated text.

[0080] In the specific implementation process, the word error rate reward value can be obtained based on the word error rate of the first candidate translated text (see the relevant content in steps 201 to 203 for a detailed description); and a confidence alignment reward sub-item can be constructed based on the difference between the first confidence score and the word error rate reward value of the first candidate translated text.

[0081] The confidence alignment reward sub-item is constructed based on the difference between the first confidence score and the word error rate reward value of the first candidate translated text in the following manner: Determine the first difference between the first confidence score and the word error rate reward value of the first candidate translated text; based on the value 1 and the first difference, obtain the confidence alignment reward sub-item. The confidence alignment reward sub-item is expressed by the following formula.

[0082] ; in, For the first i The confidence score of each candidate translated text is aligned with the reward sub-item. For the first i The confidence score of each candidate translated text. For the first i The word error rate reward value for each candidate translated text.

[0083] Step 205: Align the reward sub-items, error rate reward sub-items, and weighted word error rate reward sub-items according to the confidence level to obtain the target reward function.

[0084] The target reward function is obtained by merging the confidence alignment reward sub-item, the error rate reward sub-item, and the weighted word error rate reward sub-item through a weighted summation, and is expressed by the following formula.

[0085] ; in, , These are adjustable hyperparameters.

[0086] Figure 3 This is the second flowchart illustrating the confidence assessment method for automatic speech recognition results provided in this application. (Refer to...) Figure 3 This application provides a method for evaluating the confidence level of automatic speech recognition results, which may include: Step 301: Obtain the audio to be processed.

[0087] The audio to be processed is a digital audio file used in the inference phase of the confidence assessment model.

[0088] Step 302: Using a speech recognition model, obtain the frame-level acoustic features of the audio to be processed and the second candidate translated text based on the audio to be processed; wherein, the frame-level acoustic features include the feature vectors of each audio frame of the audio to be processed.

[0089] The second candidate translated text is one or more text results generated by the speech recognition model after processing the input audio.

[0090] In some embodiments, the speech recognition model includes an acoustic encoder and an acoustic decoder, such as Figure 4As shown, an acoustic encoder can be used to process the audio to be processed and obtain the frame-level acoustic features of the audio to be processed; an acoustic decoder can be used to process the frame-level acoustic features and obtain the second candidate translated text.

[0091] In the specific implementation process, an acoustic decoder can be used to process frame-level acoustic features to obtain the posterior probability space of the translated text; a beam search can be performed on the posterior probability space of the translated text to obtain the second candidate translated text.

[0092] Step 303: Using the confidence assessment model, based on the frame-level acoustic features and the second candidate translated text, obtain the second confidence score of the second candidate translated text.

[0093] The second confidence score is a quantitative score given by the confidence assessment model on the credibility of the second candidate translated text.

[0094] In some embodiments, such as Figure 5 As shown, a confidence assessment model can be used to obtain the second confidence score of the second candidate translated text based on frame-level acoustic features, the second candidate translated text, and the keyword weight table; whereby the keyword weight table records the keyword weights of keyword elements.

[0095] In some embodiments, the audio to be processed relates to a target technical field, and the keyword elements include specialized terms within that target technical field.

[0096] In some embodiments, the keyword element also includes high-frequency word elements; wherein, the high-frequency word elements are extracted from the corpus based on word frequency and inverse document frequency.

[0097] In the specific implementation process, after obtaining the second confidence score of the second candidate translated text, the candidate translated text with the highest second confidence score can be selected as the final translated text of the audio to be processed.

[0098] In noisy, dialectal, or technical vocabulary scenarios, if the confidence score of the final translated text is lower than the preset threshold, it will be processed in a tiered manner according to the score range: medium confidence score, it will be re-identified using a high-performance model or sent to post-processing processes such as error correction and smoothing; extremely low confidence score, it will be transferred to manual review or automatically discarded.

[0099] The confidence assessment model is obtained by reinforcement learning of the initial confidence assessment model based on the target reward function. The target reward function is constructed based on the word error rate and the first confidence score of the first candidate translated text of the audio sample. The first confidence score of the audio sample is generated using the initial confidence assessment model.

[0100] In some embodiments, the target reward function includes a confidence alignment reward sub-item, which can obtain a word error rate reward value based on the word error rate of the first candidate translated text; and construct a confidence alignment reward sub-item based on the difference between the first confidence score and the word error rate reward value of the first candidate translated text.

[0101] In the specific implementation process, the first difference between the first confidence score of the first candidate translated text and the word error rate reward value can be determined; based on the value 1 and the first difference, the confidence alignment reward sub-item is obtained.

[0102] In some embodiments, a word error rate reward sub-item can be obtained based on the word error rate of the first candidate translated text; a weighted word error rate reward sub-item can be determined based on the keyword weight table and the indicator function of each word element in the first candidate translated text; wherein, the indicator function is a binary function used to indicate whether a word element has a translation error, and the keyword weight table records the weight of the word element; and a word error rate reward value can be obtained based on the word error rate reward sub-item and / or the weighted word error rate reward sub-item.

[0103] In some embodiments, a word error rate reward sub-item can be obtained based on the difference between the value 1 and the word error rate of the first candidate translated text.

[0104] In some embodiments, the weight of each word element in the first candidate translated text can be determined according to the keyword weight table; based on the weight of each word element in the first candidate translated text, the indicator functions of each word element are weighted and summed to obtain the weighted sum of word error rates; based on the sum of the weights of each word element in the first candidate translated text, the weighted sum of word error rates is normalized to obtain the weighted word error rate; and based on the weighted word error rate, a weighted word error rate reward sub-item is obtained.

[0105] In practice, the weighted word error rate reward item can be obtained based on the difference between the value 1 and the weighted word error rate.

[0106] In some embodiments, the indicator function for each lexical unit can be determined in the following manner: Based on the annotated text of the audio sample, determine whether each word in the first candidate translated text has a translation error; for each word in the first candidate translated text, if it is determined that the word has a translation error, determine the indicator function of the word as the first value; or, if it is determined that the word has been translated correctly, determine the indicator function of the word as the second value.

[0107] In some embodiments, the target reward function further includes a word error rate reward sub-item and a weighted word error rate reward sub-item.

[0108] In the embodiments provided in this application, a confidence evaluation model is used to calculate a second confidence score for the second candidate translated text by utilizing the frame-level acoustic features of the audio to be processed and the second candidate translated text. The confidence evaluation model is based on a target reward function and is trained through reinforcement learning. The target reward function is optimized by integrating the word error rate of the audio sample and the first confidence score, enabling the confidence evaluation model to specifically reduce the confidence score of candidate translated texts with high word error rates, thereby accurately evaluating the automatic speech recognition results.

[0109] The confidence assessment device for automatic speech recognition results provided in the embodiments of this application will be described below. The confidence assessment device for automatic speech recognition results described below can be referred to in correspondence with the confidence assessment method for automatic speech recognition results described above.

[0110] Figure 6 This is one of the structural schematic diagrams of the confidence assessment device for automatic speech recognition results provided in the embodiments of this application.

[0111] like Figure 6 As shown, the confidence assessment device for automatic speech recognition results includes the following modules: The acquisition module 610 is used to: acquire the first candidate transcribing text of the audio sample.

[0112] The construction module 620 is used to: construct a target reward function based on the word error rate and the first confidence score of the first candidate translated text; wherein the first confidence score is generated using an initial confidence evaluation model.

[0113] The training module 630 is used to: based on the target reward function, use a reinforcement learning algorithm to iteratively update the network parameters of the initial confidence evaluation model until the training termination condition is met, thereby obtaining the confidence evaluation model.

[0114] The determination module 640 is used to: determine a second confidence score of a second candidate translated text of the audio to be processed using the confidence evaluation model.

[0115] Figure 7 This is the second schematic diagram of the structure of the confidence assessment device for automatic speech recognition results provided in the embodiments of this application.

[0116] like Figure 7 As shown, the confidence assessment device for automatic speech recognition results includes the following modules: The first acquisition module 710 is used to acquire the audio to be processed.

[0117] The second acquisition module 720 is used to: use a speech recognition model to obtain the frame-level acoustic features of the audio to be processed and a second candidate translated text based on the audio to be processed; wherein, the frame-level acoustic features include the feature vectors of each audio frame of the audio to be processed.

[0118] The third acquisition module 730 is used to: obtain a second confidence score of the second candidate translated text based on the frame-level acoustic features and the second candidate translated text using a confidence evaluation model; wherein the confidence evaluation model is obtained by reinforcement learning of the initial confidence evaluation model based on a target reward function, the target reward function is constructed based on the word error rate and the first confidence score of the first candidate translated text of the audio sample, and the first confidence score of the audio sample is generated using the initial confidence evaluation model.

[0119] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call a computer program in the memory 830 to execute steps of a confidence assessment method for automatic speech recognition results, such as including: Obtain the first candidate translated text of the audio sample; construct a target reward function based on the word error rate and first confidence score of the first candidate translated text; wherein, the first confidence score is generated using an initial confidence evaluation model; based on the target reward function, iteratively update the network parameters of the initial confidence evaluation model using a reinforcement learning algorithm until the training termination condition is met, thereby obtaining a confidence evaluation model; use the confidence evaluation model to determine the second confidence score of the second candidate translated text of the audio to be processed; or The process involves: acquiring the audio to be processed; using a speech recognition model to obtain frame-level acoustic features and a second candidate translated text for the audio to be processed; wherein the frame-level acoustic features include feature vectors for each audio frame of the audio to be processed; using a confidence evaluation model to obtain a second confidence score for the second candidate translated text based on the frame-level acoustic features and the second candidate translated text; wherein the confidence evaluation model is obtained by reinforcement learning of an initial confidence evaluation model based on a target reward function, the target reward function being constructed based on the word error rate and first confidence score of the first candidate translated text of the audio sample, and the first confidence score of the audio sample being generated using the initial confidence evaluation model.

[0120] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the confidence assessment method for the automatic speech recognition results provided in the above embodiments, such as including: Obtain the first candidate translated text of the audio sample; construct a target reward function based on the word error rate and first confidence score of the first candidate translated text; wherein, the first confidence score is generated using an initial confidence evaluation model; based on the target reward function, iteratively update the network parameters of the initial confidence evaluation model using a reinforcement learning algorithm until the training termination condition is met, thereby obtaining a confidence evaluation model; use the confidence evaluation model to determine the second confidence score of the second candidate translated text of the audio to be processed; or The process involves: acquiring the audio to be processed; using a speech recognition model to obtain frame-level acoustic features and a second candidate translated text for the audio to be processed; wherein the frame-level acoustic features include feature vectors for each audio frame of the audio to be processed; using a confidence evaluation model to obtain a second confidence score for the second candidate translated text based on the frame-level acoustic features and the second candidate translated text; wherein the confidence evaluation model is obtained by reinforcement learning of an initial confidence evaluation model based on a target reward function, the target reward function being constructed based on the word error rate and first confidence score of the first candidate translated text of the audio sample, and the first confidence score of the audio sample being generated using the initial confidence evaluation model.

[0122] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to perform the steps of the methods provided in the above embodiments, such as including: Obtain the first candidate translated text of the audio sample; construct a target reward function based on the word error rate and first confidence score of the first candidate translated text; wherein, the first confidence score is generated using an initial confidence evaluation model; based on the target reward function, iteratively update the network parameters of the initial confidence evaluation model using a reinforcement learning algorithm until the training termination condition is met, thereby obtaining a confidence evaluation model; use the confidence evaluation model to determine the second confidence score of the second candidate translated text of the audio to be processed; or The process involves: acquiring the audio to be processed; using a speech recognition model to obtain frame-level acoustic features and a second candidate translated text for the audio to be processed; wherein the frame-level acoustic features include feature vectors for each audio frame of the audio to be processed; using a confidence evaluation model to obtain a second confidence score for the second candidate translated text based on the frame-level acoustic features and the second candidate translated text; wherein the confidence evaluation model is obtained by reinforcement learning of an initial confidence evaluation model based on a target reward function, the target reward function being constructed based on the word error rate and first confidence score of the first candidate translated text of the audio sample, and the first confidence score of the audio sample being generated using the initial confidence evaluation model.

[0123] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0124] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method of confidence assessment of automatic speech recognition results, characterized in that, include: Obtain the first candidate translated text of the audio sample; A target reward function is constructed based on the word error rate and the first confidence score of the first candidate translated text; wherein the first confidence score is generated using the initial confidence evaluation model; Based on the target reward function, the network parameters of the initial confidence evaluation model are iteratively updated using a reinforcement learning algorithm until the training termination condition is met, thus obtaining the confidence evaluation model. Using the confidence assessment model, a second confidence score is determined for the second candidate translated text of the audio to be processed.

2. The method of confidence assessment of automatic speech recognition results according to claim 1, characterized in that, The target reward function includes a confidence-aligned reward sub-item; The step of constructing a target reward function based on the word error rate and the first confidence score of the first candidate translated text includes: The word error rate reward value is obtained based on the word error rate of the first candidate translated text; The confidence alignment reward sub-item is constructed based on the difference between the first confidence score of the first candidate translated text and the word error rate reward value.

3. The confidence assessment method for automatic speech recognition results according to claim 2, characterized in that, The step of constructing the confidence alignment reward sub-item based on the difference between the first confidence score and the word error rate reward value of the first candidate translated text includes: Determine the first difference between the first confidence score and the word error rate reward value of the first candidate translated text; Based on the value 1 and the first difference, the confidence alignment reward sub-item is obtained.

4. The confidence assessment method for automatic speech recognition results according to claim 2, characterized in that, The step of obtaining a word error rate reward value based on the word error rate of the first candidate translated text includes: Based on the word error rate of the first candidate translated text, a word error rate reward sub-item is obtained; Based on the keyword weight table and the indicator function of each word in the first candidate translated text, a weighted word error rate reward item is determined; wherein, the indicator function is a binary function used to indicate whether a word has a translation error, and the keyword weight table records the keyword weight of the word; The word error rate reward value is obtained based on the word error rate reward sub-item and / or the weighted word error rate reward sub-item.

5. The confidence evaluation method for automatic speech recognition results according to claim 4, characterized in that, The step of obtaining a word error rate reward sub-item based on the word error rate of the first candidate translated text includes: The word error rate reward sub-item is obtained based on the difference between the value 1 and the word error rate of the first candidate translated text.

6. The confidence evaluation method for automatic speech recognition results according to claim 4, characterized in that, The step of determining the weighted word error rate reward item for the first candidate translated text based on the keyword weight table and the indicator function of each word in the first candidate translated text includes: Based on the keyword weight table, determine the keyword weight of each word element in the first candidate translated text; Based on the keyword weight of each word element in the first candidate translated text, the indicator function of each word element is weighted and summed to obtain the weighted sum of word error rates; Based on the sum of the keyword weights of each word element in the first candidate translated text, the weighted sum of the word error rates is normalized to obtain the weighted word error rate; Based on the weighted word error rate, the weighted word error rate reward sub-item is obtained.

7. The confidence assessment method for automatic speech recognition results according to claim 6, characterized in that, The step of obtaining the weighted word error rate reward sub-item based on the weighted word error rate includes: The weighted word error rate reward sub-item is obtained based on the difference between the value 1 and the weighted word error rate.

8. The confidence assessment method for automatic speech recognition results according to claim 4, characterized in that, The indicator function for each lexical unit is determined as follows: Based on the labeled text of the audio sample, determine whether each word in the first candidate translated text has a translation error; For each word in the first candidate translated text, if it is determined that the word has a translation error, the indicator function of the word is determined to be a first value; Alternatively, if it is determined that the word translation is correct, the indicator function of the word is determined to be a second value.

9. The confidence assessment method for automatic speech recognition results according to claim 4, characterized in that, The target reward function also includes the word error rate reward sub-item and the weighted word error rate reward sub-item.

10. The method for evaluating the confidence level of automatic speech recognition results according to any one of claims 1 to 9, characterized in that, The first candidate translated text for obtaining the audio sample includes: The audio samples are processed using an acoustic encoder to obtain frame-level acoustic features of the audio samples; wherein, the frame-level acoustic features include the feature vectors of each audio frame of the audio samples. The first candidate translated text is obtained using an acoustic decoder based on the frame-level acoustic features.

11. The method for evaluating the confidence level of automatic speech recognition results according to claim 10, characterized in that, The process of obtaining the first candidate translated text using an acoustic decoder based on the frame-level acoustic features includes: The frame-level acoustic features are processed using an acoustic decoder to obtain the posterior probability space of the translated text. A beam search is performed on the posterior probability space of the translated text to obtain the first candidate translated text.

12. The method for evaluating the confidence level of automatic speech recognition results according to claim 10, characterized in that, The first confidence score of the first candidate translated text is obtained in the following way: Using the initial confidence assessment model, a first confidence score for the first candidate translated text is generated based on the frame-level acoustic features of the audio sample and the first candidate translated text.

13. The method for evaluating the confidence level of automatic speech recognition results according to claim 12, characterized in that, The step of generating a first confidence score for the first candidate translated text using the initial confidence evaluation model, based on the frame-level acoustic features of the audio sample and the first candidate translated text, includes: Using the initial confidence evaluation model, the first confidence score of the first candidate translated text is obtained based on the frame-level acoustic features of the training audio, the first candidate translated text, and the keyword weight table; The keyword weight table records the keyword weights of keyword elements.

14. The method for evaluating the confidence level of automatic speech recognition results according to claim 13, characterized in that, The audio sample relates to the target technical field, and the keyword elements include professional terms related to the target technical field.

15. The confidence assessment method for automatic speech recognition results according to claim 14, characterized in that, The keyword elements also include high-frequency word elements; wherein, the high-frequency word elements are extracted from the corpus based on word frequency and inverse document frequency.

16. A method for evaluating the confidence level of automatic speech recognition results, characterized in that, include: Obtain the audio to be processed; Using a speech recognition model, frame-level acoustic features of the audio to be processed and a second candidate translated text are obtained based on the audio to be processed; wherein, the frame-level acoustic features include the feature vectors of each audio frame of the audio to be processed; Using a confidence assessment model, a second confidence score for the second candidate translated text is obtained based on the frame-level acoustic features and the second candidate translated text. The confidence assessment model is obtained by reinforcement learning of an initial confidence assessment model based on a target reward function. The target reward function is constructed based on the word error rate and first confidence score of the first candidate translated text of the audio sample. The first confidence score of the audio sample is generated using the initial confidence assessment model.

17. The method for evaluating the confidence level of automatic speech recognition results according to claim 16, characterized in that, The speech recognition model includes an acoustic encoder and an acoustic decoder; The step of using a speech recognition model to obtain the frame-level acoustic features of the audio to be processed and the second candidate translated text based on the audio to be processed includes: The acoustic encoder is used to process the audio to be processed to obtain the frame-level acoustic features of the audio to be processed. The frame-level acoustic features are processed using the acoustic decoder to obtain the second candidate translated text.

18. The method for evaluating the confidence level of automatic speech recognition results according to claim 17, characterized in that, The process of using the acoustic decoder to process the frame-level acoustic features to obtain the second candidate translated text includes: Using the acoustic decoder, the frame-level acoustic features are processed to obtain the posterior probability space of the translated text; A beam search is performed on the posterior probability space of the translated text to obtain the second candidate translated text.

19. The method for evaluating the confidence level of automatic speech recognition results according to claim 16, characterized in that, The confidence assessment model, based on the frame-level acoustic features and the second candidate translated text, yields a second confidence score for the second candidate translated text, including: Using the confidence assessment model, the second confidence score of the second candidate translated text is obtained based on the frame-level acoustic features, the second candidate translated text, and the keyword weight table; The keyword weight table records the keyword weights of keyword elements.

20. The method for evaluating the confidence level of automatic speech recognition results according to claim 19, characterized in that, The audio to be processed relates to the target technical field, and the keyword elements include professional terms from the target technical field.

21. The method for evaluating the confidence level of automatic speech recognition results according to claim 20, characterized in that, The keyword elements include high-frequency word elements; wherein, the high-frequency word elements are extracted from the corpus based on word frequency and inverse document frequency.

22. A confidence assessment device for automatic speech recognition results, characterized in that, include: The acquisition module is used to: acquire the first candidate transcribing text of the audio sample; The construction module is used to: construct a target reward function based on the word error rate and the first confidence score of the first candidate translated text; wherein the first confidence score is generated using an initial confidence evaluation model; The training module is used to: based on the target reward function, use a reinforcement learning algorithm to iteratively update the network parameters of the initial confidence evaluation model until the training termination condition is met, thereby obtaining the confidence evaluation model; The determination module is used to: determine a second confidence score of a second candidate translated text of the audio to be processed using the confidence evaluation model.

23. A confidence assessment device for automatic speech recognition results, characterized in that, include: The first acquisition module is used to: acquire the audio to be processed; The second acquisition module is used to: use a speech recognition model to obtain the frame-level acoustic features of the audio to be processed and a second candidate translated text based on the audio to be processed; wherein, the frame-level acoustic features include the feature vectors of each audio frame of the audio to be processed; The third acquisition module is used to: obtain a second confidence score of the second candidate translated text based on the frame-level acoustic features and the second candidate translated text using a confidence evaluation model; wherein the confidence evaluation model is obtained by reinforcement learning of the initial confidence evaluation model based on a target reward function, the target reward function is constructed based on the word error rate and the first confidence score of the first candidate translated text of the audio sample, and the first confidence score of the audio sample is generated using the initial confidence evaluation model.

24. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the confidence assessment method for automatic speech recognition results according to any one of claims 1 to 15, or the steps of the confidence assessment method for automatic speech recognition results according to any one of claims 16 to 21.

25. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the confidence assessment method for the automatic speech recognition results as described in any one of claims 1 to 21.

26. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the confidence assessment method for the automatic speech recognition result as described in any one of claims 1 to 15, or the steps of the confidence assessment method for the automatic speech recognition result as described in any one of claims 16 to 21.

Images