Speech recognition evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides a speech recognition evaluation method and device, electronic equipment and a storage medium, and belongs to the technical field of natural language processing, and the method comprises the steps: obtaining a speech recognition text and a speech recognition text tag of a target speech; translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; aligning the speech recognition text with the first translation to obtain first alignment information, and aligning the speech recognition text tag with the second translation to obtain second alignment information; inputting the first alignment information and the second alignment information into an evaluation model to obtain an evaluation result of the speech recognition text; the evaluation model is obtained by training based on first sample alignment information corresponding to the sample voice recognition text, second sample alignment information corresponding to the sample voice recognition text label and an evaluation result label of the sample voice recognition text. The accuracy of speech recognition evaluation can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a speech recognition evaluation method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of computer and artificial intelligence technology, speech recognition technology is being used more and more widely. In some key areas, the accuracy of speech recognition is crucial. How to accurately evaluate the effect of speech recognition is a key issue that needs to be solved urgently. Traditional manual evaluation methods are costly and inefficient. Traditional word accuracy evaluation methods cannot accurately evaluate the true quality of speech recognition effects, resulting in misjudgments in actual applications. Summary of the invention

[0003] The present invention provides a speech recognition evaluation method, device, electronic device and storage medium, which are used to solve the defect that the word accuracy evaluation method in the prior art cannot accurately evaluate the real quality of the speech recognition effect, resulting in the existence of evaluation misjudgment in practical applications.

[0004] The present invention provides a speech recognition evaluation method, comprising: Acquire the speech recognition text of the target speech, and determine the speech recognition text label of the target speech; Translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; Aligning the speech recognition text with the first translation to obtain first alignment information, and aligning the speech recognition text label with the second translation to obtain second alignment information; Inputting the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; Among them, the evaluation model is trained based on first sample alignment information between a sample speech recognition text and a corresponding first sample translation, second sample alignment information between a sample speech recognition text label and a corresponding second sample translation, and an evaluation result label of the sample speech recognition text.

[0005] In some embodiments, inputting the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model includes: Based on the evaluation model, feature extraction is performed on the first alignment information to obtain semantic features of the first translation, feature extraction is performed on the second alignment information to obtain semantic features of the second translation, semantic similarity between the first translation and the second translation is calculated based on the semantic features of the first translation and the semantic features of the second translation, and an intelligibility evaluation result of the speech recognition text is obtained based on the semantic similarity.

[0006] In some embodiments, the calculating the semantic similarity between the first translation and the second translation based on the semantic features of the first translation and the semantic features of the second translation, and obtaining the intelligibility evaluation result of the speech recognition text based on the semantic similarity, includes: Aligning each word of each sentence of the first translation with each word of each sentence of the second translation to obtain a plurality of word pairs; Determining semantic features of two words in each word pair based on the semantic features of the first translation and the semantic features of the second translation; Calculating the semantic similarity between the two words in each vocabulary pair based on the semantic features of the two words in each vocabulary pair; The weight of each vocabulary pair is determined, and the intelligibility of the speech recognition text is calculated based on the semantic similarity of the two vocabulary words in each vocabulary pair and the weight of each vocabulary pair.

[0007] In some embodiments, aligning the speech recognition text with the first translation to obtain first alignment information includes: Inputting the speech recognition text and the first translation of the speech recognition text into a pre-trained first alignment model to obtain the first alignment information output by the first alignment model; The first alignment model is trained based on a sample speech recognition text and a first sample translation of the sample speech recognition text, and a first alignment information label of the sample speech recognition text and the first sample translation.

[0008] In some embodiments, aligning the speech recognition text label with the second translation to obtain second alignment information includes: Inputting the speech recognition text label and the second translation of the speech recognition text label into a pre-trained second alignment model to obtain the second alignment information output by the second alignment model; The second alignment model is trained based on the sample speech recognition text label and the second sample translation of the sample speech recognition text label, and the second alignment information label of the sample speech recognition text label and the second sample translation.

[0009] In some embodiments, translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label includes: Using a pre-trained machine translation model, the speech recognition text and the speech recognition text label are translated to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; The machine translation model is trained based on the sample speech recognition text and the sample speech recognition text label, as well as the first translation label of the sample speech recognition text and the second translation label of the sample speech recognition text label.

[0010] In some embodiments, the training process of the evaluation model includes: Acquire a sample speech recognition text of the sample speech, determine a sample speech recognition text label of the sample speech, and determine an evaluation result label of the sample speech recognition text; Translating the sample speech recognition text and the sample speech recognition text label to obtain a first sample translation of the sample speech recognition text and a second sample translation of the sample speech recognition text label; Align the sample speech recognition text with the first sample translation to obtain first sample alignment information, and align the sample speech recognition text label with the second sample translation to obtain second sample alignment information; The first sample alignment information and the second sample alignment information are used as training samples, and the evaluation result labels of the sample speech recognition texts are used as sample labels to train an initial evaluation model. After the training is completed, the evaluation model is obtained.

[0011] The present invention also provides a speech recognition evaluation device, comprising: An acquisition unit, used to acquire a speech recognition text of a target speech and determine a speech recognition text label of the target speech; A translation unit, configured to translate the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; an alignment unit, configured to align the speech recognition text with the first translation to obtain first alignment information, and to align the speech recognition text label with the second translation to obtain second alignment information; An evaluation unit, configured to input the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; Among them, the evaluation model is trained based on first sample alignment information between a sample speech recognition text and a corresponding first sample translation, second sample alignment information between a sample speech recognition text label and a corresponding second sample translation, and an evaluation result label of the sample speech recognition text.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned speech recognition evaluation methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the speech recognition evaluation method described in any one of the above is implemented.

[0014] The speech recognition evaluation method, device, electronic device and storage medium provided by the present invention can improve the accuracy of speech recognition evaluation by acquiring the speech recognition text and speech recognition text label of the target speech; translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; aligning the speech recognition text and the first translation to obtain first alignment information, aligning the speech recognition text label and the second translation to obtain second alignment information; and inputting the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 This is one of the flow charts of the speech recognition evaluation method provided by the embodiment of the present invention.

[0017] Figure 2 It is a schematic diagram of a process of translating a speech recognition text of a target speech provided by an embodiment of the present invention.

[0018] Figure 3 This is an example of the first alignment information and the second alignment information provided by the embodiment of the present invention.

[0019] Figure 4 This is the second flowchart of the speech recognition evaluation method provided by the embodiment of the present invention.

[0020] Figure 5This is the third flow chart of the speech recognition evaluation method provided by the embodiment of the present invention.

[0021] Figure 6 It is a flowchart of the training process of the evaluation model provided by an embodiment of the present invention.

[0022] Figure 7 It is a structural diagram of a speech recognition evaluation device provided by an embodiment of the present invention.

[0023] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0026] Existing speech recognition effect evaluation schemes mainly focus on evaluating the accuracy of speech recognition systems through traditional quantitative indicators. These evaluation methods are usually based on the following aspects: (1) Word Error Rate (WER): Word accuracy is the most common speech recognition evaluation metric, which evaluates the accuracy of speech recognition by calculating the difference between the speech recognition result and the reference text. Its advantages are simplicity, intuitiveness and wide application, but it cannot capture the accuracy at the semantic level. It only evaluates the literal matching degree and cannot distinguish the impact of misrecognition, such as the situation where the word is wrong but the semantics is still correct (which may affect the user experience). It has a greater impact on speech noise or accent and cannot distinguish between slight pronunciation deviations and key semantic errors.

[0027] (2) Character Error Rate (CER): The character error rate is similar to WER, but the character error rate is evaluated on characters instead of words. The calculation method of CER is the same as WER, except that the text is split into characters instead of words. Its advantage is that for fine-grained evaluations in speech recognition (such as pinyin, syllables, etc.), CER is more refined and can be applied to certain application scenarios that require character-level evaluation. However, like WER, CER cannot capture errors at the semantic level and focuses on literal accuracy. In the case of long texts or low word accuracy, it may be difficult to directly reflect the overall effect of the speech recognition system.

[0028] (3) Sentence Error Rate (SER): Evaluate whether each sentence of speech recognition completely matches the reference sentence. If the recognition result is completely consistent with the reference sentence, the sentence is considered to be correctly recognized. Sentence-level accuracy evaluation is applicable to multi-sentence recognition tasks. It can evaluate the overall performance of the speech recognition system on multiple sentences and can quickly provide an evaluation of the overall recognition ability of the speech recognition system. However, the information provided is small and cannot reveal the specific error type. For sentences with a small number of errors but correct semantics, this evaluation method may be too harsh.

[0029] The solution of evaluating the accuracy of speech recognition systems based on traditional quantitative indicators cannot capture the accuracy at the semantic level. It can only reflect the precision of recognition and cannot consider the semantic correctness of the recognition results. Even if a speech recognition system has a high word accuracy, it may still not be able to accurately convey the actual meaning of the sentence, especially when dealing with synonyms or sentences with strong context dependence. In addition, traditional methods cannot effectively identify and distinguish the accuracy of key information. In actual applications, errors in certain words or vocabulary will not significantly affect the overall semantics, but errors in some key information will seriously affect the effectiveness of the speech recognition system, and traditional evaluation methods often fail to reflect this situation.

[0030] To this end, an embodiment of the present invention provides a speech recognition evaluation method, device, electronic device and storage medium, which obtains a speech recognition text and a speech recognition text label of a target speech; translates the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; aligns the speech recognition text and the first translation to obtain first alignment information, aligns the speech recognition text label and the second translation to obtain second alignment information; inputs the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text. The present invention can improve the accuracy of speech recognition evaluation.

[0031] Figure 1 This is a flow chart of a speech recognition evaluation method provided by an embodiment of the present invention. Figure 1As shown, a speech recognition evaluation method is provided, comprising the following steps: step 110, step 120, step 130 and step 140. The steps of the method flow are only a possible implementation of the present invention.

[0032] Step 110: Acquire the speech recognition text of the target speech and determine the speech recognition text label of the target speech.

[0033] Optionally, a traditional speech recognition system, a deep learning-based speech recognition method, or a professional speech recognition software or platform is used to obtain speech recognition text of the target speech.

[0034] Optionally, a sound signal emitted by a user is captured by an audio device (such as a microphone) to obtain a target voice, and the target voice is recognized based on a voice recognition technology to obtain a voice recognition text.

[0035] Step 120: Translate the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label.

[0036] Optionally, a transformer-based end-to-end machine translation system is used to translate the speech recognition text and the speech recognition text labels.

[0037] Among them, the Transformer-based end-to-end machine translation system uses the self-attention mechanism and parallel computing capabilities of the Transformer model to directly translate source language sentences into target language sentences, which can improve the effect and efficiency of translation.

[0038] It should be noted that differences in speech recognition texts often lead to different machine translation results. If there are some problems in the speech recognition text, the machine translation system can amplify them to a certain extent through the translation model's understanding of the semantics, and compare them with the translation of the speech recognition text label to more significantly discover the problems.

[0039] Figure 2 The flowchart of translating the speech recognition text of the target speech provided by the embodiment of the present invention is as follows. Figure 2 As shown, translating the speech recognition text of the target speech includes the following steps: Performing speech recognition on the target speech to obtain a speech recognition text of the target speech; The speech recognition text is machine translated to obtain a first translation of the speech recognition text.

[0040] For example, the target speech is "I will meet you at the park", the speech recognition text is "I willmeet you at the bark", the corresponding first translation is "I will meet you at the bark", and the second translation of the speech recognition text label is "I will meet you at the park"; the meeting place is the key information. Through machine translation, it can be found that although the word accuracy of the speech recognition result is high, there are major problems at the semantic level and the accuracy is low.

[0041] Step 130: align the speech recognition text with the first translation to obtain first alignment information, and align the speech recognition text label with the second translation to obtain second alignment information.

[0042] Optionally, the Mgiza++ alignment tool is used to align the speech recognition text and the first translation to obtain first alignment information, and the speech recognition text label and the second translation are aligned to obtain second alignment information.

[0043] It should be noted that word alignment refers to the process of matching words or phrases in the source language with words or phrases in the target language one by one.

[0044] Step 140: input the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; Among them, the evaluation model is trained based on the first sample alignment information of the sample speech recognition text and the corresponding first sample translation, the second sample alignment information of the sample speech recognition text label and the corresponding second sample translation, and the evaluation result label of the sample speech recognition text.

[0045] Optionally, the first alignment information and the second alignment information are compared based on the evaluation model, the semantic similarity between the two is calculated, and the speech recognition text is evaluated according to the semantic similarity to obtain an evaluation result.

[0046] In an embodiment of the present invention, a speech recognition text and a speech recognition text label of a target speech are obtained; the speech recognition text and the speech recognition text label are translated to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; the speech recognition text and the first translation are aligned to obtain first alignment information, and the speech recognition text label and the second translation are aligned to obtain second alignment information; the first alignment information and the second alignment information are input into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text, so that a semantic-level evaluation of the speech recognition text can be performed, thereby improving the accuracy of the speech recognition evaluation.

[0047] In some embodiments, the first alignment information and the second alignment information are input into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model, including: Based on the evaluation model, feature extraction is performed on the first alignment information to obtain semantic features of the first translation, feature extraction is performed on the second alignment information to obtain semantic features of the second translation, based on the semantic features of the first translation and the semantic features of the second translation, the semantic similarity between the first translation and the second translation is calculated, and based on the semantic similarity, an intelligibility evaluation result of the speech recognition text is obtained.

[0048] Among them, intelligibility refers to a measure of speech comprehension ability under given conditions, and intelligibility can be quantified by counting the number of correctly recognized words, phonemes or other language units.

[0049] It can be understood that by adopting an evaluation model, the semantic similarity between the first translation and the second translation is calculated based on the semantic features of the first translation and the semantic features of the second translation, and then the speech recognition text can be comprehensively evaluated based on the semantic similarity to obtain the intelligibility evaluation result of the speech recognition text, thereby improving the efficiency and accuracy of the speech recognition evaluation.

[0050] In some embodiments, based on the semantic features of the first translation and the semantic features of the second translation, the semantic similarity between the first translation and the second translation is calculated, and based on the semantic similarity, an intelligibility evaluation result of the speech recognition text is obtained, including: Aligning each word of each sentence of the first translation with each word of each sentence of the second translation to obtain a plurality of word pairs; Determining semantic features of two words in each word pair based on the semantic features of the first translation and the semantic features of the second translation; Based on the semantic features of the two words in each word pair, the semantic similarity of the two words in each word pair is calculated; The weight of each vocabulary pair is determined, and the intelligibility of the speech recognition text is calculated based on the semantic similarity of the two vocabulary words in each vocabulary pair and the weight of each vocabulary pair.

[0051] Optionally, feature extraction is performed on the first translation and the second translation to obtain semantic features of the first translation and semantic features of the second translation.

[0052] Optionally, the weight of each word pair is determined according to the importance of each word.

[0053] It can be understood that by aligning each word in each sentence of the first translation with each word in each sentence of the second translation, a plurality of word pairs are obtained, the semantic features of the two words in each word pair are determined, and the semantic similarity of the two words in each word pair is calculated, thereby quantifying the differences between different parts of the first translation and the second translation, thereby improving the accuracy and precision of speech recognition evaluation; by determining the weight of each word pair, the intelligibility of the speech recognition text is calculated based on the semantic similarity of the two words in each word pair and the weight of each word pair, and the accuracy of the key information can be identified, especially when the key information is recognized incorrectly or the word accuracy is low but does not affect the overall semantics, misjudgment can be avoided.

[0054] Figure 3 This is an example of the first alignment information and the second alignment information provided by the embodiment of the present invention. Figure 3 As shown, the speech recognition text is “I went to the market to sell vegetables”, the first translation is “I went to the market to sell vegetables”, and the first alignment information includes “I / I, went to / go, the market / market, to sell / sell, vegetables / vegetables”; the speech recognition text label is “I went to the market to buy vegetables”, the second translation is “I went to the market to buyvegetables”, and the second alignment information includes “I / I, went to / go, the market / market, to buy / buy, vegetables / vegetables”.

[0055] In this example, "I go to the market to sell vegetables" and "I go to the market to buy vegetables" differ by only one word, and the word accuracy is relatively high, at 83%. However, key information recognition errors result in completely different semantics, and the intelligibility score is only 50 points.

[0056] In some embodiments, aligning the speech recognition text with the first translation to obtain first alignment information includes: Inputting the speech recognition text and the first translation of the speech recognition text into a pre-trained first alignment model to obtain first alignment information output by the first alignment model; The first alignment model is obtained by training based on the sample speech recognition text and the first sample translation of the sample speech recognition text, as well as the first alignment information label of the sample speech recognition text and the first sample translation.

[0057] Optionally, the first alignment model includes a first alignment layer, a first differentiation layer and a first mapping layer; the first alignment layer is used to align the speech recognition text and the first translation, the first differentiation layer is used to differentiate the speech recognition text and the first translation, and the first mapping layer is used to determine the mapping relationship between each word of the speech recognition text and each word of the first translation.

[0058] Optionally, the training process of the first alignment model includes: Acquire a sample speech recognition text and a first sample translation of the sample speech recognition text; Determine a first alignment information tag of the sample speech recognition text and the first sample translation; The sample speech recognition text and the first sample translation are used as training samples, and the first alignment information labels of the sample speech recognition text and the first sample translation are used as sample labels to train an initial first alignment model. After the training is completed, a first alignment model is obtained.

[0059] It can be understood that based on the first alignment model, the speech recognition text and the first translation of the speech recognition text are aligned to obtain the first alignment information, which improves the alignment efficiency, facilitates the alignment of large-scale data, and reduces the alignment cost.

[0060] In some embodiments, aligning the speech recognition text label with the second translation to obtain second alignment information includes: Inputting the speech recognition text label and the second translation of the speech recognition text label into a pre-trained second alignment model to obtain second alignment information output by the second alignment model; The second alignment model is obtained by training based on the sample speech recognition text label and the second sample translation of the sample speech recognition text label, and the second alignment information label of the sample speech recognition text label and the second sample translation.

[0061] Optionally, the second alignment model includes a second alignment layer, a second differentiation layer and a second mapping layer; the second alignment layer is used to align the speech recognition text label and the second translation, the second differentiation layer is used to differentiate the speech recognition text label and the second translation, and the second mapping layer is used to determine the mapping relationship between each word of the speech recognition text label and each word of the second translation.

[0062] Optionally, the training process of the second alignment model includes: Obtaining a sample speech recognition text label and a second sample translation of the sample speech recognition text label; Determine a sample speech recognition text label and a second alignment information label of a second sample translation; The sample speech recognition text label and the second sample translation are used as training samples, and the second alignment information label of the sample speech recognition text label and the second sample translation is used as sample label to train the initial second alignment model. After the training is completed, the second alignment model is obtained.

[0063] It can be understood that based on the second alignment model, the speech recognition text label and the second translation of the speech recognition text label are aligned to obtain the second alignment information output by the second alignment model, which improves the alignment efficiency, facilitates the alignment of large-scale data, and reduces the alignment cost.

[0064] In some embodiments, translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label includes: Using a pre-trained machine translation model, the speech recognition text and the speech recognition text label are translated to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; The machine translation model is trained based on the sample speech recognition text and the sample speech recognition text label, as well as the first translation label of the sample speech recognition text and the second translation label of the sample speech recognition text label.

[0065] It can be understood that translating the speech recognition text and the speech recognition text label based on the machine translation model improves the translation efficiency.

[0066] Optionally, the training process of the machine translation model includes: Obtain sample speech recognition text and sample speech recognition text label; Determine a first translation label of the sample speech recognition text and a second translation label of the sample speech recognition text label; The sample speech recognition text and the sample speech recognition text label are used as training samples, the first translation label of the sample speech recognition text and the second translation label of the sample speech recognition text label are used as sample labels, and an initial machine translation model is trained. After the training is completed, a machine translation model is obtained.

[0067] Figure 4 The second flow chart of the speech recognition evaluation method provided by the embodiment of the present invention. Figure 4 As shown, in some embodiments, a speech recognition evaluation method is provided, comprising the following steps: Get the target voice; Determine the speech recognition text and speech recognition text label of the target speech; Based on the machine translation model, the speech recognition text and the speech recognition text label are translated to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; Based on the first alignment model, the speech recognition text and the first translation are word-aligned to obtain first alignment information, and based on the second alignment model, the speech recognition text label and the second translation are aligned to obtain second alignment information; Based on the evaluation model, the first alignment information and the second alignment information are compared to obtain an automatic evaluation result of the speech recognition text; Get expert evaluation results of speech recognition text; Based on the expert evaluation results, the automatic evaluation results are corrected.

[0068] Among them, the expert evaluation results are obtained by conducting a semantic rationality evaluation based on the first translation of the speech recognition text and the speech recognition text label.

[0069] It can be understood that by comparing the automatic evaluation results of the speech recognition text with the expert evaluation results, the parameters of the evaluation model, the first alignment model and the second alignment model can be optimized, thereby improving the accuracy of the automatic evaluation.

[0070] Figure 5 The third flow chart of the speech recognition evaluation method provided by the embodiment of the present invention is as follows: Figure 5 As shown, in some embodiments, a speech recognition evaluation method is provided, comprising the following steps: Obtain the speaker's target voice; Based on the speech recognition system, the target speech is recognized to obtain the speech recognition text (i.e. the real original text of the speech recognition); Determine the speech recognition text label of the target speech (i.e. the speech recognition annotated original text); Based on the large model machine translation engine, the speech recognition text is translated to obtain a first translation (i.e., the real speech recognition translation), and the speech recognition text label is translated to obtain a second translation (i.e., the speech recognition annotated translation); Based on a pre-trained intelligibility evaluation model, the speech recognition text and the first translation, as well as the speech recognition text label and the second translation are processed to obtain an intelligibility score of the speech recognition text.

[0071] Optionally, the intelligibility evaluation model includes a first alignment layer, a second alignment layer, a similarity calculation layer, and an intelligibility assessment layer.

[0072] Among them, the first alignment layer is used to align the speech recognition text and the first translation to obtain the first alignment information; the second alignment layer is used to align the speech recognition text label and the second translation to obtain the second alignment information; the similarity calculation layer is used to calculate the semantic similarity between the first translation and the second translation based on the first alignment information and the second alignment information; the intelligibility evaluation layer is used to evaluate the intelligibility of the speech recognition text based on the semantic similarity to obtain the intelligibility score of the speech recognition text.

[0073] Among them, the intelligibility evaluation model is trained based on the sample speech recognition text and its corresponding sample first translation, the sample speech recognition text label and its corresponding sample second translation, and the intelligibility score label of the sample speech recognition text.

[0074] Figure 6 A flow chart of the training process of the evaluation model provided in the embodiment of the present invention is as follows: Figure 6 As shown, in some embodiments, the training process of the evaluation model includes: Step 610: Obtain a sample speech recognition text of the sample speech, determine a sample speech recognition text label of the sample speech, and determine an evaluation result label of the sample speech recognition text; Step 620: Translate the sample speech recognition text and the sample speech recognition text label to obtain a first sample translation of the sample speech recognition text and a second sample translation of the sample speech recognition text label; Step 630: align the sample speech recognition text with the first sample translation to obtain first sample alignment information, and align the sample speech recognition text label with the second sample translation to obtain second sample alignment information; Step 640: Use the first sample alignment information and the second sample alignment information as training samples and the evaluation result labels of the sample speech recognition text as sample labels to train an initial evaluation model. After the training is completed, an evaluation model is obtained.

[0075] Optionally, based on a machine translation model, the sample speech recognition text and the sample speech recognition text label are translated to obtain a first sample translation of the sample speech recognition text and a second sample translation of the sample speech recognition text label.

[0076] Optionally, based on the first alignment model, the sample speech recognition text and the first sample translation are aligned to obtain first sample alignment information.

[0077] Optionally, based on the second alignment model, the sample speech recognition text label and the second sample translation are aligned to obtain second sample alignment information.

[0078] Optionally, the first sample alignment information and the second sample alignment information are input into an initial evaluation model to obtain a predicted evaluation result of the sample speech recognition text output by the initial evaluation model.

[0079] Optionally, based on the predicted evaluation results of the sample speech recognition text and the evaluation result label of the sample speech recognition text, a loss function value is calculated, and based on the loss function value, the parameters of the initial evaluation model are iteratively optimized to obtain an evaluation model.

[0080] The speech recognition evaluation device provided by an embodiment of the present invention is described below. The speech recognition evaluation device described below and the speech recognition evaluation method described above can be referenced to each other.

[0081] Figure 7 A schematic diagram of the structure of a speech recognition evaluation device provided by an embodiment of the present invention is shown in FIG. Figure 7 As shown, the speech recognition evaluation device 700 includes: An acquisition unit 710 is used to acquire a speech recognition text of a target speech and determine a speech recognition text label of the target speech; A translation unit 720, configured to translate the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; An alignment unit 730 is used to align the speech recognition text with the first translation to obtain first alignment information, and to align the speech recognition text label with the second translation to obtain second alignment information; An evaluation unit 740, configured to input the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; Among them, the evaluation model is trained based on the first sample alignment information of the sample speech recognition text and the corresponding first sample translation, the second sample alignment information of the sample speech recognition text label and the corresponding second sample translation, and the evaluation result label of the sample speech recognition text.

[0082] Optionally, the first alignment information and the second alignment information are input into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model, including: Based on the evaluation model, feature extraction is performed on the first alignment information to obtain semantic features of the first translation, feature extraction is performed on the second alignment information to obtain semantic features of the second translation, based on the semantic features of the first translation and the semantic features of the second translation, the semantic similarity between the first translation and the second translation is calculated, and based on the semantic similarity, an intelligibility evaluation result of the speech recognition text is obtained.

[0083] Optionally, based on the semantic features of the first translation and the semantic features of the second translation, the semantic similarity between the first translation and the second translation is calculated, and based on the semantic similarity, an intelligibility evaluation result of the speech recognition text is obtained, including: Aligning each word of each sentence of the first translation with each word of each sentence of the second translation to obtain a plurality of word pairs; Determining semantic features of two words in each word pair based on the semantic features of the first translation and the semantic features of the second translation; Based on the semantic features of the two words in each word pair, the semantic similarity of the two words in each word pair is calculated; The weight of each vocabulary pair is determined, and the intelligibility of the speech recognition text is calculated based on the semantic similarity of the two vocabulary words in each vocabulary pair and the weight of each vocabulary pair.

[0084] Optionally, aligning the speech recognition text with the first translation to obtain first alignment information includes: Inputting the speech recognition text and the first translation of the speech recognition text into a pre-trained first alignment model to obtain first alignment information output by the first alignment model; The first alignment model is obtained by training based on the sample speech recognition text and the first sample translation of the sample speech recognition text, as well as the first alignment information label of the sample speech recognition text and the first sample translation.

[0085] Optionally, aligning the speech recognition text label with the second translation to obtain second alignment information includes: Inputting the speech recognition text label and the second translation of the speech recognition text label into a pre-trained second alignment model to obtain second alignment information output by the second alignment model; The second alignment model is obtained by training based on the sample speech recognition text label and the second sample translation of the sample speech recognition text label, and the second alignment information label of the sample speech recognition text label and the second sample translation.

[0086] Optionally, translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label includes: Using a pre-trained machine translation model, the speech recognition text and the speech recognition text label are translated to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; The machine translation model is trained based on the sample speech recognition text and the sample speech recognition text label, as well as the first translation label of the sample speech recognition text and the second translation label of the sample speech recognition text label.

[0087] Optionally, the training process of the evaluation model includes: Obtaining a sample speech recognition text of the sample speech, determining a sample speech recognition text label of the sample speech, and determining an evaluation result label of the sample speech recognition text; Translating the sample speech recognition text and the sample speech recognition text label to obtain a first sample translation of the sample speech recognition text and a second sample translation of the sample speech recognition text label; Align the sample speech recognition text with the first sample translation to obtain first sample alignment information, and align the sample speech recognition text label with the second sample translation to obtain second sample alignment information; The first sample alignment information and the second sample alignment information are used as training samples, and the evaluation result labels of the sample speech recognition texts are used as sample labels to train an initial evaluation model. After the training is completed, an evaluation model is obtained.

[0088] It should be noted here that the speech recognition evaluation device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned speech recognition evaluation method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail here.

[0089] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 8 As shown, the electronic device may include: a processor (processor) 810, a communication interface (Communications Interface) 820, a memory (memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute a speech recognition evaluation method, which includes: obtaining a speech recognition text of a target speech and determining a speech recognition text label of the target speech; translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; aligning the speech recognition text and the first translation to obtain first alignment information, aligning the speech recognition text label and the second translation to obtain second alignment information; inputting the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; wherein the evaluation model is trained based on first sample alignment information of a sample speech recognition text and a corresponding first sample translation, second sample alignment information of a sample speech recognition text label and a corresponding second sample translation, and an evaluation result label of the sample speech recognition text.

[0090] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0091] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the speech recognition evaluation method provided by the above-mentioned methods, the method comprising: obtaining a speech recognition text of a target speech and determining a speech recognition text label of the target speech; translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; aligning the speech recognition text and the first translation to obtain first alignment information, aligning the speech recognition text label and the second translation to obtain second alignment information; inputting the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; wherein the evaluation model is trained based on first sample alignment information of a sample speech recognition text and a corresponding first sample translation, second sample alignment information of a sample speech recognition text label and a corresponding second sample translation, and an evaluation result label of the sample speech recognition text.

[0092] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A speech recognition evaluation method, characterized in that: include: Acquire the speech recognition text of the target speech, and determine the speech recognition text label of the target speech; Translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; Aligning the speech recognition text with the first translation to obtain first alignment information, and aligning the speech recognition text label with the second translation to obtain second alignment information; Inputting the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; Among them, the evaluation model is trained based on first sample alignment information between a sample speech recognition text and a corresponding first sample translation, second sample alignment information between a sample speech recognition text label and a corresponding second sample translation, and an evaluation result label of the sample speech recognition text.

2. The speech recognition evaluation method according to claim 1, characterized in that: The step of inputting the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model includes: Based on the evaluation model, feature extraction is performed on the first alignment information to obtain semantic features of the first translation, feature extraction is performed on the second alignment information to obtain semantic features of the second translation, semantic similarity between the first translation and the second translation is calculated based on the semantic features of the first translation and the semantic features of the second translation, and an intelligibility evaluation result of the speech recognition text is obtained based on the semantic similarity.

3. The speech recognition evaluation method according to claim 2, characterized in that: The calculating the semantic similarity between the first translation and the second translation based on the semantic features of the first translation and the semantic features of the second translation, and obtaining the intelligibility evaluation result of the speech recognition text based on the semantic similarity, includes: Aligning each word of each sentence of the first translation with each word of each sentence of the second translation to obtain a plurality of word pairs; Determining semantic features of two words in each word pair based on the semantic features of the first translation and the semantic features of the second translation; Calculating the semantic similarity between the two words in each vocabulary pair based on the semantic features of the two words in each vocabulary pair; The weight of each vocabulary pair is determined, and the intelligibility of the speech recognition text is calculated based on the semantic similarity of the two vocabulary words in each vocabulary pair and the weight of each vocabulary pair.

4. The speech recognition evaluation method according to claim 1, characterized in that: The step of aligning the speech recognition text with the first translation to obtain first alignment information includes: Inputting the speech recognition text and the first translation of the speech recognition text into a pre-trained first alignment model to obtain the first alignment information output by the first alignment model; The first alignment model is trained based on a sample speech recognition text and a first sample translation of the sample speech recognition text, and a first alignment information label of the sample speech recognition text and the first sample translation.

5. The speech recognition evaluation method according to claim 1, characterized in that: The step of aligning the speech recognition text label with the second translation to obtain second alignment information includes: Inputting the speech recognition text label and the second translation of the speech recognition text label into a pre-trained second alignment model to obtain the second alignment information output by the second alignment model; The second alignment model is trained based on the sample speech recognition text label and the second sample translation of the sample speech recognition text label, and the second alignment information label of the sample speech recognition text label and the second sample translation.

6. The speech recognition evaluation method according to any one of claims 2 to 5, characterized in that: The translating the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label includes: Using a pre-trained machine translation model, the speech recognition text and the speech recognition text label are translated to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; The machine translation model is trained based on the sample speech recognition text and the sample speech recognition text label, as well as the first translation label of the sample speech recognition text and the second translation label of the sample speech recognition text label.

7. The speech recognition evaluation method according to claim 1, characterized in that: The training process of the evaluation model includes: Acquire a sample speech recognition text of the sample speech, determine a sample speech recognition text label of the sample speech, and determine an evaluation result label of the sample speech recognition text; Translating the sample speech recognition text and the sample speech recognition text label to obtain a first sample translation of the sample speech recognition text and a second sample translation of the sample speech recognition text label; Align the sample speech recognition text with the first sample translation to obtain first sample alignment information, and align the sample speech recognition text label with the second sample translation to obtain second sample alignment information; The first sample alignment information and the second sample alignment information are used as training samples, and the evaluation result labels of the sample speech recognition texts are used as sample labels to train an initial evaluation model. After the training is completed, the evaluation model is obtained.

8. A speech recognition evaluation device, characterized in that: include: An acquisition unit, used to acquire a speech recognition text of a target speech and determine a speech recognition text label of the target speech; A translation unit, configured to translate the speech recognition text and the speech recognition text label to obtain a first translation of the speech recognition text and a second translation of the speech recognition text label; an alignment unit, configured to align the speech recognition text with the first translation to obtain first alignment information, and to align the speech recognition text label with the second translation to obtain second alignment information; An evaluation unit, configured to input the first alignment information and the second alignment information into a pre-trained evaluation model to obtain an evaluation result of the speech recognition text output by the evaluation model; Among them, the evaluation model is trained based on first sample alignment information between a sample speech recognition text and a corresponding first sample translation, second sample alignment information between a sample speech recognition text label and a corresponding second sample translation, and an evaluation result label of the sample speech recognition text.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the speech recognition evaluation method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the speech recognition evaluation method according to any one of claims 1 to 7 is implemented.

Images