Speech recognition evaluation method based on artificial intelligence

Abstract

The invention relates to the technical field of speech recognition effect evaluation, in particular to a speech recognition evaluation method based on artificial intelligence. The method comprises the following steps: acquiring to-be-evaluated voice data, analyzing the to-be-evaluated voice data to obtain a voice fluency index, a voice clarity index and a voice accuracy index, and calculating to obtain a first voice quality index of the to-be-evaluated voice data based on the voice fluency index, the voice clarity index and the voice accuracy index; inputting the to-be-evaluated voice data into the voice content recognition model, outputting a prediction text of the to-be-evaluated voice data, and performing comparative analysis on the prediction text and the labeled text to obtain a second voice quality index of the to-be-evaluated voice data; and obtaining a voice recognition evaluation result corresponding to the to-be-evaluated voice data based on the first voice quality index and the second voice quality index. According to the invention, the evaluation efficiency and accuracy of the speech recognition effect can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition effect evaluation, and in particular to a speech recognition evaluation method based on artificial intelligence. Background Art

[0002] With the advancement of science and technology, speech recognition technology has made significant progress and has been widely used in many fields, such as smart home, autonomous driving, intelligent customer service, etc. However, the performance evaluation of speech recognition systems is still a challenge because speech signals have the characteristics of diversity, complexity and uncertainty, which requires the evaluation method to have high accuracy and robustness.

[0003] Although the existing technology has made some progress, there are still the following problems in the evaluation of speech recognition systems: The evaluation methods lack unified standards and processes, resulting in insufficient comparability and reliability of the evaluation results.

[0004] Evaluation methods often only focus on the accuracy of recognition results, while ignoring performance indicators such as recognition speed and resource consumption.

[0005] When it comes to evaluating speech recognition systems in complex scenarios and noisy environments, existing methods often fail to achieve ideal evaluation results.

[0006] Therefore, there is an urgent need for artificial intelligence-based speech recognition evaluation methods to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide a speech recognition evaluation method based on artificial intelligence to solve the technical problem of insufficient reliability and accuracy in the evaluation of speech recognition systems in the prior art.

[0008] The purpose of the present invention can be achieved through the following technical solutions: The speech recognition evaluation method based on artificial intelligence is applied to the speech recognition system, and the method includes: Acquire speech data to be evaluated, wherein the speech data to be evaluated is speech data recognized by the speech recognition system; Analyzing the speech data to be evaluated to obtain a speech fluency index, a speech clarity index, and a speech accuracy index, and calculating a first speech quality index of the speech data to be evaluated based on the speech fluency index, the speech clarity index, and the speech accuracy index; Inputting the speech data to be evaluated into a speech content recognition model, outputting a predicted text of the speech data to be evaluated, and comparing and analyzing the predicted text with the annotated text to obtain a second speech quality index of the speech data to be evaluated; A speech recognition evaluation result corresponding to the speech data to be evaluated is obtained based on the first speech quality index and the second speech quality index.

[0009] Furthermore, analyzing the speech data to be evaluated to obtain the speech fluency index specifically includes the following process: Based on the speech data to be evaluated, the total number of syllables produced and the length of the corpus of the speech data to be evaluated are obtained, and the ratio of the total number of syllables produced to the length of the corpus is recorded as a speech rate feature parameter SR of the speech data to be evaluated; Based on the speech data to be evaluated, the pronunciation duration of the speech data to be evaluated is obtained, wherein the pronunciation duration is the duration of continuous pronunciation in the speech data to be evaluated, and the ratio of the total number of syllables produced to the pronunciation duration is recorded as the pronunciation speed characteristic parameter AR of the speech data to be evaluated; The number of silent pauses of the speech data to be evaluated is obtained based on the speech data to be evaluated, and the ratio of the number of silent pauses to the corpus duration is recorded as a silent pause rate feature parameter SPF of the speech data to be evaluated; Substitute the speech rate characteristic parameter SR, the pronunciation speed characteristic parameter AR and the silent pause rate characteristic parameter SPF into the speech fluency index calculation formula to calculate the speech fluency index YYS, which is as follows: ; in, is the total duration of silent pauses in the voice data to be evaluated, , are weight coefficients, their values ​​are 0.6 and 0.4 respectively, and the value of e is 2.72.

[0010] Furthermore, analyzing the speech data to be evaluated to obtain the speech intelligibility index specifically includes the following process: Based on the speech data to be evaluated, the number of correct pronunciations of phonemes in the speech data to be evaluated is M, and the number of phonemes in the speech data to be evaluated is N. Then the speech clarity index is .

[0011] Furthermore, analyzing the speech data to be evaluated to obtain the speech accuracy index specifically includes the following process: Based on the GOP algorithm, the number of correct phoneme pronunciations A in the speech data to be evaluated is identified, and the number of phonemes B in the speech data to be evaluated is obtained, then the speech accuracy index .

[0012] Further, calculating the first speech quality index of the speech data to be evaluated based on the speech fluency index, the speech clarity index and the speech accuracy index specifically includes the following process: Speech Fluency Index , Speech Intelligibility Index and speech accuracy index Substitute into the first voice quality index calculation formula to calculate the first voice quality index DYZ, the calculation formula is as follows: ; in, , , are weight coefficients, represents the speech fluency parameter, represents the speech intelligibility parameter, Represents the speech accuracy parameter.

[0013] Furthermore, the speech data to be evaluated is input into the speech content recognition model, and the predicted text of the speech data to be evaluated is output, which specifically includes the following process: Given a set of speech data samples to be tested, X, where ; The speech content recognition model consists of two parts: the encoder network f and the context network. The encoder network f consists of a five-layer convolutional network. In the structure of the five-layer convolutional network, the sizes of the convolution kernels are (10, 8, 4, 4, 4) respectively. The encoder outputs a low-frequency feature. , the hidden space after the encoder network mapping is represented by Z, where : ; Among them, m is the convolution receptive field; The context network is a convolutional neural network with the same structure as the encoding network f. It maps the hidden space into the context information space C. The output of the context network is ,in, : ; Among them, v is the convolution receptive field; Loss function based on speech content recognition model To train the speech content recognition model, the loss function It is expressed as follows: ; in, is the sequence length of the speech data sample set X to be evaluated, is the loss of noise contrast estimation, is the true probability of the speech data sample to be evaluated, is the learning parameter; Output the predicted text of the speech data to be evaluated.

[0014] Furthermore, comparing and analyzing the predicted text with the annotated text to obtain a second speech quality index of the speech data to be evaluated specifically includes the following process: Traverse each character of the predicted text and the annotated text, count the number of mismatched characters, and then the second speech quality index ; in, is the number of characters that do not match, The number of characters for the label text.

[0015] Further, obtaining a speech recognition evaluation result corresponding to the speech data to be evaluated based on the first speech quality index and the second speech quality index specifically includes the following process: The first speech quality index and the second speech quality index are added to obtain a sum value, and it is determined whether the sum value exceeds a preset threshold value. If so, the speech recognition result corresponding to the speech data to be evaluated is determined to be qualified; if not, the speech recognition result corresponding to the speech data to be evaluated is determined to be unqualified.

[0016] Compared with the existing solutions, the present invention achieves the following beneficial effects: 1. Improve evaluation efficiency and accuracy The artificial intelligence speech recognition evaluation method can automatically process and analyze speech data, significantly improving the efficiency of the evaluation. Compared with manual evaluation, automated evaluation can reduce errors caused by human factors and improve the accuracy of the evaluation. This method can conduct a comprehensive evaluation of speech from multiple dimensions such as pronunciation accuracy, completeness, fluency, etc.

[0017] 2. Reduce evaluation costs Automated evaluation methods can reduce the manpower and time costs required for manual evaluation, which is particularly important for the evaluation of large-scale voice data and can significantly reduce the operating costs of enterprises. Automated evaluation methods can quickly process large amounts of voice data and improve evaluation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0019] Figure 1 is a workflow diagram of a first artificial intelligence-based speech recognition evaluation method according to an embodiment of the present invention; Figure 2 This is a workflow diagram of a second artificial intelligence-based speech recognition evaluation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In addition, the described features, structures or characteristics may be combined in one or more example embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, well-known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0022] This embodiment provides a speech recognition evaluation method based on artificial intelligence, which is applied to a speech recognition system. Figure 1 is a flowchart of the first artificial intelligence-based speech recognition evaluation method of an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps: Step S101: Acquire speech data to be evaluated, wherein the speech data to be evaluated is speech data recognized by a speech recognition system; Step S102: Analyze the speech data to be evaluated to obtain a speech fluency index, a speech clarity index and a speech accuracy index, and calculate a first speech quality index of the speech data to be evaluated based on the speech fluency index, the speech clarity index and the speech accuracy index; Step S103: inputting the speech data to be evaluated into the speech content recognition model, outputting the predicted text of the speech data to be evaluated, and performing comparative analysis on the predicted text and the annotated text to obtain a second speech quality index of the speech data to be evaluated; Step S104: obtaining a speech recognition evaluation result corresponding to the speech data to be evaluated based on the first speech quality index and the second speech quality index.

[0023] In summary, the present invention can automatically process and analyze speech data, significantly improving the efficiency of evaluation. Compared with manual evaluation, automated evaluation can reduce errors caused by human factors and improve the accuracy of evaluation. This method can conduct a comprehensive evaluation of speech from multiple dimensions such as pronunciation accuracy, completeness, fluency, etc.

[0024] Furthermore, the present invention can reduce the manpower and time costs required for manual evaluation, which is particularly important for the evaluation of large-scale voice data and can significantly reduce the operating costs of enterprises. The automated evaluation method can quickly process large amounts of voice data and improve the efficiency of the evaluation.

[0025] In some embodiments, Figure 2 is a workflow diagram of a second artificial intelligence-based speech recognition evaluation method according to an embodiment of the present invention, such as Figure 2 As shown, analyzing the speech data to be evaluated to obtain the speech fluency index specifically includes the following steps: Step S201: based on the speech data to be evaluated, the total number of syllables produced and the length of the corpus of the speech data to be evaluated are obtained, and the ratio of the total number of syllables produced to the length of the corpus is recorded as the speech rate characteristic parameter SR of the speech data to be evaluated; Step S202: based on the speech data to be evaluated, obtaining the pronunciation duration of the speech data to be evaluated, wherein the pronunciation duration is the duration of continuous pronunciation in the speech data to be evaluated, and recording the ratio of the total number of syllables produced to the pronunciation duration as the pronunciation speed characteristic parameter AR of the speech data to be evaluated; Step S203: obtaining the number of silent pauses of the speech data to be evaluated based on the speech data to be evaluated, and recording the ratio of the number of silent pauses to the corpus duration as a silent pause rate feature parameter SPF of the speech data to be evaluated; Step S204: Substitute the speech rate characteristic parameter SR, the pronunciation speed characteristic parameter AR and the silent pause rate characteristic parameter SPF into the speech fluency index calculation formula to calculate the speech fluency index YYS. The calculation formula is as follows: ; in, is the total duration of silent pauses in the voice data to be evaluated, , are weight coefficients, their values ​​are 0.6 and 0.4 respectively, and the value of e is 2.72.

[0026] In some embodiments, analyzing the speech data to be evaluated to obtain the speech intelligibility index specifically includes the following process: Based on the speech data to be evaluated, the number of correct pronunciations of phonemes in the speech data to be evaluated is M, and the number of phonemes in the speech data to be evaluated is N. Then the speech clarity index is .

[0027] In some embodiments, analyzing the speech data to be evaluated to obtain the speech accuracy index specifically includes the following process: Based on the GOP (Goodness of Pronunciation) algorithm, the number of correct phoneme pronunciations A in the speech data to be evaluated is identified, and the number of phonemes B in the speech data to be evaluated is obtained, then the speech accuracy index .

[0028] Specifically, the number of correct pronunciations of phonemes in the speech data to be evaluated is identified based on the GOP algorithm, including: Data preparation: Get the voice data to be evaluated, which is usually stored in WAV or other audio formats.

[0029] Prepare reference text, that is, the text content corresponding to the correct pronunciation.

[0030] Speech preprocessing: Perform preprocessing operations such as framing and windowing on the speech data to facilitate subsequent feature extraction.

[0031] Extract features of speech data, such as MFCC (Mel-Frequency Cepstral Coefficients) or other features suitable for speech recognition.

[0032] Model preparation: Use a trained speech recognition model (such as a GMM-HMM or DNN-based model) to recognize phonemes in speech data.

[0033] Make sure the model can output the posterior probability of the phoneme or other relevant metrics.

[0034] GOP calculation: For each phoneme, the GOP algorithm is used to calculate its pronunciation quality.

[0035] The calculation of GOP usually involves the posterior probability of the observation under the given phoneme condition and the time boundary information of the phoneme.

[0036] Setting the threshold: According to actual needs, a GOP threshold is set to determine whether the pronunciation of the phoneme is correct.

[0037] The threshold setting can be adjusted according to different application scenarios to achieve the best recognition effect.

[0038] Count the number of correctly pronounced phonemes: Traverse all recognized phonemes and determine whether their pronunciation is correct based on the GOP value and the set threshold.

[0039] Count the number of phonemes pronounced correctly and output the result.

[0040] Further, calculating the first speech quality index of the speech data to be evaluated based on the speech fluency index, the speech clarity index and the speech accuracy index specifically includes the following process: Speech Fluency Index , Speech Intelligibility Index and speech accuracy index Substitute into the first voice quality index calculation formula to calculate the first voice quality index DYZ, the calculation formula is as follows: ; in, , , are weight coefficients, represents the speech fluency parameter, represents the speech intelligibility parameter, Represents the speech accuracy parameter.

[0041] In some embodiments, inputting the speech data to be evaluated into the speech content recognition model and outputting the predicted text of the speech data to be evaluated specifically includes the following process: Given a set of speech data samples X to be tested, where ; The speech content recognition model consists of two parts: the encoder network f and the context network. The encoder network f consists of a five-layer convolutional network. In the structure of the five-layer convolutional network, the sizes of the convolution kernels are (10, 8, 4, 4, 4) respectively. The encoder outputs a low-frequency feature. , the hidden space after the encoder network mapping is represented by Z, where : ; Among them, m is the convolution receptive field; The context network is a convolutional neural network with the same structure as the encoding network f. It maps the hidden space into the context information space C. The output of the context network is ,in, : ; Among them, v is the convolution receptive field; Loss function based on speech content recognition model To train the speech content recognition model, the loss function It is expressed as follows: ; in, is the sequence length of the speech data sample set X to be evaluated, is the loss of noise contrast estimation, is the true probability of the speech data sample to be evaluated, is the learning parameter; Output the predicted text of the speech data to be evaluated.

[0042] In some embodiments, comparing and analyzing the predicted text with the annotated text to obtain the second speech quality index of the speech data to be evaluated specifically includes the following process: Traverse each character of the predicted text and the annotated text, count the number of mismatched characters, and then the second speech quality index ; in, is the number of characters that do not match, The number of characters for the label text.

[0043] Further, obtaining a speech recognition evaluation result corresponding to the speech data to be evaluated based on the first speech quality index and the second speech quality index specifically includes the following process: The first speech quality index and the second speech quality index are added to obtain a sum value, and it is determined whether the sum value exceeds a preset threshold value. If so, the speech recognition result corresponding to the speech data to be evaluated is determined to be qualified; if not, the speech recognition result corresponding to the speech data to be evaluated is determined to be unqualified.

[0044] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0045] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0046] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0047] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only some logical function divisions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0048] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0049] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A speech recognition evaluation method based on artificial intelligence, characterized in that: Applied to a speech recognition system, the method includes: Acquire speech data to be evaluated, wherein the speech data to be evaluated is speech data recognized by the speech recognition system; Analyzing the speech data to be evaluated to obtain a speech fluency index, a speech clarity index, and a speech accuracy index, and calculating a first speech quality index of the speech data to be evaluated based on the speech fluency index, the speech clarity index, and the speech accuracy index; Inputting the speech data to be evaluated into a speech content recognition model, outputting a predicted text of the speech data to be evaluated, and comparing and analyzing the predicted text with the annotated text to obtain a second speech quality index of the speech data to be evaluated; A speech recognition evaluation result corresponding to the speech data to be evaluated is obtained based on the first speech quality index and the second speech quality index.

2. The speech recognition evaluation method based on artificial intelligence according to claim 1, characterized in that: Analyzing the speech data to be evaluated to obtain the speech fluency index specifically includes the following processes: Based on the speech data to be evaluated, the total number of syllables produced and the length of the corpus of the speech data to be evaluated are obtained, and the ratio of the total number of syllables produced to the length of the corpus is recorded as a speech rate feature parameter SR of the speech data to be evaluated; Based on the speech data to be evaluated, the pronunciation duration of the speech data to be evaluated is obtained, wherein the pronunciation duration is the duration of continuous pronunciation in the speech data to be evaluated, and the ratio of the total number of syllables produced to the pronunciation duration is recorded as the pronunciation speed characteristic parameter AR of the speech data to be evaluated; The number of silent pauses of the speech data to be evaluated is obtained based on the speech data to be evaluated, and the ratio of the number of silent pauses to the corpus duration is recorded as a silent pause rate feature parameter SPF of the speech data to be evaluated; Substitute the speech rate characteristic parameter SR, the pronunciation speed characteristic parameter AR and the silent pause rate characteristic parameter SPF into the speech fluency index calculation formula to calculate the speech fluency index YYS, which is as follows: ； in, is the total duration of silent pauses in the voice data to be evaluated, , are weight coefficients, their values ​​are 0.6 and 0.4 respectively, and the value of e is 2.

72.

3. The speech recognition evaluation method based on artificial intelligence according to claim 1, characterized in that: The speech data to be evaluated is analyzed to obtain the specific speech clarity index. The process includes: Based on the speech data to be evaluated, the number of correct pronunciations of phonemes in the speech data to be evaluated is M, and the number of phonemes in the speech data to be evaluated is N. Then the speech clarity index is .

4. The artificial intelligence-based speech recognition evaluation method according to claim 1, characterized in that: Analyzing the speech data to be evaluated to obtain the speech accuracy index specifically includes the following process: Based on the GOP algorithm, the number of correct phoneme pronunciations A in the speech data to be evaluated is identified, and the number of phonemes B in the speech data to be evaluated is obtained, then the speech accuracy index .

5. The artificial intelligence-based speech recognition evaluation method according to claim 1, characterized in that: The first speech quality index of the speech data to be evaluated is calculated based on the speech fluency index, the speech clarity index and the speech accuracy index. The process includes: Speech Fluency Index , Speech Intelligibility Index and speech accuracy index Substitute into the first voice quality index calculation formula to calculate the first voice quality index DYZ, the calculation formula is as follows: ； in, , , are weight coefficients, represents the speech fluency parameter, represents the speech intelligibility parameter, Represents the speech accuracy parameter.

6. The artificial intelligence-based speech recognition evaluation method according to claim 1, characterized in that: Inputting the speech data to be evaluated into the speech content recognition model and outputting the predicted text of the speech data to be evaluated specifically includes the following process: Given a set of speech data samples to be tested, X, where ; The speech content recognition model consists of two parts: the encoder network f and the context network. The encoder network f consists of a five-layer convolutional network. In the structure of the five-layer convolutional network, the sizes of the convolution kernels are (10, 8, 4, 4, 4) respectively. The encoder outputs a low-frequency feature. , the hidden space after the encoder network mapping is represented by Z, where : ； Among them, m is the convolution receptive field; The context network is a convolutional neural network with the same structure as the encoding network f. It maps the hidden space into the context information space C. The output of the context network is ,in, : ； Among them, v is the convolution receptive field; Loss function based on speech content recognition model To train the speech content recognition model, the loss function It is expressed as follows: ； in, is the sequence length of the speech data sample set X to be evaluated, is the loss of noise contrast estimation, is the true probability of the speech data sample to be evaluated, is the learning parameter; Output the predicted text of the speech data to be evaluated.

7. The artificial intelligence-based speech recognition evaluation method according to claim 1, characterized in that: Comparing and analyzing the predicted text with the annotated text to obtain the second speech quality index of the speech data to be evaluated specifically includes the following process: Traverse each character of the predicted text and the annotated text, count the number of mismatched characters, and then the second speech quality index ; in, is the number of characters that do not match, The number of characters for the label text.

8. The artificial intelligence-based speech recognition evaluation method according to claim 1, characterized in that: The speech recognition evaluation result corresponding to the speech data to be evaluated is obtained based on the first speech quality index and the second speech quality index. The process includes: The first speech quality index and the second speech quality index are added to obtain a sum value, and it is determined whether the sum value exceeds a preset threshold value. If so, the speech recognition result corresponding to the speech data to be evaluated is determined to be qualified; if not, the speech recognition result corresponding to the speech data to be evaluated is determined to be unqualified.

Images