Comprehensive Assessment Method for Oral English Test

Abstract

The present invention discloses a comprehensive evaluation method for oral English examinations, comprising: establishing a speech recognition model based on a continuous speech recognition algorithm with dynamic time warping, recognizing speech in the oral English examination, and obtaining speech recognition data; extracting sound features from the speech recognition data using acoustic feature extraction technology; inputting the sound features into a scoring model constructed based on a deep learning algorithm to perform speech quality analysis, and outputting a quality score for the sound features; performing grammatical analysis on the speech recognition data using natural language processing technology, and outputting a grammatical score for the speech recognition data; and summing the quality score and the grammatical score to obtain a final comprehensive score. The present invention can effectively handle differences in speaking speed, thereby improving the accuracy of speech recognition, not only enhancing the model's adaptability to oral fluency and naturalness, but also ensuring the data quality and reliability of subsequent analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition and natural language processing, and in particular to a comprehensive evaluation method for oral English examinations. Background Art

[0002] In recent years, speech recognition technology has been used to convert spoken speech into processable text data, enabling computers to understand and analyze human language. Significant progress has also been made in the field of natural language processing, especially in grammatical analysis and text understanding. The integration of these technologies has provided support for language learning, assessment and many other applications.

[0003] However, although existing technologies have achieved success in many aspects, there are still some shortcomings in the automated assessment of English oral examinations. Existing systems often rely on single-dimensional evaluation criteria, such as only evaluating the pronunciation accuracy of speech or only analyzing grammatical correctness, and lack a comprehensive assessment of the language ability of candidates. In addition, existing technologies still face challenges in dealing with the complexity of natural language, especially non-standard expressions and diverse voice changes in spoken language, which limits the reliability and fairness of the evaluation system and also affects the objectivity and accuracy of the evaluation results.

[0004] To address these issues, the present invention proposes a comprehensive method based on multi-dimensional evaluation. This method not only includes a continuous speech recognition algorithm with dynamic time warping, but also combines acoustic feature extraction and deep learning technology to analyze speech quality. In this way, multiple aspects of speech, such as pronunciation, intonation, fluency, and grammatical correctness, are comprehensively evaluated. In addition, grammatical analysis of the text of speech transcription is performed through natural language processing technology, which can provide a more accurate and comprehensive score. Summary of the Invention

[0005] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0006] In view of the above existing problems, the present invention is proposed.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: a speech recognition model is established based on a continuous speech recognition algorithm using dynamic time warping to recognize speech in an oral English test and obtain speech recognition data;

[0008] Extracting sound features from the speech recognition data using acoustic feature extraction technology;

[0009] Inputting the voice features into a scoring model built based on a deep learning algorithm to perform voice quality analysis, and outputting a quality score for the voice features;

[0010] Performing grammatical analysis on the speech recognition data using natural language processing technology, and outputting a grammatical score for the speech recognition data;

[0011] The quality score and the grammatical score are added together to obtain a final comprehensive score.

[0012] As a preferred embodiment of the comprehensive evaluation method for oral English examinations described in the present invention, the speech recognition data includes but is not limited to vocabulary, sentences, pronunciation, intonation, stress, speaking speed, and grammatical structure of the complete speech transcription of the examinee.

[0013] As a preferred solution of the comprehensive evaluation method for oral English examination described in the present invention, obtaining the speech recognition data includes preprocessing, feature extraction, dynamic time warping, speech recognition and text transcription output.

[0014] As a preferred embodiment of the comprehensive evaluation method for oral English test of the present invention, the preprocessing collects speech data from the examinee's oral test, uses noise elimination technology to remove background noise and improve speech clarity, and enhances the high frequency part of the speech signal by filtering to improve speech recognizability;

[0015] The feature extraction process divides the speech signal into short time frames to capture the short-term characteristics of the speech, applies a window function to each frame to reduce the discontinuity of the frame edge, and then extracts the features of each frame;

[0016] Dynamic Time Warping uses a computer program to adaptively create a feature template and uses the DTW algorithm to align the feature sequence of the real-time speech with the template feature sequence to find the best matching path. DTW adjusts the speed and rhythm of the time sequence to match the changes in speech and adapt to the speed differences of different speakers.

[0017] The speech recognition and output is based on the DTW matching results, uses a classifier to recognize words or phrases in the speech, and converts the recognition results into text output, that is, a text record of the examinee's spoken content.

[0018] As a preferred embodiment of the comprehensive evaluation method for oral English examinations according to the present invention, the speech recognition model can be represented by the following formula:

[0019]

[0020] Among them, R is the recognition result, is the set of recognized languages or keywords, f k(x) is the kth feature extracted from the input speech x, such as frequency, Mel-frequency cepstral coefficients, intonation, speaking speed, and stress, is the kth feature in the language or keyword The mean under σ is the kth feature in the language or keyword The variance under g j (x) is the jth feature extracted from the input speech x, such as letters, words, is the jth feature in the language or keyword The mean under is the jth feature in the language or keyword The variance under .

[0021] As a preferred embodiment of the comprehensive evaluation method for oral English examinations of the present invention, the mathematical expression formula of the sound features is as follows:

[0022]

[0023] Pitch(t)=Cepstrum(X real )

[0024]

[0025] Among them, MFCC(t) is the calculated Mel frequency cepstral coefficient, Intensity(t) is the overall sound intensity calculated by taking the logarithm of the spectrum energy, Formants(t) is the resonance peak determined by searching the local maximum of the second-order derivative of the spectrum, SpeechRate(t) is the speaking rate obtained by calculating the zero crossing rate per unit time, which is an indicator of speech fluency, ZeroCrossingRate(t) is used to directly calculate the number of zero crossings of the speech signal, which is used to measure the frequency component of the signal, t is the time series of the input speech data, X(k) is the spectrum obtained by applying the fast Fourier transform to t, Mel(k) is the effect of the Mel filter, Cepstrum(X real ) is the real spectrum X real The fundamental frequency Pitch(t), Peaks obtained by applying the cepstrum method It represents the local maximum of the function above the threshold θ, identifying the resonance peak. ZeroCrossing(t) is a function that calculates the zero crossing rate in the time series t, where T is the total duration of the time series.

[0026] As a preferred solution of the comprehensive evaluation method for oral English examinations described in the present invention, the sound features include Mel-frequency cepstral coefficients, fundamental frequency, pitch, sound intensity, formant, speech rate, and zero-crossing rate.

[0027] As a preferred embodiment of the comprehensive evaluation method for oral English examinations of the present invention, the mathematical expression formula for performing the quality scoring by the scoring model is as follows:

[0028]

[0029] Among them, Q(z) is the output speech quality score, z is the input sound feature vector set, z i is the i-th sound feature vector, ω i is the weight, b is the bias term, obtained through learning, δ is the activation function, used to convert the output into a value in the range of [0,1], h i (z i ) is a nonlinear transformation function applied to each sound feature vector.

[0030] As a preferred embodiment of the comprehensive evaluation method for oral English examinations of the present invention, the mathematical expression of the grammar score is as follows:

[0031]

[0032] Among them, G(S) is the grammatical score obtained for the speech recognition data S, ParseError i (S) is the number of correct grammatical analyses of the i-th sentence, M is the number of grammatically correct categories, β is the bias term for adjusting the sensitivity of the score, and α i is the weight associated with the correct type of i, ∈ is a constant used to ensure that the denominator is non-zero, and τ is a scaling factor used to adjust the slope of the scoring function.

[0033] Beneficial effects of the present invention:

[0034] 1. The DTW algorithm (Dynamic Time Warping) effectively handles differences in speaking speed by adjusting time series to match speech data of varying speeds and rhythms, thereby improving speech recognition accuracy. This not only enhances the model's ability to adapt to spoken fluency and naturalness, but also ensures data quality and reliability for subsequent analysis.

[0035] 2. Using acoustic feature extraction techniques, such as Mel-Frequency Cepstral Coefficients (MFCCs), fundamental frequency, and pitch, key acoustic parameters are extracted from speech recognition data. These features are crucial for understanding the fundamental properties of speech and the emotion and intonation it conveys. By accurately capturing these acoustic features, the scoring model can conduct a more in-depth analysis of speech quality, providing data support for evaluating speech clarity and professionalism.

[0036] 3. The application of natural language processing technology enables the system to perform grammatical analysis on the text obtained from speech conversion and output a grammatical score for the speech recognition data, which strengthens another important dimension of evaluation, namely grammatical correctness. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0038] Figure 1 Schematic diagram of the flow of the comprehensive evaluation method for oral English examination shown in the present invention. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0040] Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without making any creative work should fall within the scope of protection of the present invention.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0042] Example 1

[0043] According to an embodiment of the present invention, Figure 1 The flowchart shown is a comprehensive evaluation method for an oral English test, which specifically includes the following steps:

[0044] S1. Establish a speech recognition model based on the continuous speech recognition algorithm of dynamic time warping to recognize the speech in the English oral test and obtain speech recognition data. Among them, the following points need to be explained in this step:

[0045] Speech recognition data includes but is not limited to the vocabulary, sentences, pronunciation, intonation, stress, speaking speed, and grammatical structure of the candidate's complete speech transcription.

[0046] It should be noted that the complete voice transcription of the candidate includes the written record of all the words and sentences spoken by the candidate, which is the basis for evaluating language use and composition. The candidate's spoken language is recorded in writing through the text transcription application.

[0047] Through pronunciation and intonation analysis, the test-taker's pronunciation accuracy (i.e. the similarity between the test-taker's spoken pronunciation and the standard pronunciation) is assessed, including the correct pronunciation of phonemes, the appropriate use of stress and intonation, and whether the test-taker can accurately imitate the rhythm and prosody of English.

[0048] As an example, the correctness and complexity of grammar are analyzed through grammatical structure analysis, including sentence structure, tense usage, and grammatical consistency.

[0049] For example, by analyzing intonation and stress, we can understand the emotional expression and intonation changes of the voice to evaluate the candidate's expression and communication skills.

[0050] Furthermore, obtaining speech recognition data includes the following steps:

[0051] (1) Pretreatment:

[0052] Collect voice data from candidates during oral exams;

[0053] Use noise cancellation technology to remove background noise and improve speech clarity;

[0054] By filtering, the high-frequency part of the speech signal is enhanced to improve the intelligibility of the speech.

[0055] (2) Feature extraction:

[0056] Segment the speech signal into short time frames, such as 20 to 30 milliseconds per frame, to capture the short-term characteristics of speech;

[0057] Apply a window function, such as a Hanning window, to each frame to reduce discontinuities at the frame edges;

[0058] Extract features of each frame, such as Mel-frequency cepstral coefficients (MFCC), letters, and words.

[0059] (3) Dynamic time warping:

[0060] Adaptively create feature templates through the application of a computer program;

[0061] Use the DTW algorithm to align the feature sequence of real-time speech and the feature sequence of template to find the best matching path;

[0062] DTW adapts the speed and rhythm of the time series to match the changes in speech and adapts to the speed differences between different speakers.

[0063] (4) Speech recognition and output:

[0064] Based on the DTW matching results, a classifier (such as a Hidden Markov Model HMM) is used to identify words or phrases in the speech.

[0065] The recognition results are converted into text output, that is, a text record of the examinee's spoken content.

[0066] Preferably, this step enhances the high-frequency part of the speech signal by filtering, which not only improves the recognizability of the speech, but also effectively improves the accuracy of subsequent feature extraction. It has a significant effect on coping with various speaking speeds and pronunciation habits in oral exams, ensuring the high quality of data input.

[0067] In an optional embodiment, the basic properties of speech are captured by using Mel-frequency cepstral coefficients (MFCCs). This is because MFCCs can effectively represent the short-term energy dynamics and spectral envelope of speech, which is particularly critical for recognizing subtle differences in speech. In addition, the application of framing and window functions reduces edge effects in signal processing and ensures the continuity and consistency of features.

[0068] Preferably, because different examinees have different speaking speeds and pause habits, traditional speech recognition algorithms often have difficulty in accurately matching them. By adjusting the speed and rhythm of the time series to match the changes in speech, DTW enables the system to adapt to the differences in speaking speed between different speakers.

[0069] Specifically, the speech recognition model can be expressed as follows:

[0070]

[0071] Among them, R is the recognition result, is the set of recognized languages or keywords, f k (x) is the kth feature extracted from the input speech x, such as frequency, Mel-frequency cepstral coefficients, intonation, speaking speed, and stress, is the kth feature in the language or keyword The mean under σ is the kth feature in the language or keyword The variance under g j (x) is the jth feature extracted from the input speech x, such as letters, words, is the jth feature in the language or keyword The mean under is the jth feature in the language or keyword The variance under .

[0072] As an example, There are two languages ​​recognized in the , namely Chinese and English.

[0073] As an example, the voice recognition data is stored in a corresponding single room according to the candidate's test number, that is, multiple independent storage grids are formed in the cloud database. The storage grid is the storage space, and each grid only stores the voice data of the candidate with the corresponding test number to avoid confusion and omissions in calculation.

[0074] In an optional embodiment, this step can also identify abnormal speech during the speech recognition process, such as Chinese (Chinese characters, sentences), singing (tones, vocabulary), and abusive words. By identifying such abnormal speech, the efficiency of data processing is improved for subsequent scoring calculations.

[0075] Preferably, the separate storage solution adopted in the embodiment of the present invention is to independently store the voice recognition data according to the examinee's test number, which effectively avoids confusion and omissions when retrieving data. This storage solution is particularly suitable for the English oral examination environment, ensuring the security and privacy of the data while improving the efficiency of data processing.

[0076] S2. Extract the sound features from the speech recognition data using acoustic feature extraction technology. Note that:

[0077] The spectrum is obtained through fast Fourier transform (FFT), and then refined calculations are performed based on the definitions of different features. For example, the fundamental frequency and pitch are calculated through the auto-correlation function or cepstrum method, and the resonance peaks are obtained by finding the significant peaks of the spectrum. The extracted features are normalized to have a unified parameter form to facilitate subsequent model training and accurate recognition of the classifier.

[0078] Furthermore, the mathematical formula for calculating the sound characteristics is as follows:

[0079]

[0080] Pitch(t)=Cepstrum(X real )

[0081]

[0082] Among them, MFCC(t) is the calculated Mel frequency cepstral coefficient, Intensity(t) is the overall sound intensity calculated by taking the logarithm of the spectrum energy, Formants(t) is the resonance peak determined by searching the local maximum of the second-order derivative of the spectrum, SpeechRate(t) is the speaking rate obtained by calculating the zero crossing rate per unit time, which is an indicator of speech fluency, ZeroCrossingRate(t) is used to directly calculate the number of zero crossings of the speech signal, which is used to measure the frequency component of the signal, t is the time series of the input speech data, X(k) is the spectrum obtained by applying the fast Fourier transform to t, Mel(k) is the effect of the Mel filter, Cepstrum(X real ) is the real spectrum X real The fundamental frequency Pitch(t), Peaks obtained by applying the cepstrum method It represents the local maximum of the function above the threshold θ, identifying the resonance peak. ZeroCrossing(t) is a function that calculates the zero crossing rate in the time series t, where T is the total duration of the time series.

[0083] As examples, the sound features include Mel-frequency cepstral coefficients, fundamental frequency, pitch, sound intensity, formant, speech rate, and zero-crossing rate.

[0084] In order to better illustrate the advantages of using the above data as sound features in this embodiment, this embodiment illustrates them as follows:

[0085] Mel-frequency cepstral coefficients are one of the most commonly used features in speech recognition. They are used to capture the envelope of the short-term power spectrum of speech. Based on the human ear's nonlinear perception of sounds of different frequencies, they perform a Mel-scale transformation on the frequency. This makes them suitable for capturing language content and an essential feature.

[0086] Fundamental frequency is the basic frequency of a speaker's voice and is a key parameter for distinguishing different intonations and voice tones; pitch variation can be used to analyze the rising and falling tones of sentences and is very important for understanding emotions and intentions.

[0087] The energy of an audio signal is expressed as the loudness of the sound. In spoken language, the dynamically changing sound intensity can reflect the speaker's emotional state and the words they emphasize.

[0088] Formants are regions of frequency enhancement created by the shape of the vocal tract that are particularly helpful in identifying vowels.

[0089] Speech rate involves measures of speech rate, including words or syllables per minute, which can reflect a speaker's fluency and spoken language complexity.

[0090] The zero-crossing rate is the frequency with which the signal changes between positive and negative amplitudes and can be used to distinguish speech from non-speech segments, especially against noisy backgrounds.

[0091] Therefore, this embodiment selects the above attribute data as sound features to improve the accuracy and efficiency of subsequent model evaluation calculations.

[0092] S3. Input the voice features into the scoring model built based on the deep learning algorithm to perform voice quality analysis and output a quality score for the voice features. The mathematical expression of the scoring model is as follows:

[0093]

[0094] Among them, Q(z) is the output speech quality score, z is the input sound feature vector set, z i is the i-th sound feature vector, ω i is the weight, b is the bias term, obtained through learning, δ is the activation function, used to convert the output into a value in the range of [0,1], h i (z i ) is a nonlinear transformation function applied to each sound feature vector.

[0095] Q(z) is the output speech quality score, which takes a continuous value in the range [0,1]. This continuous score can capture subtle changes in speech quality more carefully than the traditional discrete rating system, thereby providing more accurate feedback for the improvement of speech technology.

[0096] h i (z i ) are nonlinear transformation functions applied to each sound feature. The purpose of these functions is to improve the responsiveness and robustness of the model to various sound features. Through normalization, exponential or other complex mathematical functions, the consistency and high quality of the model input are ensured, so that the speech quality can be accurately predicted.

[0097] b is the bias term, which is used to adjust the output baseline of the model when there is no input. In practical applications, the bias term can help the model better adapt to any systematic biases that may exist in the data.

[0098] It should be noted that this embodiment uses a deep learning model to perform voice quality scoring, which has significant advantages over traditional methods, including but not limited to higher scoring accuracy, the ability to automatically learn and adapt to new voice patterns from big data, and efficient processing of complex and nonlinear relationships.

[0099] S4. Perform grammatical analysis on the speech recognition data using natural language processing technology and output a grammatical score for the speech recognition data. It should be noted that:

[0100] Segment the transcribed text and assign appropriate part-of-speech tags to each word, such as noun, verb, or adjective;

[0101] Dependency parsing technology is used to analyze the dependency relationships between words in a sentence, such as the subject-verb-object structure, to evaluate the grammatical correctness of the sentence;

[0102] Classify and label types of grammatical correctness, such as correct subject-verb agreement, correct tense, and correct word order;

[0103] A fixed score is assigned to each correct type, such as 2 points for correct subject-verb agreement, 1 point for correct tense, and 1 point for correct word order;

[0104] The final grammar score is calculated based on the number and type of correct words detected, and its mathematical expression is as follows:

[0105]

[0106] Among them, G(S) is the grammatical score obtained for the speech recognition data S, ParseError i (S) is the number of correct grammatical analyses of the i-th sentence, M is the number of grammatically correct categories, β is the bias term for adjusting the sensitivity of the score, and α i is the weight associated with the i-th correct type, ∈ is a constant used to ensure that the denominator is non-zero, and τ is a scaling factor used to adjust the slope of the scoring function.

[0107] It should be noted that this step matches the candidate's test answers with the standard answers stored in the database to judge the degree of consistency of the candidate's test answers, and then to know the accuracy of the test answers. For example, the above-mentioned grammatical correctness analysis method can know whether the grammatical structure in the candidate's test answers is consistent with the grammatical structure in the standard answer. If they are consistent, points will be awarded, otherwise no points will be awarded.

[0108] Furthermore, for similarity matching between the examinee's test answers and the standard answers, in addition to the aforementioned grammatical analysis, similarity matching of vocabulary and pronunciation can also be performed. The matching method is the same as the aforementioned grammatical analysis method, and will not be described in detail in this example.

[0109] As an example, the matching method can be implemented by using the text similarity technology and fuzzy matching algorithm in the prior art, which will not be described in detail in this example.

[0110] S5. Add the quality score and grammar score to get the final comprehensive score.

[0111] As an example, the comprehensive score is based on a 10-point scale and is divided into four score ranges to better judge the candidate's English speaking test results, including:

[0112] Scores in the range of [8,10] are considered excellent;

[0113] Scores in the range of [6,8] are considered good;

[0114] Scores within the range of [5,6] are considered passing;

[0115] Scores in the range [0,5] are considered failing.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A comprehensive evaluation method for oral English examination, characterized in that: include: A speech recognition model is established based on the continuous speech recognition algorithm of dynamic time warping to recognize the speech in the English oral test and obtain speech recognition data; The speech recognition model is represented by the following formula: Among them, R is the recognition result, is the set of recognized languages or keywords, f k (x) is the kth feature extracted from the input speech x, such as frequency, Mel-frequency cepstral coefficients, intonation, speaking speed, and stress, is the kth feature in the language or keyword The mean under σ is the kth feature in the language or keyword The variance under g j (x) is the jth feature extracted from the input speech x, such as letters, words, is the jth feature in the language or keyword The mean under is the jth feature in the language or keyword Variance under ; The acoustic feature extraction technology is used to extract the sound features in the speech recognition data; the mathematical expression formula of the sound features is as follows: Pitch(t)=Cepstrum(X real ) Among them, MFCC(t) is the calculated Mel frequency cepstral coefficient, Intensity(t) is the overall sound intensity calculated by taking the logarithm of the spectrum energy, Formants(t) is the resonance peak determined by searching the local maximum of the second-order derivative of the spectrum, SpeechRate(t) is the speaking rate obtained by calculating the zero crossing rate per unit time, which is an indicator of speech fluency, ZeroCrossingRate(t) is used to directly calculate the number of zero crossings of the speech signal, which is used to measure the frequency component of the signal, t is the time series of the input speech data, X(k) is the spectrum obtained by applying the fast Fourier transform to t, Mel(k) is the effect of the Mel filter, Cepstrum(X real ) is the real spectrum X real The fundamental frequency Pitch(t) obtained by applying the cepstrum method, It represents the local maximum of the function above the threshold θ, identifying the resonance peak. ZeroCrossing(t) is the function that calculates the zero crossing rate in the time series t, where T is the total duration of the time series. Inputting the voice features into a scoring model built based on a deep learning algorithm to perform voice quality analysis, and outputting a quality score for the voice features; The mathematical expression formula for the quality score through the scoring model is as follows: Among them, Q(z) is the output speech quality score, z is the input sound feature vector set, z i is the i-th sound feature vector, ω i is the weight, b is the bias term, obtained through learning, δ is the activation function, used to convert the output into a value in the range of [0,1], h i (z i ) is a nonlinear transformation function applied to each sound feature vector; Performing grammatical analysis on the speech recognition data using natural language processing technology, and outputting a grammatical score for the speech recognition data; The quality score and the grammatical score are added together to obtain a final comprehensive score.

2. The comprehensive evaluation method for oral English examination according to claim 1, characterized in that: The speech recognition data includes but is not limited to the vocabulary, sentences, pronunciation, intonation, stress, speaking speed, and grammatical structure of the complete speech transcription of the examinee.

3. The comprehensive evaluation method for oral English examination according to claim 1 or 2, characterized in that: Obtaining the speech recognition data includes preprocessing, feature extraction, dynamic time warping, speech recognition and text transcription output.

4. The comprehensive evaluation method for oral English examination according to claim 3, characterized in that: The preprocessing collects voice data from the examinee's oral test, uses noise cancellation technology to remove background noise, improves voice clarity, and enhances the high-frequency part of the voice signal through filtering to improve voice recognizability; The feature extraction process divides the speech signal into short time frames to capture the short-term characteristics of the speech, applies a window function to each frame to reduce the discontinuity of the frame edge, and then extracts the features of each frame; Dynamic Time Warping uses a computer program to adaptively create a feature template and uses the DTW algorithm to align the feature sequence of the real-time speech with the template feature sequence to find the best matching path. DTW adjusts the speed and rhythm of the time sequence to match the changes in speech and adapt to the speed differences of different speakers. The speech recognition and output is based on the DTW matching results, uses a classifier to recognize words or phrases in the speech, and converts the recognition results into text output, that is, a text record of the examinee's spoken content.

5. The comprehensive evaluation method for oral English examination according to claim 1, characterized in that: The sound features include Mel-frequency cepstral coefficients, fundamental frequency, pitch, sound intensity, formant, speech rate, and zero-crossing rate.

6. The comprehensive evaluation method for oral English examination according to claim 1, characterized in that: The mathematical expression of the grammar score is as follows: Among them, G(S) is the grammatical score obtained for the speech recognition data S, ParseError i (S) is the number of correct grammatical analyses of the i-th sentence, M is the number of grammatically correct categories, β is the bias term for adjusting the sensitivity of the score, and α i is the weight associated with the i-th correct type, ∈ is a constant used to ensure that the denominator is non-zero, and τ is a scaling factor used to adjust the slope of the scoring function.

Images