Human-computer interaction oral evaluation method and system based on speech recognition

Abstract

The present invention relates to the field of data processing technology, and more specifically, to a human-computer interactive oral language assessment method and system based on speech recognition, the method comprising: obtaining a phoneme sequence of a target candidate's speech signal, calculating an initial segmentation probability of a target moment within a time window of the target phoneme, taking a time window between the target moment and the end moment of the target phoneme as a first time period, taking a time window between a start moment of a next phoneme and any moment of the next phoneme as a second time period, calculating a similarity overlap between the sound intensity of the first time period and the second time period, calculating a final segmentation probability of the target moment based on the initial segmentation probability and the similarity overlap, and using the final segmentation probability to screen a segmentation point to segment the phoneme sequence, so as to obtain a matching score between the segmented phoneme sequence and a pre-stored standard phoneme sequence based on a GOP algorithm. The present invention can improve the accuracy of the obtained matching score, thereby ensuring the accuracy of the oral language assessment result.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a human-computer interactive spoken language assessment method and system based on speech recognition. Background Art

[0002] With the growing demand for foreign language education, especially English learning, traditional manual scoring can no longer meet the requirements of rapid evaluation and large-scale testing. Therefore, oral assessment based on speech recognition technology has emerged and has become a key assessment tool in the education, examination and training industries. The GOP algorithm is a technology used for phoneme matching and pronunciation assessment. It mainly evaluates pronunciation quality by comparing the matching degree between recognized phonemes and standard phonemes. Therefore, the GOP algorithm can be used to evaluate oral assessment results.

[0003] However, in the oral assessment process, individual differences (such as speaking too fast, too slow, and pauses) often lead to significant differences between the recognized phoneme sequence and the standard phoneme sequence on the time axis. When using the GOP algorithm to evaluate pronunciation quality, it may not be possible to accurately align the phoneme sequence with the standard phoneme sequence, thus affecting the quality assessment results. Therefore, before using the GOP algorithm to evaluate pronunciation quality, the recognized phoneme sequence can be segmented to eliminate the impact of individual differences, so that the recognized phoneme sequence can be accurately aligned with the standard phoneme sequence to ensure the accuracy of the assessment results.

[0004] In the related art, a text segmentation method, apparatus, device and storage medium are disclosed in a patent application document with publication number CN112686018A. The method includes: obtaining a recognized text of a target speech of a target speaker as a target text to be segmented; obtaining speech pause features of the target speaker according to the target speech and the target text, wherein the speech pause features of the target speaker reflect the rhythmic changes of the target speaker; and segmenting the target text according to the speech pause features of the target speaker.

[0005] However, the above scheme only considers the speech pause feature when segmenting the target text. However, when obtaining the recognition text of the target speech, the automatic speech recognition system may mistakenly recognize adjacent speech signals with similar pronunciation as the same speech signal, thereby affecting the accuracy of the recognition text. This results in the low accuracy of the speech pause feature extracted based on the target speech and target text, and thus the segmentation probability cannot be accurately determined, which ultimately affects the accuracy of the segmentation result. Summary of the invention

[0006] In order to solve the problem of low accuracy of oral evaluation results due to the inability to accurately segment the acquired phoneme sequence, the present invention provides a human-computer interactive oral evaluation method and system based on speech recognition.

[0007] According to a first aspect of the present invention, a method for evaluating spoken language in human-computer interaction based on speech recognition is provided, comprising:

[0008] Acquire the phoneme sequence of the target examinee's speech signal during the oral test;

[0009] Select the target phoneme in the phoneme sequence and calculate the initial segmentation probability at the target time within the time window of the target phoneme: ; For the The time window of the phoneme The initial segmentation probability at the moment; is the entropy value of the sound intensity within the time window; , and The first moment, The moment and The intensity of the sound at a given moment; , Respectively The start and end times of each phoneme; is the normalization function; is the time interval between any two moments;

[0010] The period consisting of the target moment and the end moment of the target phoneme is taken as the first period, and the period consisting of the start moment of the next phoneme and any moment in the time window of the next phoneme is taken as the second period. The similarity overlap of the sound intensity of the first period and the second period is calculated. The similarity overlap is negatively correlated with the difference in average sound intensity and duration between the first period and the second period.

[0011] The final segmentation probability at the target moment is calculated. The final segmentation probability is positively correlated with the similarity overlap and the initial segmentation probability. The final segmentation probability is used to screen the segmentation points to segment the phoneme sequence, so as to obtain the matching score between the segmented phoneme sequence and the pre-stored standard phoneme sequence based on the GOP algorithm.

[0012] When calculating the final segmentation probability, the present invention takes into account the fluctuation of the sound intensity within a single phoneme, that is, the calculation process of the initial segmentation probability, and also takes into account the transition characteristics between different phonemes, that is, the calculation process of the similarity overlap, thereby ensuring the accuracy of the final segmentation probability, so as to achieve accurate segmentation of the phoneme sequence based on the final segmentation probability, so that the GOP algorithm can be used to obtain accurate matching scores and realize accurate evaluation of spoken language.

[0013] Preferably, the difference between the duration of the first time period and the duration of the second time period is a comprehensive difference, and a method for obtaining the comprehensive difference includes:

[0014] The length of the time window of the target phoneme and the length of the time window of the next phoneme are obtained and summed. The absolute value of the difference between the length of the first period and the length of the second period and the ratio of the obtained cumulative sum are used as the comprehensive difference between the lengths of the first period and the second period.

[0015] The present invention utilizes the feature that the boundary between the rear or front position and the adjacent phoneme in the time window of any phoneme is relatively fuzzy, and can accurately measure the fuzziness of the boundary between the first time period corresponding to the target moment and the second time period corresponding to any moment in the time window of the next phoneme.

[0016] Preferably, the similarity overlap between the sound intensity of the first time period and the sound intensity of the second time period satisfies the following relationship:

[0017] ;

[0018] In the formula, For the The first phoneme The sound intensity of the first period corresponding to the moment is The first phoneme The similarity overlap of the sound intensity of the second time period corresponding to the moment, where The phoneme is The next phoneme of a phoneme; is the average sound intensity during the first period; is the average sound intensity during the second period; is the absolute value symbol; is the combined difference between the duration of the first period and the second period; is a natural exponential function.

[0019] The present invention can provide another dimension of data basis for determining the possibility of the existence of a boundary at a target moment by evaluating the transition characteristics between phonemes.

[0020] Preferably, the final segmentation probability at any time within the time window of any phoneme satisfies the following relationship:

[0021] ;

[0022] In the formula, For the The time window of the phoneme The final segmentation probability at the moment; For the The time window of the phoneme The initial segmentation probability at the moment; For the The first phoneme The sound intensity of the first period corresponding to the moment is The first phoneme The similarity overlap of the sound intensity of the second time period corresponding to the moment; For the The total number of moments in the time window of a phoneme; is the normalization function.

[0023] The present invention can accurately evaluate the possibility of the existence of a boundary at each moment, thereby ensuring the accuracy of determining the segmentation point.

[0024] Preferably, the moment in the time window of any phoneme is the temporal index of each frame signal in the time window of the phoneme.

[0025] Preferably, using the final segmentation probability to select the segmentation point to segment the phoneme sequence includes:

[0026] A preset segmentation threshold is obtained. If the final segmentation probability at any moment in the time window of any phoneme is greater than the segmentation threshold, the moment is taken as a segmentation point to segment the phoneme sequence at all segmentation points to obtain a segmented phoneme sequence.

[0027] Preferably, before converting the speech signal of the target examinee into a phoneme sequence, the method further comprises:

[0028] The target examinee's speech signal is preprocessed to filter out noise and silence in the speech signal.

[0029] The present invention can reduce the amount of data to be processed and avoid the influence of noise.

[0030] According to a second aspect of the present invention, a human-computer interaction oral assessment system based on speech recognition is provided. The system includes a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement the steps of the first aspect of the present invention.

[0031] The present invention has the following effects:

[0032] 1. The present invention can achieve accurate segmentation of phoneme sequences and eliminate the influence of individual differences. Therefore, when using the GOP algorithm to obtain the matching score between the recognized phoneme sequence and the pre-stored standard phoneme sequence, it can ensure the accurate alignment between the phoneme sequence and the pre-stored standard phoneme sequence, thereby ensuring the accuracy of the obtained matching score, thereby achieving accurate evaluation of spoken language.

[0033] 2. When calculating the final segmentation probability, the present invention takes into account both the dynamic change characteristics of the sound intensity within a single phoneme and the transition characteristics between adjacent phonemes, thereby being able to accurately evaluate the possibility of the existence of boundaries at each moment in the time window of each phoneme in the phoneme sequence, providing an accurate data basis for determining the segmentation point. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0035] Figure 1 It is a schematic diagram of the steps of the human-computer interaction oral evaluation method based on speech recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0037] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0038] Reference Figure 1 The human-computer interaction oral evaluation method based on speech recognition includes steps S1 to S5, which are as follows:

[0039] S1: Acquire the phoneme sequence of the target candidate's speech signal during the oral test.

[0040] Specifically, a microphone can be used to collect the voice (analog signal) of a candidate who is taking an oral test, that is, the target candidate, and then sampled using an analog-to-digital converter (ADC). For example, the sampling frequency is set to 1kHz, that is, 1000 samples are sampled per second to obtain discrete sampling values. After that, the sampling values ​​are linearly quantized using 16 bits, the sampling values ​​are converted into discrete digital values, and curve fitting is performed to obtain the voice signal of the target candidate. This embodiment does not specifically limit the size of the sampling frequency. It should be noted that the process of curve fitting of data is a prior art and is not described in detail in this embodiment.

[0041] In an exemplary embodiment of the present invention, before converting the speech signal of the target examinee into a phoneme sequence, the following steps are further included:

[0042] The target examinee's speech signal is preprocessed to filter out noise and silence in the speech signal.

[0043] It should be noted that during the oral assessment, although the environmental noise interference is small, due to the long assessment time, the electronic components that collect sound may generate noise due to long-term operation, which will affect the accuracy of speech recognition. In addition, the target candidates may have silent parts during the interactive assessment, which will lead to an increase in the amount of data. Therefore, it is necessary to pre-process the speech signal to ensure the accuracy of speech recognition.

[0044] Specifically, the speech signal of the target candidate can be denoised using spectral subtraction, Wiener filtering, etc., and the silent part in the speech signal can be removed using endpoint detection method, zero crossing rate method, etc. Then, the speech signal with noise and silent parts filtered out can be framed and windowed to obtain a preprocessed speech signal, so that the phoneme sequence of the target candidate can be obtained based on the preprocessed speech signal.

[0045] Furthermore, after the preprocessed speech signal is obtained, the preprocessed speech signal can be input into an automatic speech recognition system (ASR) to output a corresponding phoneme sequence.

[0046] For example, if the ASR system recognizes the speech signal "I am learning", the ASR system will output , thereby obtaining the phoneme sequence of the input speech signal.

[0047] S2: Select a target phoneme in the phoneme sequence and calculate the initial segmentation probability at the target time within the time window of the target phoneme.

[0048] It should be noted that due to individual differences in the oral assessment process (such as speaking too fast, too slow, pauses), etc., the phoneme sequence output by the ASR system and the standard phoneme sequence have significant differences on the time axis, and due to the existence of individual differences, the ASR system may divide several adjacent phonemes into one phoneme, resulting in the inability to accurately compare the phoneme sequence output by the ASR system with the standard phoneme sequence, thereby affecting the accuracy of speech recognition. Therefore, in order to ensure the accuracy of the comparison results, the present invention evaluates the possibility of the existence of boundaries at each moment, so that the segmentation points can be selected based on the possibility of the existence of boundaries at each moment, and the recognized phoneme sequence is segmented based on the segmentation points.

[0049] The target phoneme refers to any phoneme in the phoneme sequence; the target moment refers to any moment in the time window of the target phoneme; the time window refers to the duration of any phoneme, which can be obtained by the ASR system. It should be noted that for each phoneme, within its time window, the audio signal is usually divided into several frames, each frame signal contains an amplitude value, and in the present invention, the amplitude value corresponding to each frame signal is used as the sound intensity at the corresponding moment.

[0050] In an exemplary embodiment of the present invention, the time in the time window of any phoneme is the time index of each frame signal in the time window of the phoneme. The first phoneme in the time window The moment is the first moment in the time window of the phoneme. The time point corresponding to the frame signal is recorded as .

[0051] The initial segmentation probability refers to a parameter determined based on the fluctuation of the sound intensity within the time window of any phoneme, which is used to reflect the possibility of the existence of a boundary within the time window of any phoneme. It should be noted that when the sound intensity within the time window of any phoneme fluctuates greatly, it means that the phoneme may have more complex dynamic changes, such as different pronunciation components (such as the plosive or fricative sounds of consonants, intonation changes), and different pronunciation components will blur the boundaries between phonemes, causing the ASR system to recognize adjacent phonemes as the same phoneme. Therefore, when the sound intensity within the time window of any phoneme fluctuates more violently, the possibility of the existence of a boundary within the time window of the phoneme is relatively large; on the contrary, when the sound intensity within the time window of any phoneme is relatively stable, it means that the boundary of the phoneme is relatively clear, and the probability of boundary changes within the time window of the phoneme is relatively low, and the corresponding possibility of the existence of a boundary within the time window of the phoneme is relatively small.

[0052] Specifically, the initial segmentation probability at any time within the time window of any phoneme satisfies the following relationship:

[0053] ;

[0054] In the formula, For the The time window of the phoneme The initial segmentation probability at the moment; , and The first moment, The moment and The intensity of the sound at a given moment; is the entropy value of the sound intensity in the time window. It should be noted that the process of determining the entropy value of the data sequence is a prior art and will not be described in detail in this embodiment. , Respectively The start and end times of each phoneme; is the normalization function; is the time interval between any two moments;

[0055] in, Reflects the The overall fluctuation degree of sound intensity within the time window of a phoneme. The larger the value, the more drastic the fluctuation of sound intensity within the time window of the phoneme, which further indicates that there may be different pronunciation components in the time window. The possibility of the existence of boundaries in the time window is relatively large, and the corresponding initial segmentation probability at each moment in the time window of the phoneme is relatively large.

[0056] Reflects the The time window of the phoneme The larger the value is, the more drastic the change in the sound intensity of the adjacent moments of the moment is, and the possibility of the existence of a boundary at this moment is relatively high.

[0057] Reflects the The length of the time window of a phoneme. The larger the value, the greater the possibility that there is a boundary in the time window of the phoneme. It should be noted that since the ASR system may classify adjacent phonemes as the same phoneme, if the length of the time window of any phoneme is longer, the possibility that there is a boundary in the time window of the phoneme is relatively large.

[0058] Reflects the The first phoneme in the time window The time interval between the first moment and the end moment of the phoneme. The smaller the value, the closer the moment is to the end moment of the phoneme. When any moment is close to the end moment of the current phoneme, the time interval between the first moment and the end moment of the phoneme is The boundary between the audio signal in the time period composed of the end time of a phoneme and the next phoneme may be relatively vague, and the possibility of the existence of a boundary at the corresponding moment is relatively high.

[0059] S3: The time period consisting of the target moment and the end moment of the target phoneme is taken as the first time period, and the time period consisting of the start moment of the next phoneme and any moment within the time window of the next phoneme is taken as the second time period. The similarity overlap of the sound intensity of the first time period and the second time period is calculated. The similarity overlap is negatively correlated with the difference in average sound intensity and duration between the first time period and the second time period.

[0060] Among them, similarity overlap refers to data determined based on the similarity of sound intensity in the time window of adjacent phonemes, which reflects the transition characteristics between phonemes; the difference in average sound intensity in the first time period and the second time period reflects the similarity of the sound intensity in the two corresponding time periods. The smaller the difference, the more similar the change trend of the sound intensity in the two corresponding time periods is, and the similarity overlap of the sound intensity of the two corresponding phonemes is relatively large.

[0061] In an exemplary embodiment of the present invention, the difference between the duration of the first time period and the duration of the second time period is a comprehensive difference, and the comprehensive difference can be determined by the following steps:

[0062] The length of the time window of the target phoneme and the length of the time window of the next phoneme are obtained and summed. The absolute value of the difference between the length of the first period and the length of the second period and the ratio of the obtained cumulative sum are used as the comprehensive difference between the lengths of the first period and the second period.

[0063] Specifically, when the phoneme as the target phoneme, When a phoneme is used as the next phoneme of the target phoneme, the comprehensive difference between the duration of the first period and the second period satisfies the following relationship:

[0064] ;

[0065] In the formula, is the combined difference between the duration of the first period and the second period; is the duration of the first period; is the duration of the second period; For the The length of the time window for each phoneme; For the The length of the time window for each phoneme; is the absolute value symbol.

[0066] in, It reflects the closeness of the target moment to the end moment of the target phoneme; reflects the closeness of any moment in the time window of the next phoneme to the starting moment of the next phoneme; When is smaller, it means that the target time is at a later time in the time window of the target phoneme, and the possibility of fuzzy boundary between the audio signal in the first period and the next phoneme is relatively large; when If is smaller, it means that any moment is in the front moment of the time window of the next phoneme, and the possibility of fuzzy boundary between the audio signal and the target phoneme in the second time period is relatively large; if When it is smaller, it means that the possibility of blurred boundary between the audio signals in the first time period and the second time period is relatively large, which further indicates that the similarity overlap of the sound intensity in the corresponding two time periods is relatively large.

[0067] The larger it is, the greater the possibility that the boundary between the two phonemes is blurred, and the corresponding similarity overlap is relatively large.

[0068] Furthermore, based on the difference in average sound intensity in the first time period and the second time period, and the comprehensive difference in duration, the similarity overlap of the sound intensity in the two time periods can be calculated. Specifically, the similarity overlap of the sound intensity in the first time period and the second time period satisfies the following relationship:

[0069] ;

[0070] In the formula, For the The time window of the phoneme The moment and The time window of the phoneme The similarity overlap of the sound intensity at the moment, among which, The phoneme is The next phoneme of a phoneme; is the average sound intensity during the first period; is the average sound intensity during the second period; is the absolute value symbol; is the combined difference between the duration of the first period and the second period, where ; is the natural exponential function, where the natural exponential function is a natural constant An exponential function with base .

[0071] in, It reflects the difference between the average sound intensity in the first time period and the average sound intensity in the second time period. The smaller the value is, the greater the similarity overlap of the sound intensity in the two time periods is. The smaller it is, the greater the possibility of blurred boundaries between the audio signals in the two time periods, and the greater the similarity overlap between the two time periods.

[0072] Optionally, the ratio of the average sound intensity in the first time period to the average sound intensity in the second time period may be used as the difference in the average sound intensity in the two corresponding time periods.

[0073] In another embodiment, other calculation formulas may be used to calculate the similarity overlap, such as the calculation formula: ; In the formula is a preset hyperparameter used to prevent the denominator from being zero. There is no special limitation on the value of .

[0074] S4: Calculate the final segmentation probability at the target moment. The final segmentation probability is positively correlated with the similarity overlap and the initial segmentation probability.

[0075] Specifically, the final segmentation probability at any time within the time window of any phoneme satisfies the following relationship:

[0076] ;

[0077] In the formula, For the The time window of the phoneme The final segmentation probability at the moment; For the The time window of the phoneme The initial segmentation probability at the moment; For the The first phoneme The sound intensity of the first period corresponding to the moment is The first phoneme The similarity overlap of the sound intensity of the second time period corresponding to the moment; For the The total number of moments in the time window of a phoneme; is the normalization function.

[0078] in, Reflects the The time window of the phoneme The sound intensity of the first period corresponding to the moment is The normalized value of the cumulative similarity and overlap of the sound intensity of the second time period corresponding to each moment in the time window of the phoneme. The larger the value, the The boundary between the audio signal of the first period corresponding to the moment and the next phoneme is relatively fuzzy, which indicates that The possibility of a boundary at this moment is relatively large, and when When is also large, the final segmentation probability at that moment is relatively large.

[0079] In another embodiment, other calculation formulas may be used to calculate the final segmentation probability at the target moment, such as using a product calculation formula instead of a summation or averaging method.

[0080] S5: Using the final segmentation probability to select segmentation points to segment the phoneme sequence, so as to obtain a matching score between the segmented phoneme sequence and a pre-stored standard phoneme sequence based on the GOP algorithm.

[0081] It should be noted that the GOP algorithm is a technology used for phoneme matching and pronunciation evaluation. It mainly evaluates the pronunciation quality by comparing the matching degree between the recognized phonemes and the standard phonemes. The process is as follows: First, the input speech is recognized by the ASR system to obtain the phoneme sequence in the recognition result. Then, the standard pronunciation phoneme sequence is aligned with the recognized phoneme sequence through the dynamic time warping (DTW) method. Then, for each phoneme, the matching degree between each phoneme and the given standard phoneme is calculated, and the average of the matching degrees between all phonemes and the given standard phoneme is used as the matching score of the entire phoneme sequence, thereby achieving an accurate evaluation of the pronunciation.

[0082] However, in the oral assessment process, individual differences (such as speaking too fast, too slow, and pauses) often lead to significant differences between the recognized phoneme sequence and the standard phoneme sequence on the time axis. When the DTW algorithm handles large-scale time series changes, inaccurate alignment may occur. For example, speaking too fast may cause some phonemes to be compressed or omitted, while speaking too slowly may cause phonemes to be elongated or repeated, thereby causing some phonemes to be incorrectly matched to irrelevant phonemes, affecting the accuracy of phoneme matching. Therefore, when the present invention uses the GOP algorithm to obtain the matching score of the phoneme sequence, the phoneme sequence is first segmented using the calculated final segmentation probability, and then the segmented phoneme sequence is aligned with the pre-stored standard phoneme sequence using the DTW algorithm, thereby improving the alignment effect, ensuring the accuracy of the matching score of the obtained phoneme sequence, and achieving accurate evaluation of the oral assessment results of the target examinee.

[0083] In an exemplary embodiment of the present invention, the segmentation of the phoneme sequence can be achieved by the following steps:

[0084] A preset segmentation threshold is obtained. If the final segmentation probability at any moment in the time window of any phoneme is greater than the segmentation threshold, the moment is taken as a segmentation point to segment the phoneme sequence at all segmentation points to obtain a segmented phoneme sequence.

[0085] Optionally, the segmentation threshold may be set to 0.85, and when the final segmentation probability at any moment in the time window of any phoneme is greater than 0.85, the moment is taken as a segmentation point, thereby obtaining all segmentation points, so as to segment the phoneme sequence at the segmentation points to obtain a segmented phoneme sequence. Of course, a suitable segmentation threshold may also be set according to specific circumstances, and the present embodiment does not specifically limit the setting of the segmentation threshold.

[0086] Furthermore, after obtaining the segmented phoneme sequence, the alignment step in the GOP algorithm can be used to align the segmented phoneme sequence with the pre-stored standard phoneme sequence, and calculate the matching degree between each phoneme in the segmented phoneme sequence and the corresponding standard phoneme, thereby determining the average matching degree between all phonemes in the phoneme sequence and the corresponding standard phoneme, obtaining the matching score of the phoneme sequence, and determining the oral assessment result of the target candidate. It should be noted that the alignment step in the GOP algorithm and the process of calculating the matching score between the phoneme sequence and the pre-stored standard phoneme sequence are prior arts, and are not described in detail in this embodiment.

[0087] The present invention also provides a human-computer interaction oral evaluation system based on speech recognition. The system includes a memory and a processor, and a computer program is stored in the memory. The computer program integrates the functions of the human-computer interaction oral evaluation method based on speech recognition. When the computer program is executed, the accuracy of the evaluation results obtained by the human-computer interaction oral evaluation method based on speech recognition can be improved.

[0088] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0089] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A human-computer interactive spoken language assessment method based on speech recognition, characterized in that: include: Acquire the phoneme sequence of the target examinee's speech signal during the oral test; Select the target phoneme in the phoneme sequence and calculate the initial segmentation probability at the target time within the time window of the target phoneme: ; For the The time window of the phoneme The initial segmentation probability at the moment; is the entropy value of the sound intensity within the time window; , and The first moment, The moment and The intensity of the sound at a given moment; , Respectively The start and end times of each phoneme; is the normalization function; is the time interval between any two moments; The time window between the target time and the end time of the target phoneme is taken as the first time period, the time window between the start time of the next phoneme and any time of the next phoneme is taken as the second time period, and the difference between the duration of the first time period and the second time period is the comprehensive difference. The method for obtaining the comprehensive difference includes: Obtaining the length of the time window of the target phoneme and the length of the time window of the next phoneme, summing them up, and taking the ratio of the absolute value of the difference between the duration of the first time period and the duration of the second time period to the obtained cumulative sum as the comprehensive difference between the duration of the first time period and the duration of the second time period; Calculate the similarity overlap between the sound intensity of the first period and the second period, and satisfy the relationship: ； In the formula, For the The first phoneme The sound intensity of the first period corresponding to the moment is The first phoneme The similarity overlap of the sound intensity of the second time period corresponding to the moment, where The phoneme is The next phoneme of a phoneme; is the average sound intensity during the first period; is the average sound intensity during the second period; is the absolute value symbol; is the combined difference between the duration of the first period and the second period; is the natural exponential function; Calculate the final segmentation probability at the target time, satisfying the relationship: ； In the formula, For the The time window of the phoneme The final segmentation probability at the moment; For the The total number of moments in the time window of a phoneme; The final segmentation probability is used to select the segmentation points to segment the phoneme sequence, so as to obtain the matching score between the segmented phoneme sequence and the pre-stored standard phoneme sequence based on the GOP algorithm.

2. The human-computer interaction oral evaluation method based on speech recognition according to claim 1 is characterized in that: The moment in the time window of any phoneme is the temporal index of each frame signal in the time window of the phoneme.

3. The human-computer interaction oral evaluation method based on speech recognition according to claim 1 is characterized in that: The method of using the final segmentation probability to select the segmentation points to segment the phoneme sequence includes: A preset segmentation threshold is obtained. If the final segmentation probability at any moment in the time window of any phoneme is greater than the segmentation threshold, the moment is taken as a segmentation point to segment the phoneme sequence at all segmentation points to obtain a segmented phoneme sequence.

4. The human-computer interaction oral evaluation method based on speech recognition according to claim 1 is characterized in that: Before converting the speech signal of the target examinee into a phoneme sequence, the method further includes: The speech signal of the target examinee is preprocessed to filter out noise and silence parts in the speech signal.

5. The human-computer interactive oral evaluation system based on speech recognition is characterized by: The human-computer interaction oral evaluation system based on speech recognition includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the human-computer interaction oral evaluation method based on speech recognition as described in any one of claims 1 to 4.

Images