Spoken language evaluation method, device, equipment, medium and program product in listen-and-read scene

Abstract

The invention discloses a spoken language evaluation method and device in a listen-and-read scene, equipment, a storage medium and a program product, and relates to the field of audio processing, and the method comprises the steps: obtaining to-be-evaluated voice information and demonstration voice information, and obtaining a spoken language evaluation result based on a voice duration difference value of a to-be-evaluated voice signal and a demonstration voice signal; determining the speech speed consistency score of the to-be-evaluated speech signal, determining the content consistency score of the to-be-evaluated speech signal based on the editing distance score of the to-be-evaluated speech text and the demonstration speech text, and determining the content consistency score of the to-be-evaluated speech signal based on the difference between the speech features of the to-be-evaluated speech signal and the speech features of the demonstration speech signal. And determining a feature consistency score of the to-be-evaluated voice signal, and evaluating the to-be-evaluated voice signal based on the speech speed consistency score, the content consistency score and the feature consistency score. According to the method, comprehensive assessment is carried out on the follow-up performance of the student from three levels of speech speed, content and multi-dimensional speech features, and the defects that in the prior art, only one dimension is concerned, and interpretive analysis is lacked are overcome.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to methods, devices, equipment, storage media, and computer products for oral evaluation in a follow-up reading scenario. Background Art

[0002] With the development of artificial intelligence and speech recognition technology, intelligent methods for language teaching are constantly being enriched. Among them, "shadow reading training" is widely used in actual teaching as an important way to improve students' voice expression skills. The typical shadow reading training process involves the teacher demonstrating a sentence and the student repeating it, imitating the pronunciation, intonation, rhythm, and emotion. The intelligent system automatically scores the students, assisting learning and providing feedback. In a smart education environment, with the help of pre-recorded teacher audio and an automated scoring system, students can independently practice pronunciation, thereby reducing teaching costs and improving learning efficiency.

[0003] However, many scoring methods in related technologies focus solely on word-level pronunciation accuracy, failing to account for global factors such as the speed and intonation of entire sentences. Furthermore, most existing systems rely on specific languages ​​or customized models, with poor cross-language generalization and insufficient adaptability. Current scoring results are often a single score, lacking explanatory analysis of multi-dimensional factors such as pitch, loudness, and emotion, making it difficult to provide students with clear improvement suggestions. These issues not only limit the performance of scoring systems but also hinder their widespread application in real-world teaching scenarios. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment, storage medium and computer product for oral evaluation in a follow-up reading scenario, aiming to solve the technical problem that the related technology has poor interpretability in oral evaluation in a follow-up reading scenario.

[0005] To achieve the above objectives, this application proposes a method for oral evaluation in a shadowing scenario, which includes: Acquire voice information to be evaluated and demonstration voice information; wherein the voice information to be evaluated includes a voice signal to be evaluated and a voice text to be evaluated, including a demonstration voice signal and a demonstration voice text; Determine the speech speed consistency score of the speech signal to be evaluated based on the speech duration difference between the speech signal to be evaluated and the demonstration speech signal; the speech duration difference is negatively correlated with the speech speed consistency score; Determining a content consistency score of the speech signal to be evaluated based on the edit distance score between the speech text to be evaluated and the demonstration speech text; Determining a feature consistency score of the speech signal to be evaluated based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal; the speech features include emotional features, pitch features, and loudness features; The speech signal to be evaluated is evaluated based on the speech rate consistency score, content consistency score, and feature consistency score.

[0006] In one embodiment, the step of obtaining the voice information to be evaluated and the demonstration voice information includes: Acquire audio files, where the audio files include a speech audio file to be evaluated and a demonstration speech audio file having the same file format; Perform noise reduction and silence detection on audio files to remove silent data from audio files; A fixed-length silence segment is added to the audio file to obtain the speech signal to be evaluated and the demonstration speech signal with the same silence length.

[0007] In one embodiment, the step of determining the content consistency score of the speech signal to be evaluated based on the edit distance score between the speech text to be evaluated and the demonstration speech text includes: Based on the demonstration voice signal and the voice signal to be evaluated, performing timestamp alignment processing on the voice text to be evaluated and the demonstration voice text to obtain the voice text array to be evaluated and the demonstration voice text array; Adopting the minimum edit distance alignment method, taking the demonstration speech text array as a benchmark, performing an editing operation on the speech text array to be evaluated, aligning the speech text array with the demonstration speech text array, and determining the aligned text array to be evaluated; The content consistency score is determined based on the number of non-empty elements in the aligned text array to be evaluated.

[0008] In one embodiment, the step of determining a feature consistency score of the speech signal to be evaluated based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal includes: For the speech signal to be evaluated and each pronounced word in the speech signal to be evaluated, resampling each pronounced word in the speech signal to be evaluated with the pronounced word in the demonstration speech signal as a benchmark to obtain a speech signal to be evaluated having the same duration as the corresponding pronounced word in the demonstration speech signal; Extract speech features from the pronounced words to obtain speech features of the speech signal to be evaluated and the demonstration speech signal; the speech features include emotional features, pitch features, and loudness features; Based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal, a feature consistency score of the speech signal to be evaluated is determined.

[0009] In one embodiment, the steps of extracting speech features from the pronounced words and obtaining speech features of the speech signal to be evaluated and the demonstration speech signal include: For the speech signal to be evaluated and each pronounced word in the speech signal to be evaluated, performing emotion feature extraction, reference pitch feature extraction, and reference loudness feature extraction on the pronounced word; The inter-frame variation of the reference pitch feature is used as the pitch feature, and the standard deviation of the reference loudness feature is used as the loudness feature.

[0010] In one embodiment, the step of determining a feature consistency score of the speech signal to be evaluated based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal includes: Calculate the emotional feature similarity basic score, pitch feature similarity basic score, and loudness feature similarity basic score between the demonstration speech signal and the pronunciation words corresponding to the speech signal to be evaluated by using the cosine similarity algorithm; Normalize the emotion feature similarity basic score, pitch feature similarity basic score, and loudness feature similarity basic score respectively to obtain the emotion feature similarity score, pitch feature similarity score, and loudness feature similarity score; Based on the emotional feature similarity score, pitch feature similarity score and loudness feature similarity score as well as the emotional feature weight, pitch feature weight and loudness feature weight, the feature consistency score of each pronounced word in the speech signal to be evaluated is determined; the sum of the emotional feature weight, pitch feature weight and loudness feature weight is one.

[0011] In addition, to achieve the above-mentioned purpose, the present application further provides a spoken language evaluation device in a follow-up reading scenario, the device comprising: An acquisition module is used to acquire the voice information to be evaluated and the demonstration voice information; wherein the voice information to be evaluated includes the voice signal to be evaluated and the voice text to be evaluated, including the demonstration voice signal and the demonstration voice text; A speech rate evaluation module is used to determine a speech rate consistency score of the speech signal to be evaluated based on the speech duration difference between the speech signal to be evaluated and the demonstration speech signal; the speech duration difference is negatively correlated with the speech rate consistency score; A text evaluation module, configured to determine a content consistency score of the speech signal to be evaluated based on an edit distance score between the speech text to be evaluated and the demonstration speech text; A feature evaluation module is used to determine a feature consistency score of the speech signal to be evaluated based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal; the speech features include emotional features, pitch features, and loudness features; The speech evaluation module is used to evaluate the speech signal to be evaluated based on the speech rate consistency score, content consistency score and feature consistency score.

[0012] In addition, to achieve the above-mentioned purpose, the present application continues to provide a spoken language evaluation device in a follow-up reading scenario, characterized in that the device includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor, and the computer program is configured to implement the steps of the above-mentioned spoken language evaluation method in the follow-up reading scenario.

[0013] In addition, to achieve the above-mentioned purpose, the present application continues to provide a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the above-mentioned oral evaluation method in the shadowing scenario are implemented.

[0014] In addition, to achieve the above-mentioned purpose, the present application continues to provide a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned oral evaluation method in the shadowing scenario.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: The oral evaluation scheme proposed in this application for the shadowing reading scenario can comprehensively evaluate students' shadowing reading performance from three aspects: speaking speed, content, and multi-dimensional speech features. This addresses the shortcomings of existing technologies that only focus on a single dimension and lack explanatory analysis. By introducing indicators such as speech duration difference, text editing distance, and speech feature similarity, a fine-grained quantification of speech expression quality is achieved, improving the accuracy and interpretability of scoring. At the same time, a unified feature extraction and scoring mechanism is adopted to enhance the system's generalization ability in different languages ​​and diverse learning scenarios, thereby better supporting automated speech training and feedback in smart education environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 Schematic diagram of the flow of the oral evaluation method in the shadow reading scenario in one embodiment of the present application.

[0019] Figure 2 This is a structural diagram of the oral evaluation device in the follow-up reading scenario of this application.

[0020] Figure 3 This is a schematic diagram of the structure of the oral evaluation equipment in the shadow reading scenario of this application.

[0021] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0023] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The present application embodiment provides a method for evaluating spoken language in a follow-up reading scenario. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the oral evaluation method in the shadow reading scenario of this application.

[0025] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, a spoken language evaluation device in a follow-up reading scenario, etc. The following uses a spoken language evaluation device in a follow-up reading scenario as an example to illustrate this embodiment and the following embodiments.

[0026] In this embodiment, the oral evaluation method in the follow-up reading scenario includes steps S10 to S50: Step S10, obtaining voice information to be evaluated and demonstration voice information; wherein the voice information to be evaluated includes a voice signal to be evaluated and a voice text to be evaluated, including a demonstration voice signal and a demonstration voice text.

[0027] It should be noted that the demonstration voice information includes a demonstration voice signal and a demonstration voice text, and the voice information to be evaluated includes a voice signal to be evaluated and a voice text to be evaluated. Among them, the demonstration voice signal is a standard spoken audio, such as a pre-recorded audio file of a spoken language teacher reading aloud, and the demonstration voice text is the content text corresponding to the demonstration voice signal, for example, the text corresponding to the voice content read aloud by the spoken language teacher. The voice signal to be evaluated is obtained by recording the sound emitted by the object to be evaluated, such as an audio file obtained by recording the voice of a student learning spoken language. The voice text to be evaluated is the content text corresponding to the voice signal to be evaluated. The voice text to be evaluated and the demonstration voice text can be obtained by performing voice recognition on the voice signal to be evaluated and the demonstration voice signal through the voice recognition service.

[0028] In a feasible implementation, in order to ensure data quality, the speech signal to be evaluated and the demonstration speech signal can be obtained by preprocessing the initially acquired audio file. Specifically, step S10 includes steps A10 to A30: Step A10: Acquire audio files, wherein the audio files include a speech audio file to be evaluated and a demonstration speech audio file in the same file format.

[0029] Step A20: performing noise reduction processing and silence detection on the audio file to remove silence data from the audio file.

[0030] Step A30: Add a fixed-length silence segment to the audio file to obtain the speech signal to be evaluated and the demonstration speech signal with the same silence length.

[0031] For example, the teacher's demonstration audio file and the student's follow-up reading file of the demonstration audio file are used as input, and the format of the demonstration audio file and the follow-up reading file are normalized and unified into a mono PCM format with 16K sampling and 16 BIT width.

[0032] Using an audio noise reduction algorithm, we processed the PCM-formatted demonstration audio files and student-reading files to reduce noise. We also used the RMS (Root Mean Square) energy detection algorithm to detect silence at the beginning and end of the PCM files, removing the silence at the beginning and end of the demonstration audio files and the student-reading files. After removing the silence, we added 0.2 seconds of silence before and after each of the PCM files. This resulted in the preprocessed demonstration audio files (demonstration speech signals) and student-reading files (speech signals to be evaluated).

[0033] It can be understood that the preprocessing of the demonstration speech signal and the speech signal to be evaluated through the above operations can facilitate the normalization of the speech length, which is beneficial to improving the accuracy of the speech recognition result and preventing the phenomenon of word interruption in the speech recognition process.

[0034] Step S20: determining a speech rate consistency score of the speech signal to be evaluated based on the speech duration difference between the speech signal to be evaluated and the demonstration speech signal.

[0035] Specifically, for speech rate evaluation, it can be understood that if the duration difference between the speech signal to be evaluated and the demonstration speech signal is large, it indicates a significant difference in speech rate. Based on this, an overall speech rate score can be calculated. The difference in speech duration is negatively correlated with the speech rate consistency score.

[0036] For example, the specific calculation method is shown in Formula 1: in, To demonstrate the speech length of the speech signal, is the speech length of the speech signal to be evaluated, is the redundant error, in this example, The default value is 0.1. Score speech rate consistency.

[0037] Step S30 : determining a content consistency score of the speech signal to be evaluated based on the edit distance scores of the speech text to be evaluated and the demonstration speech text.

[0038] It can be understood that the content consistency between the speech signal to be evaluated and the demonstration speech signal is the basis of spoken language evaluation. In this embodiment, the content consistency between the speech signal to be evaluated and the demonstration speech signal can be determined by comparing the edit distance between the speech text to be evaluated and the demonstration speech text.

[0039] In a feasible implementation manner, step S30 includes steps B10 to B30: Step B10: Based on the demonstration voice signal and the voice signal to be evaluated, performing time stamp alignment processing on the voice text to be evaluated and the demonstration voice text to obtain a voice text array to be evaluated and a demonstration voice text array.

[0040] Step B20 , using the minimum edit distance alignment method, taking the demonstration speech text array as a reference, performing an editing operation on the evaluation speech text array, aligning the speech text array with the demonstration speech text array, and determining the aligned text array to be evaluated.

[0041] Step B30: Determine the content consistency score based on the number of non-empty elements in the aligned text array to be evaluated.

[0042] For example, a general speech recognition service is first performed on a demonstration speech signal and a speech signal to be evaluated, obtaining a demonstration speech text and a speech text to be evaluated corresponding to the demonstration speech signal. Using the Montreal Forced Aliner (MFA), the timestamps of the demonstration speech text and the demonstration speech signal are aligned. The speech text to be evaluated is then aligned with the timestamps of the speech signal to be evaluated on a per-word basis. The resulting array includes an array of speech text to be evaluated and an array of demonstration speech text.

[0043] Each element in the speech text array to be evaluated / demonstration speech text array consists of word content, pronunciation start timestamp and pronunciation end timestamp corresponding to the word. For example, the format of each element in the array can be: {word:“hello”,start:0.2,end:0.5} Each element in the array represents a word in the speech text to be evaluated / demonstration speech text, the word part represents the content of the word, the start part represents the starting timestamp of the word in the speech signal to be evaluated / speech signal to be evaluated, and the end part represents the ending timestamp of the word in the speech signal to be evaluated / speech signal to be evaluated.

[0044] For the word part in the above format, the format of the speech text array to be evaluated can be expressed as: Word student {hello, how, are, today}. The format of the demonstration speech text array can be expressed as: Word teacher {hello, how, are, you, today}.

[0045] Then, use the demonstration voice text array Word teacher As a benchmark, the idea of ​​minimum edit distance alignment is adopted to evaluate the speech text array Word student Edit and get the aligned text array Word to be evaluated aligned , aligned Word aligned The content is {hello, how, are, None, today}.

[0046] The specific alignment method is shown in Formula 2: Among them, the defined two-dimensional DP table dp[i][j] shows the minimum cost of aligning Wordstudent[1..j] to Wordteacher[1.i], r represents the replacement cost, and d represents the deletion cost.

[0047] Then, the content consistency score is determined based on the number of non-empty elements in the aligned text array to be evaluated. The specific calculation method is shown in Formula 3: in, Score edit Represents the content consistency score of the speech signal to be evaluated, len( Words aligned NotNone) Represents the aligned text array to be evaluated Word aligned The number of non-empty elements (non-None elements) ,len(Wor d teacher ) represents the demonstration voice text array Word teacher The number of elements in .

[0048] Step S40: confirming a feature consistency score of the speech signal to be evaluated based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal; the speech features include emotional features, pitch features, and loudness features.

[0049] In the follow-up reading scenario, not only the factors of speaking speed and voice content should be considered, but also the emotions, tones, pauses and other factors of pronunciation should be considered for subsequent calculation steps. Specifically, the voice features considered in this embodiment include emotional features, pitch features and loudness features.

[0050] In a feasible implementation manner, step S40 includes steps C10 to C40: Step C10 , for the speech signal to be evaluated and each pronounced word in the speech signal to be evaluated, resample each pronounced word in the speech signal to be evaluated with the pronounced word in the demonstration speech signal as a benchmark, to obtain a speech signal to be evaluated with the same duration as the corresponding pronounced word in the demonstration speech signal.

[0051] Step C20 , extracting speech features of the pronounced words to obtain speech features of the speech signal to be evaluated and the demonstration speech signal; the speech features include emotional features, pitch features, and loudness features.

[0052] Step C30 : determining a feature consistency score of the speech signal to be evaluated based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal.

[0053] Wherein, step C20 includes steps C21 and C22: Step C21 : For the speech signal to be evaluated and each pronounced word in the speech signal to be evaluated, perform emotion feature extraction, reference pitch feature extraction, and reference loudness feature extraction on the pronounced word.

[0054] In step C22 , the inter-frame variation of the reference pitch feature is used as the pitch feature, and the standard deviation of the reference loudness feature is used as the loudness feature.

[0055] Step C30 includes steps C31 to C33: Step C31 , calculating the emotion feature similarity basic score, pitch feature similarity basic score, and loudness feature similarity basic score between the demonstration speech signal and the pronunciation words corresponding to the speech signal to be evaluated by using a cosine similarity algorithm.

[0056] Step C32 : normalize the emotion feature similarity basic score, the pitch feature similarity basic score, and the loudness feature similarity basic score respectively to obtain the emotion feature similarity score, the pitch feature similarity score, and the loudness feature similarity score.

[0057] Step C33: confirm the feature consistency score of the speech signal to be evaluated based on the emotion feature similarity score, pitch feature similarity score and loudness feature similarity score as well as the emotion feature weight, pitch feature weight and loudness feature weight; the sum of the emotion feature weight, pitch feature weight and loudness feature weight is one.

[0058] Specifically, this embodiment extracts the features of a pronounced word based on a single pronounced word in the speech signal to be evaluated.

[0059] Exemplarily, each element of the demonstration speech text array / the speech text array to be evaluated has a start timestamp information start and an end timestamp information end. Based on the start timestamp and end timestamp of each element in the demonstration speech text array, the pronunciation time interval of each pronounced word of the speech signal to be evaluated is confirmed, and each pronounced word of the speech signal to be evaluated is resampled so that the start timestamp and end timestamp of each pronounced word in the speech signal to be evaluated are the same as the start timestamp and end timestamp of the corresponding pronounced word in the demonstration speech signal.

[0060] Specifically, take the demonstration voice text array Word teacher Based on the start and end timestamps of each element, reconfirm the aligned speech text array Word to be evaluated aligned The start timestamp and end timestamp of each non-empty element in the text array Word teacher The start and end timestamps of the corresponding elements are the same. Then, based on the start and end timestamps of each non-empty element, the pronunciation interval of each word in the resampled speech signal to be evaluated is determined. Based on this, each word in the speech signal to be evaluated is resampled to obtain a speech signal array. The format of each element in the speech signal array is as follows: {“word”:Word aligned "pcm_teacher": the pronunciation data corresponding to "word" in the demonstration speech signal, "pcm_student": the pronunciation data obtained by resampling the pronunciation word corresponding to "word" in the speech signal to be evaluated)}.

[0061] For emotional features, this embodiment uses Mel Frequency Cepstrum Coefficient (MFCC) as the emotional feature representation.

[0062] Exemplarily, for each element of the speech signal array, 256 sampling points are used as a frame and 128 sampling points are used as the sliding step between adjacent frames. MFCC extraction is performed on the pcm_student data and pcm_teacher data of the element respectively, and the MFCC features of all pronounced words of the speech signal to be evaluated and the demonstration speech signal are obtained as emotional features.

[0063] For the pitch feature, for each element of the speech signal array, 256 sampling points are used as a frame, and 128 sampling points are used as the sliding step size between adjacent frames. The pitch of the pcm_student data and pcm_teacher data of the element is extracted respectively, which is used as the benchmark pitch feature of the demonstration speech signal and the speech signal to be evaluated. Considering the differences in gender, age, etc. between the person who sends the speech signal to be evaluated and the person who sends the demonstration speech signal (such as a teacher and a student), there may be differences in the pronunciation frequency between them. Therefore, in this embodiment, after extracting the benchmark pitch feature, the inter-frame variation of the benchmark pitch feature is used as the pitch feature. The calculation method of the pitch feature is shown in Formula 4: Among them, F0 is the pitch value of an audio data frame, and change represents the inter-frame change of the benchmark pitch feature of a single pronounced word.

[0064] Then, with one change as an element, the inter-frame change of each pronunciation word is combined into a demonstration pitch feature array / pitch feature array to be evaluated, and the demonstration pitch feature array / pitch feature array to be evaluated is normalized to [0,1] to obtain the pitch features of the speech signal to be evaluated and the demonstration speech signal. Normalized Pitch : For the loudness feature, for each element of the speech signal array, the loudness of the pcm_student data and pcm_teacher data of the element are calculated respectively. The specific calculation method is: With 256 sampling points as a frame and 128 sampling points as the sliding step between adjacent frames, the RMS root mean square amplitude value of each frame in the pcm_student data and pcm_teacher data is calculated, and then the RMS root mean square amplitude value is converted to a db value. For the db values ​​calculated for all elements, these calculated db values ​​are normalized using the z-score: Where x is the dB value, μ is the mean dB value, and σ is the standard deviation of the dB value. All standardized dB values ​​are then normalized to the interval [0, 1] to obtain the loudness characteristics of the speech signal to be evaluated and the demonstration speech signal.

[0065] After calculating the emotional features, pitch features, and loudness features of the speech signal to be evaluated and the demonstration speech signal, the feature consistency score of the speech signal to be evaluated is calculated based on the features of the demonstration speech signal.

[0066] First, the cosine similarity is used to calculate the emotional feature similarity basic score, pitch feature similarity basic score and loudness feature similarity basic score between the corresponding pronunciation words of the speech signal to be evaluated and the demonstration speech signal, and all MFCC feature similarity basic scores, pitch feature similarity basic scores and loudness feature similarity basic scores are normalized to the [0-1] range as the emotional feature similarity score. Score mfcc 、 Pitch feature similarity score Score Pitch and loudness feature similarity score Score Loudness .

[0067] Then, set the sentiment weight to Weight mfcc , the pitch weight is Weight Pitch , the loudness weight is Weight loudness .in, Weight mfcc +Weight Pitch +Weight loudness =1.

[0068] Subsequently, based on the emotional feature similarity score, pitch feature similarity score, and loudness feature similarity score, as well as the emotional feature weight, pitch feature weight, and loudness feature weight, the feature consistency score of each pronounced word in the speech signal to be evaluated is determined. The calculation method is: Step S50 : Evaluate the speech signal to be evaluated based on the speech rate consistency score, the content consistency score, and the feature consistency score.

[0069] Specifically, this embodiment also sets a speech speed evaluation weight Weight speed , content evaluation weight Weight edit and feature evaluation weights Weight Segment .

[0070] The overall evaluation method is: in, Words score Score the feature consistency of the speech signal to be evaluated.

[0071] In this embodiment, the feature consistency score of the speech signal to be evaluated is obtained by integrating the feature consistency scores of each pronounced word of the speech signal to be evaluated. Specifically, the score weight of each pronounced word of the speech signal to be evaluated is determined. : The feature consistency score of the speech signal to be evaluated is calculated as follows: It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the oral evaluation method in the follow-up reading scenario of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0072] This application also provides a spoken language evaluation device in a follow-up reading scenario, please refer to Figure 2 , the oral evaluation device in the follow-up reading scenario includes: An acquisition module is used to acquire the voice information to be evaluated and the demonstration voice information; wherein the voice information to be evaluated includes the voice signal to be evaluated and the voice text to be evaluated, including the demonstration voice signal and the demonstration voice text; A speech rate evaluation module is used to determine a speech rate consistency score of the speech signal to be evaluated based on the speech duration difference between the speech signal to be evaluated and the demonstration speech signal; the speech duration difference is negatively correlated with the speech rate consistency score; A text evaluation module, configured to determine a content consistency score of the speech signal to be evaluated based on an edit distance score between the speech text to be evaluated and the demonstration speech text; A feature evaluation module is used to determine a feature consistency score of the speech signal to be evaluated based on the difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal; the speech features include emotional features, pitch features, and loudness features; The speech evaluation module is used to evaluate the speech signal to be evaluated based on the speech rate consistency score, content consistency score and feature consistency score.

[0073] The spoken language evaluation device for follow-up reading scenarios provided by this application utilizes the spoken language evaluation method for follow-up reading scenarios described in the aforementioned embodiments, thereby resolving the technical issue of poor interpretability in related art for spoken language evaluation for follow-up reading scenarios. Compared to related art, the beneficial effects of the spoken language evaluation device for follow-up reading scenarios provided by this application are the same as those of the spoken language evaluation method for follow-up reading scenarios provided in the aforementioned embodiments. Other technical features of the spoken language evaluation device for follow-up reading scenarios are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0074] The present application provides a spoken language evaluation device in a follow-up reading scenario, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the spoken language evaluation method in the follow-up reading scenario in the above-mentioned embodiment.

[0075] Reference below Figure 3 , which shows a structural diagram of a spoken language evaluation device in a follow-up reading scenario suitable for implementing the embodiment of the present application. The spoken language evaluation device in the follow-up reading scenario in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., as well as fixed terminals such as digital TVs, desktop computers, etc. The spoken language evaluation device in the follow-up reading scenario shown is only an example and should not bring any limitations to the functions and scope of use of the embodiment of the present application.

[0076] like Figure 3As shown, the spoken language evaluation device for follow-up reading scenarios may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the spoken language evaluation device for follow-up reading scenarios. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the oral language evaluation device in the shadowing scenario to communicate with other devices wirelessly or wired to exchange data. Although the figure shows an oral language evaluation device in the shadowing scenario with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0077] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0078] The device for evaluating spoken language in a follow-up reading scenario provided by this application utilizes the method for evaluating spoken language in a follow-up reading scenario described in the aforementioned embodiment, thereby resolving the technical issue of weak interpretability in related art for evaluating spoken language in a follow-up reading scenario. Compared to related art, the beneficial effects of the device for evaluating spoken language in a follow-up reading scenario provided by this application are the same as those of the method for evaluating spoken language in a follow-up reading scenario described in the aforementioned embodiment. The other technical features of the device for evaluating spoken language in a follow-up reading scenario are the same as those disclosed in the method described in the aforementioned embodiment, and are not further elaborated upon here.

[0079] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0080] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0081] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, and the computer-readable program instructions are used to execute the oral evaluation method in the shadowing scenario in the above-mentioned embodiment.

[0082] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0083] The computer-readable storage medium may be included in the spoken language evaluation device in the follow-up reading scenario; or it may exist independently without being assembled into the spoken language evaluation device in the follow-up reading scenario.

[0084] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the oral evaluation device in the shadowing scenario, the oral evaluation device in the shadowing scenario: obtains voice information to be evaluated and demonstration voice information; wherein the voice information to be evaluated includes a voice signal to be evaluated and a voice text to be evaluated, including a demonstration voice signal and a demonstration voice text; based on the voice duration difference between the voice signal to be evaluated and the demonstration voice signal, determines the speech speed consistency score of the voice signal to be evaluated; the voice duration difference is negatively correlated with the speech speed consistency score; based on the edit distance score between the voice text to be evaluated and the demonstration voice text, determines the content consistency score of the voice signal to be evaluated; based on the difference between the voice features of the voice signal to be evaluated and the voice features of the demonstration voice signal, confirms the feature consistency score of the voice signal to be evaluated; the voice features include emotional features, pitch features and loudness features; based on the speech speed consistency score, the content consistency score and the feature consistency score, the voice signal to be evaluated is evaluated.

[0085] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0086] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0087] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0088] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned method for evaluating spoken language in a follow-up reading scenario. This computer-readable storage medium can address the technical issue of weak interpretability in related art for evaluating spoken language in a follow-up reading scenario. Compared to related art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the method for evaluating spoken language in a follow-up reading scenario provided in the aforementioned embodiments, and are not further elaborated here.

[0089] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned oral evaluation method in the shadowing scenario.

[0090] The computer program product provided in this application can address the technical issue of weak interpretability in related technologies for oral language evaluation in shadowing scenarios. Compared to related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the oral language evaluation method in shadowing scenarios provided in the aforementioned embodiments, and will not be elaborated upon here.

[0091] The above are only some embodiments of the present application and are not intended to limit the patent scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for oral evaluation in a follow-up reading scenario, characterized in that: The method includes: Acquire voice information to be evaluated and demonstration voice information; wherein the voice information to be evaluated includes a voice signal to be evaluated and a voice text to be evaluated, and the demonstration voice signal and the demonstration voice text are obtained; Determining a speech speed consistency score of the speech signal to be evaluated based on a speech duration difference between the speech signal to be evaluated and the demonstration speech signal; wherein the speech duration difference is negatively correlated with the speech speed consistency score; Determining a content consistency score of the speech signal to be evaluated based on the edit distance scores of the speech text to be evaluated and the demonstration speech text; Determining a feature consistency score of the speech signal to be evaluated based on a difference between the speech features of the speech signal to be evaluated and the speech features of the demonstration speech signal, wherein the speech features include emotional features, pitch features, and loudness features; The speech signal to be evaluated is evaluated based on the speech rate consistency score, the content consistency score, and the feature consistency score.

2. The method according to claim 1, wherein The step of obtaining the voice information to be evaluated and the demonstration voice information includes: Acquire audio files, wherein the audio files include a speech audio file to be evaluated and a demonstration speech audio file having the same file format; Performing noise reduction processing and silence detection on the audio file to remove silence data from the audio file; A silence segment of a fixed length is added to the audio file to obtain the speech signal to be evaluated and the demonstration speech signal with the same silence length.

3. The method according to claim 1, wherein The step of determining the content consistency score of the speech signal to be evaluated based on the edit distance score of the speech text to be evaluated and the demonstration speech text comprises: Based on the demonstration voice signal and the voice signal to be evaluated, performing timestamp alignment processing on the voice text to be evaluated and the demonstration voice text to obtain a voice text array to be evaluated and a demonstration voice text array; Adopting a minimum edit distance alignment method, taking the demonstration speech text array as a reference, performing an editing operation on the speech text array to be evaluated, aligning the speech text array with the demonstration speech text array, and determining an aligned text array to be evaluated; The content consistency score is determined based on the number of non-empty elements in the aligned text array to be evaluated.

4. The method according to claim 1, wherein The step of determining the feature consistency score of the speech signal to be evaluated based on the difference between the speech feature of the speech signal to be evaluated and the speech feature of the demonstration speech signal comprises: For the speech signal to be evaluated and each pronounced word in the speech signal to be evaluated, resampling each pronounced word in the speech signal to be evaluated with the pronounced word in the demonstration speech signal as a reference to obtain a speech signal to be evaluated having the same duration as the corresponding pronounced word in the demonstration speech signal; Extracting speech features of the pronounced words to obtain speech features of the speech signal to be evaluated and the demonstration speech signal; the speech features include emotional features, pitch features, and loudness features; Based on the difference between the speech feature of the speech signal to be evaluated and the speech feature of the demonstration speech signal, a feature consistency score of the speech signal to be evaluated is determined.

5. The method according to claim 4, wherein The step of extracting speech features from the pronounced word to obtain speech features of the speech signal to be evaluated and the demonstration speech signal comprises: For the speech signal to be evaluated and each pronounced word in the speech signal to be evaluated, performing emotion feature extraction, reference pitch feature extraction, and reference loudness feature extraction on the pronounced word; The inter-frame variation of the reference pitch feature is used as the pitch feature, and the standard deviation of the reference loudness feature is used as the loudness feature.

6. The method according to claim 5, wherein The step of determining the feature consistency score of the speech signal to be evaluated based on the difference between the speech feature of the speech signal to be evaluated and the speech feature of the demonstration speech signal comprises: Calculating the emotional feature similarity basic score, the pitch feature similarity basic score, and the loudness feature similarity basic score between the demonstration speech signal and the pronunciation words corresponding to the speech signal to be evaluated by using a cosine similarity algorithm; Normalizing the emotion feature similarity basic score, the pitch feature similarity basic score, and the loudness feature similarity basic score respectively to obtain an emotion feature similarity score, a pitch feature similarity score, and a loudness feature similarity score; Based on the emotional feature similarity score, pitch feature similarity score and loudness feature similarity score as well as the emotional feature weight, pitch feature weight and loudness feature weight, the feature consistency score of each pronounced word in the speech signal to be evaluated is confirmed; the sum of the emotional feature weight, pitch feature weight and loudness feature weight is one.

7. A spoken language evaluation device in a follow-up reading scenario, characterized in that: The device comprises: An acquisition module, configured to acquire voice information to be evaluated and demonstration voice information; wherein the voice information to be evaluated includes a voice signal to be evaluated and a voice text to be evaluated, and the demonstration voice signal and demonstration voice text are included; a speech rate evaluation module, configured to determine a speech rate consistency score of the speech signal to be evaluated based on a speech duration difference between the speech signal to be evaluated and the demonstration speech signal; wherein the speech duration difference is negatively correlated with the speech rate consistency score; a text evaluation module, configured to determine a content consistency score of the speech signal to be evaluated based on an edit distance score between the speech text to be evaluated and the demonstration speech text; a feature evaluation module, configured to determine a feature consistency score of the speech signal to be evaluated based on differences between speech features of the speech signal to be evaluated and speech features of the demonstration speech signal; the speech features including emotion features, pitch features, and loudness features; The speech evaluation module is configured to evaluate the speech signal to be evaluated based on the speech rate consistency score, the content consistency score, and the feature consistency score.

8. A spoken language evaluation device in a follow-up reading scenario, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the oral language evaluation method in a shadowing scenario according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the oral evaluation method in the shadowing scenario according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the oral language evaluation method in a shadowing scenario are implemented as described in any one of claims 1 to 6.

Images