Oral language assessment method, device, electronic device and computer-readable storage medium

Abstract

The embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for oral evaluation, which relate to the field of speech recognition. The method comprises: obtaining spoken audio, marking at least one vowel segment of the spoken audio; obtaining the rhythm feature of the spoken audio according to the duration of at least one vowel segment; and evaluating the rhythm of the spoken audio according to the rhythm feature. When the embodiments of the present application are applied in cloud education, especially oral rhythm education, the rhythm level of the user's spoken language can be judged more accurately.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and more specifically, to a spoken language evaluation method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Cloud Computing Education (CCEDU for short) refers to an education platform service based on the cloud computing business model. On the cloud platform, all educational institutions, training institutions, enrollment service institutions, publicity institutions, industry associations, management institutions, industry media, legal structures, etc. are integrated into a resource pool. Each resource is displayed and interacted with each other, and communication is carried out on demand to reach an agreement, thereby reducing education costs and improving efficiency.

[0003] Oral prosody (rhythm) education is an important branch of cloud education. Currently, there are two main methods for evaluating the prosody of second language oral prosody:

[0004] 1. Through speech recognition and natural speech processing technologies, all rhythm-related parameters of the second language speech are extracted and modeled, and the scoring results of artificial experts are learned by the machine to form a scoring standard consistent with the calibration standard of artificial experts. However, this method cannot provide the errors in speech rhythm.

[0005] 2. By extracting the speech prosodic parameters of the second language and the native speaker and comparing their similarities, if the classification result is closer to the native language, the pronunciation prosody is better.

[0006] The shortcomings of existing technologies for evaluating spoken English prosody are mainly reflected in the following aspects:

[0007] 1) It is believed that the time between stresses in English sentences is equal, and this standard is also used to judge the rhythm of the second language. This approach is not entirely reasonable. In fact, stress-rhythmic English has this tendency, but due to the differences in sentence structure and syllable structure, although there is such an impression in perception, acoustic phonetics measurements have found that the time between stresses cannot be equal.

[0008] 2) Use manual stress marking to compare the stressed syllables of native speakers with those of second language speakers. However, since stress is marked based on subjective perception, the perceptual ability of the annotator has a great influence on the marking results. The perception of non-native annotators is also different from that of native speakers, and the correlation between different annotators is difficult to guarantee. Therefore, the method of evaluating prosody through manual subjective stress marking may introduce erroneous judgments by the annotator, thereby providing unreliable input information for prosody evaluation.

[0009] 3) Even if the subjective judgment of the stress annotators is correct, the correlation between different annotators is very high. In many cases, there is not only one correct rhythmic pattern for the same sentence. The rhythmic patterns and stress patterns used by different native speakers are also different, but at the same time they are all acceptable. Therefore, it is not reasonable to use a certain stress rhythm pattern to measure the prosody of a second language. Summary of the invention

[0010] The embodiments of the present application provide a spoken language assessment method, device, electronic device, and computer-readable storage medium that overcome the above-mentioned problems or at least partially solve the above-mentioned problems.

[0011] In a first aspect, a spoken language assessment method is provided, the method comprising:

[0012] Acquire spoken audio and mark at least one vowel segment of the spoken audio;

[0013] Obtaining rhythmic features of spoken audio based on the duration of at least one vowel segment;

[0014] Evaluate the rhythm of spoken audio based on rhythmic features.

[0015] In a possible implementation, the rhythm features include: a normalized pairwise difference index of consecutive vowel segments and a vowel segment duration variation coefficient.

[0016] The vowel rhythm features also include: one or more of the vowel segment duration ratio, the standard deviation of the vowel segment duration, and the vowel segment rate.

[0017] In a possible implementation, the process of acquiring spoken audio further includes: marking pause segments in the spoken audio;

[0018] Rhythmic features also include the number and total duration of pauses.

[0019] In a possible implementation, evaluating the rhythm of spoken audio according to the rhythm feature includes:

[0020] Perform weighted summation on the rhythm features to obtain a rhythm score;

[0021] Determine a scoring interval in which the rhythm score is located, and determine the rating of the spoken audio according to a predetermined correspondence between the scoring interval and the rating.

[0022] In a possible implementation, the method further includes:

[0023] Obtain the text corresponding to the spoken audio and determine the target vocabulary in the text;

[0024] Determine the vowel segment corresponding to the target vocabulary from the spoken audio as the target segment;

[0025] The rhythm of the target words was evaluated based on the duration of the target segment and the vowel segments adjacent to the target segment.

[0026] In a possible implementation, the rhythm of the target vocabulary is evaluated according to the duration of the target segment and the same segment adjacent to the target segment, including:

[0027] Calculate the difference in duration between the target segment and the two adjacent vowel segments before and after the target segment;

[0028] If at least one of the two differences is greater than a second preset threshold, it is determined that the rhythm of the target vocabulary is correct; if both differences are less than the second preset threshold, it is determined that the rhythm of the target vocabulary is wrong.

[0029] In a second aspect, a spoken language evaluation device is provided, the device comprising:

[0030] An audio annotation module, used to obtain spoken audio, obtain the spoken audio, and annotate at least one vowel segment of the spoken audio;

[0031] A rhythm feature acquisition module, used to obtain the rhythm feature of the spoken audio according to the duration of at least one vowel segment;

[0032] The audio rhythm judgment module is used to evaluate the rhythm of spoken audio according to rhythm features.

[0033] In a possible implementation, the rhythm features include: a normalized pairwise difference index of consecutive vowel segments and a vowel segment duration variation coefficient.

[0034] In a possible implementation, the vowel rhythm feature further includes: one or more of a vowel segment duration ratio, a standard deviation of a vowel segment duration, and a vowel segment rate.

[0035] In a possible implementation, the audio annotation module is further used to: annotate pause segments in the spoken audio;

[0036] Rhythmic features also include the number and total duration of pauses.

[0037] In a possible implementation, the audio rhythm determination module includes:

[0038] A rhythm score acquisition unit, used for performing weighted summation of rhythm features to obtain a rhythm score;

[0039] The threshold comparison unit is used to determine the scoring interval where the rhythm score is located, and determine the rating of the spoken audio according to the predetermined correspondence between the scoring interval and the rating.

[0040] In a possible implementation, the spoken language evaluation device further includes:

[0041] The vocabulary acquisition module is used to acquire the text corresponding to the spoken audio and determine the target vocabulary in the text;

[0042] A target segment acquisition module is used to determine a vowel segment corresponding to a target word from the spoken audio as a target segment;

[0043] The vocabulary rhythm judgment module is used to judge the rhythm of the target vocabulary according to the duration of the target segment and the vowel segment adjacent to the target segment.

[0044] In a possible implementation, the vocabulary rhythm determination module includes:

[0045] A difference calculation unit, used to calculate the difference between the duration of the target segment and the two adjacent vowel segments before and after the target segment;

[0046] The rhythm judgment unit is used to determine that the rhythm of the target vocabulary is correct if at least one of the two differences is greater than a second preset threshold, and to determine that the rhythm of the target vocabulary is wrong if both differences are less than the second preset threshold.

[0047] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method provided in the first aspect are implemented.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect.

[0049] The oral assessment method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application can avoid the influence of subjective perception on oral assessment by acquiring oral audio and marking the vowel segments of the oral audio, compared with the existing method of directly manually marking stress vowels, and is also suitable for the execution of massive oral assessment tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0051] Figure 1 A schematic diagram of an implementation environment provided for an embodiment of the present application;

[0052] Figure 2 A flowchart of a spoken language assessment method provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a terminal display reading interface provided in an embodiment of the present application;

[0054] Figure 4 A speech annotation diagram of a native English speaker provided in an embodiment of the present application;

[0055] Figure 5 Another speech annotation diagram of a native English speaker provided in an embodiment of the present application;

[0056] Figure 6 A speech annotation diagram of a Chinese second language learner provided in an embodiment of the present application;

[0057] Figure 7 Another speech annotation diagram of a Chinese second language learner provided in an embodiment of the present application;

[0058] Figure 8 Distribution diagram of rhythm characteristics of three groups of people provided in the embodiment of the present application;

[0059] Fig. 9 A flowchart diagram of another embodiment of the present application is provided;

[0060] Fig.10 A schematic diagram of a terminal displaying an evaluation result interface according to an embodiment of the present application;

[0061] Fig.11 A schematic diagram of the structure of a spoken language evaluation device provided in an embodiment of the present application;

[0062] Fig.12 A schematic diagram of the structure of a sub-device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0064] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "an" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0065] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0066] The oral evaluation method, device, electronic device and computer-readable storage medium provided in this application are intended to solve the above technical problems in the prior art.

[0067] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0068] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an embodiment of the present application. The implementation environment may include: a terminal 10 and a server 20. The terminal 10 may be directly or indirectly connected to the server 20 via a wired or wireless network.

[0069] In the embodiment of the present application, a client running a target application is installed in the terminal 10. Optionally, the target application may be a language teaching application having the functions of displaying text required for oral evaluation, collecting spoken audio, and sending the spoken audio to a server.

[0070] The terminal 10 may be an electronic device such as a mobile phone, a tablet computer, an e-book reader, a multimedia player, a wearable device, a PC (Personal Computer), etc.

[0071] The server 20 may be a background server of the target application, and is used to provide background services for the terminal, such as obtaining the spoken audio uploaded by the terminal 10, marking at least one vowel segment of the spoken audio, and obtaining the rhythmic features of the spoken audio according to the duration of the at least one vowel segment; judging whether the rhythm of the spoken audio is wrong according to the rhythmic features, and returning the judgment result to the terminal 10.

[0072] The server 20 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0073] The execution method of the server of the embodiment of the present application can be completed in the form of cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0074] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0075] According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer. SaaS can also be deployed directly on IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is a variety of business software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0076] It can be understood that in other embodiments, the terminal 10 may not send the spoken audio to the server 20, and the above series of processing of the spoken audio by the server 20 may also be completed by the terminal 10 itself. This is only an example of an application scenario, and the execution subject of the spoken language evaluation method provided by the present application is not limited to the server 20, and the terminal 10 itself may also execute the spoken language evaluation method.

[0077] In the technical solution provided in the embodiment of the present application, by acquiring spoken audio and marking the vowel segments of the spoken audio, compared with the existing method of directly marking stress manually, it is possible to avoid the influence of subjective perception on spoken language evaluation, and it is also suitable for the execution of massive spoken language evaluation tasks. According to the characteristics that the segment lengths of stressed rhythmic languages ​​are significantly different, while the segment lengths of syllable rhythmic speech are almost the same, the embodiment of the present application further obtains the rhythm features of the spoken audio according to the duration of the vowel segments, and judges whether the rhythm of the spoken audio is wrong by the rhythm features, so as to more accurately judge the rhythm level of spoken language.

[0078] In addition, the technical solution provided in the embodiment of the present application can be applied to oral assessment of multiple different languages, such as English, French, German, Chinese, Japanese, Korean, etc. In the embodiment of the present application, the application in English is mainly used as an example for introduction and description, but it does not constitute a limitation on the technical solution of the present application.

[0079] The present application embodiment provides a spoken language evaluation method, such as Figure 2 As shown, the method includes:

[0080] S101: Acquire spoken audio, and mark at least one vowel segment of the spoken audio.

[0081] The spoken audio is the audio generated by the user reading aloud the text for spoken language evaluation. It should be understood that the embodiment of the present application needs to evaluate the rhythm of spoken language, so the text is preferably in the form of sentences or paragraphs.

[0082] The execution subject of this method can be a terminal or a server. When the execution subject is a terminal, the terminal first displays the text for oral evaluation on the display interface, collects the audio generated when the user reads the text as the oral audio, and performs subsequent processing. The text displayed by the terminal can be the text entered by the user himself through typing, taking pictures, copying and pasting, etc., or it can be the text sent by the server, and the embodiments of this application are not specifically limited.

[0083] When the execution subject is a server, after collecting the spoken audio, the terminal will send the spoken audio to the server, and the server will execute the technical solution.

[0084] like Figure 3As shown, the terminal displays a reading interface including the text content "I know the fact, do you know?" and a start reading button, which is an operation control for triggering the terminal to start recording. After the user clicks the start reading button, the text content is read aloud, and the terminal starts collecting the user's audio content after receiving a trigger signal corresponding to the start reading button. In addition, after receiving a trigger signal corresponding to the end reading button, the terminal stops collecting the user's audio content and obtains spoken audio based on the collected audio content.

[0085] Phonemes are the smallest units of speech divided according to the natural properties of speech. From the perspective of acoustic properties, phonemes are the smallest units of speech divided from the perspective of sound quality. Vowels, also known as vowels, are a type of phoneme, as opposed to consonants. Vowels (vowel, hereinafter referred to as v) are sounds produced when the airflow passes through the mouth without obstruction during the pronunciation process. There are five vowel letters in English: a, e, i, o, and u. Consonants (consonant, hereinafter referred to as c), also called consonants, refer to sounds formed when the airflow is obstructed in the mouth or pharynx. The main types of consonants in English include plosives, fricatives, affricates, nasals, lateral sounds, and semivowels.

[0086] The rhythm of different languages ​​is different. For example, English belongs to stress-timed rhythm, while Chinese belongs to syllable-timed language. Stress is the stressed sound, and stress-timed language is the language that controls the rhythm by stress. Syllable is the phonetic unit that is pronounced by combining phonemes (including vowels and consonants). Syllable-timed language refers to the language that controls the rhythm by syllable. This leads to the fact that although native English speakers may use different rhythms or rhythms to produce the same sentence, they always avoid stress on adjacent syllables at the same time, which is difficult for Chinese speakers to do. It should be noted that stress-timed languages ​​also include German, Spanish, French, etc., and syllable-timed speech also includes: Korean, Japanese, etc. The oral evaluation method of the embodiment of the present application is applicable to the situation where the native language and the second language (referred to as the second language) are different rhythm languages ​​in one scenario, such as oral evaluation of Japanese-German, Korean-Spanish, Chinese-French, etc.

[0087] Although the prior art also considers using stress to distinguish between native speakers and second language speakers when evaluating spoken rhythm, the prior art often adopts a method of directly marking stress from spoken audio. However, this method has the disadvantage of not being able to be applied on a large scale due to the use of manual subjective annotation. On the other hand, the subjective control ability of the annotator has a great influence on the annotation results, thereby providing unreliable input information for training and evaluation. In order to replace the method of directly marking stress, the embodiment of the present application adopts a method of marking vowel segments in spoken audio. Specifically, the present application can mark the boundaries of vowels (or consonants) through a speech annotation tool, and then merge adjacent vowels that are not separated by pauses into vowel segments. The embodiment of the present application does not specifically limit the speech annotation tool, for example, it can be Praat phonetics software, labelstudio annotation platform, Transcriber phonetics software, etc.

[0088] It should be understood that vowels and consonants are two corresponding phonemes. When a vowel segment appears in the audio, a consonant segment will generally appear immediately afterwards. Therefore, the embodiments of the present application can directly mark the vowel segment, or mark the consonant segment first and then determine the vowel segment by treating the unmarked segment as the vowel segment.

[0089] S102: Obtain rhythm features of spoken audio according to the duration of at least one vowel segment.

[0090] Since English is a stress-rhythm language, unstressed syllables are required to be pronounced weakly. The duration of unstressed syllables is shortened accordingly according to the interval of stress, so as to ensure that adjacent syllables are not all stressed as much as possible, and the stressed syllables are kept as close as possible in consideration of the correct expression of semantics and the shortening of the length of each unstressed syllable between stresses. Chinese is a syllable-rhythm language, and the pronunciation of each syllable is in place. One of the biggest prosody problems for Chinese students speaking English is the rhythm problem. The main reason why Chinese students are affected by their mother tongue and cannot produce a rhythm similar to that of native speakers is that unstressed syllables are not weakened. Therefore, the embodiment of the present application obtains the rhythm characteristics of spoken audio according to the duration of vowel segments.

[0091] Taking the text “What do you do on Sundays?” as an example, we first obtain the pronunciations of two native English speakers and the spoken audios of two Chinese second language learners. We then annotate the four spoken audios using a speech annotation tool. The results of the speech annotation are shown in Figures 4 to 7 .

[0092] exist Figures 4 to 7In the figure, the annotation graph has four layers of information, from top to bottom, namely waveform information (wave), spectrogram information (spectrogram), phone annotation information (phone annotation), consonant segment vowel segment annotation (CV intervalannotation), where s represents pause segment, v represents vowel segment, and c represents consonant segment. Figure 4 The vowel of do (the second v) in the second syllable of the first two syllables is longer. Figure 4 The spoken audio of whatdoyoudo emphasizes the first auxiliary verb do; Figure 5 Of the first two syllables of the word, the vowel of do in the second syllable (the second v) is not long, but the first syllable of what (the first v) is longer. Figure 5 The spoken audio of "whatdoyoudo" emphasizes the first word "do" in "whatdoyoudo". It can be seen that although the two native speakers have inconsistent stress, if there is a pause between the two words, the length of other vowel segments will be adjusted so that the adjacent vowel segments will not be very long. Therefore, the embodiment of the present application finds that the stress length of English native speakers varies when speaking, and there will be a situation to avoid stressing adjacent syllables. Therefore, if there is a large difference between two adjacent vowel segments, it means that the rhythm is correct. For example, the difference in the length of adjacent vowel segments can be used as a rhythm feature.

[0093] S103. Evaluate the rhythm of the spoken audio according to the rhythm features.

[0094] After obtaining the rhythm features, the embodiments of the present application can evaluate the rhythm of the spoken audio according to the rhythm features. For example, the duration difference between adjacent vowel segments is a rhythm feature. If the number of audio differences between adjacent vowel segments in the spoken audio that is greater than a preset threshold meets the preset conditions, then it is determined that the rhythm of the spoken audio is correct. For example, there are 6 vowel segments marked in a certain spoken audio, so there are 5 duration differences. If there are more than 3 duration differences that are greater than the preset threshold, then the rhythm of the spoken audio is considered to be correct.

[0095] By acquiring spoken audio and marking the vowel segments of the spoken audio, the embodiment of the present application can avoid the influence of subjective perception on spoken language evaluation compared to the existing method of directly marking stress manually, and is also suitable for the execution of massive spoken language evaluation tasks. According to the characteristics that the segment lengths of stress-rhythmic languages ​​are significantly different, while the segment lengths of syllable-rhythmic speech are almost the same, the embodiment of the present application further obtains the rhythm features of the spoken audio according to the duration of the vowel segments, and evaluates the rhythm of the spoken audio through the rhythm features, which can more accurately judge the rhythm level of spoken language.

[0096] In fact, many rhythm features can be obtained according to the duration of vowel segments, such as the standard deviation of the segment duration, the segment rate, the segment rate ratio, etc. How to find the rhythm feature that can most significantly distinguish the rhythm of the second language and the native speaker is another technical problem solved by the embodiment of the present application. The following abbreviations are used in the subsequent embodiments of the present application to express the rhythm features in the embodiments of the present application:

[0097] C: consonant segment, that is, all continuous consonant segments without pauses in between;

[0098] V: Vowel segment, that is, all continuous vowel segments without pauses in between;

[0099] %V: proportion of vowel segment duration, that is, the total duration of vowel segment divided by the total duration of vowel segment and consonant segment;

[0100] %C: the proportion of consonant segment duration, that is, the total duration of the consonant segment divided by the total duration of the vowel segment and the consonant segment;

[0101] ΔV: standard deviation of vowel segment duration;

[0102] ΔC: standard deviation of consonant segment duration;

[0103] rPVI_C: consonant segment raw pairwise difference index;

[0104] nPVI_V: Normalized pairwise difference index of vowel segments;

[0105] VarcoC: standard deviation of consonant segment duration divided by the mean number of consonant segments;

[0106] VarcoV: standard deviation of vowel segments divided by the mean number of vowel segments;

[0107] rateC: consonant segment rate (number of consonant segments per second);

[0108] rateV: vowel segment rate (number of vowel segments per second);

[0109] rateCV: vowel and consonant segment rate (number of vowel-consonant segments per second);

[0110] nS: the number of pause segments;

[0111] sumS: The total duration of the pause segment.

[0112] The embodiment of the present application divides the test population into three groups, namely, native English speakers, high-scoring second language speakers, and low-scoring second language speakers, wherein high scores and low scores refer to the results of manual (expert) evaluation of the spoken language of second language speakers (the native language of the second language speakers in the embodiment of the present application is Chinese). High-scoring second language speakers are second language speakers with a better English speaking rhythm, and low-scoring second language speakers are second language speakers with a poor English speaking rhythm.

[0113] The significance of using relevant rhythm features to distinguish three groups of people is shown in Table 1:

[0114]

[0115]

[0116] Table 1 Rhythm Significance Table

[0117] Among them, [-] indicates insignificance, [*] indicates P < 0.05, [**] indicates P < 0.01, and [***] indicates P < 0.001. P is the abbreviation of the overall population. Generally, P > 0.05 indicates insignificant differences; 0.01 < P < 0.05 indicates significant differences; P < 0.01 indicates extremely significant differences. From Table 1, it can be seen that nPVI_V and VarcoV can simultaneously distinguish the three groups of speakers. Therefore, the rhythm features of the embodiments of this application include the normalized pairwise difference index of consecutive vowel segments and the vowel segment duration change coefficient.

[0118] Specifically, the calculation formula for the normalized pairwise difference index of consecutive vowel segments is:

[0119]

[0120] where m represents the total number of vowel segments in the audio, d k represents the duration of the k-th vowel segment, and d k+1 represents the duration of the (k + 1)-th vowel segment.

[0121] The calculation formula for the vowel segment duration change coefficient is:

[0122]

[0123] where ΔV represents the standard deviation of the vowel segment durations in the audio, represents the average duration of the vowel segments in the audio.

[0124] To further determine the impact of the normalized pairwise difference index of consecutive vowel segments and the vowel segment duration change coefficient on rhythm, the embodiments of this application use the above two rhythm features as the abscissa and ordinate respectively, and investigate the rhythm features of the oral audio of nearly a hundred different sentences for three groups of people (native speakers, high-score second language speakers, and low-score second language speakers). It can be seen Figure 8 , where the circular dots represent the positions of the two rhythm features of native speakers (Nat), the triangular dots represent the positions of the two rhythm features of high-score second language speakers (NonHigh), and the square dots represent the positions of the two rhythm features of low-score second language speakers (NonLow). It can be seen that there is almost no intersection between low-score second language speakers and native speakers in the figure.

[0125] It can be further seen from Table 1 that %V, ΔV and rateV can also distinguish between native speakers and low-scoring bilingual speakers. Therefore, based on the above embodiments, as an optional embodiment, the rhythmic feature also includes: one or more of the proportion of vowel segment duration, the standard deviation of vowel segment duration and the rate of vowel segment.

[0126] It should be noted that, in addition to the rhythm of the stress affecting the quality of the spoken rhythm, the reader's control of the pauses can assist in the evaluation of the quality of the rhythm. Therefore, after obtaining the spoken audio, the embodiment of the present application also includes: marking the pause segments in the spoken audio, and taking the number and total duration of the pause segments as rhythm features. In the embodiment of the present application, the longer the duration of the pause segments and the more the number, the less ideal the rhythm.

[0127] Based on the above embodiments, as an optional embodiment, the rhythm of the spoken audio is evaluated according to the rhythm feature, including:

[0128] S201. Perform weighted summation on rhythm features to obtain a rhythm score.

[0129] For example, if continuous vowel segments are normalized into pair difference index, vowel segment duration variation coefficient, vowel segment rate, number and duration of pause segments, a total of five parameters are used as rhythm features. By setting weights for each parameter and then summing the weighted results, a weighted summation result can be obtained.

[0130] It should be noted that the weighted value of the rhythm feature in the embodiment of the present application is related to the significance of distinguishing the rhythm. The stronger the significance, the higher the weighted value. For example, since the normalized pairwise difference index nPVI_V of continuous vowel segments and the coefficient of variation of vowel segment duration VarcoV are the most significant in Table 1, the weighted values ​​of these two rhythm features are positive numbers and have the largest values ​​among all optional rhythm features. The number and duration of pause segments have a negative impact on the rhythm, so the weighted value is negative. Considering that there are generally not too many pause segments when reading a sentence, the absolute value of the weight corresponding to the pause segment can be It is set to be greater than 1, and the duration of a speech segment is usually measured in milliseconds, so the absolute value of the weighted value can be two or three decimal places. Specifically, the weight of the normalized pairwise difference index nPVI_V of continuous vowel segments can be set to 1, the weight of the vowel segment duration variation coefficient VarcoV can be set to 1, the weight of the vowel segment rate rateV can be set to 0.2, the weight of the number of pause segments nS can be set to -3, and the weight of the total duration of the pause segments sumS can be set to -0.05. Then the rhythm score is: nPVI_V+VarcoV+rateV*0.2-nS*3-sumS*0.05.

[0131] S202: Determine a scoring interval in which the rhythm score is located, and determine the rating of the spoken audio according to a predetermined correspondence between the scoring interval and the rating.

[0132] Optionally, after obtaining the rhythm score, the embodiment of the present application can also determine the rating of the spoken audio in combination with the correspondence between the preset score interval and the rating. For example, 0 to 20 points are preset as level 1, 21 to 40 points are preset as level 2, 41 to 60 points are preset as level 3, 61 to 80 points are preset as level 4, and 80 points or more are preset as level 5. By determining the interval where the rhythm score is located, the corresponding rhythm rating can be determined. Obviously, the higher the rating (score), the better the rhythm of the reading.

[0133] In addition, the embodiment of the present application may also set a preset threshold. If the rhythm score is not less than the first preset threshold, the rhythm of the spoken audio is determined to be correct; if the rhythm score is less than the first preset threshold, the rhythm of the spoken audio is determined to be wrong.

[0134] In the embodiment of the present application, a large number of spoken audio samples of native speakers are collected and then the rhythm scores of the spoken audio samples are calculated. The mean of the rhythm scores is multiplied by a preset ratio (e.g., 0.6) as the first preset threshold. In addition, the rhythm scores can be weighted according to information such as the age of the native speakers, and the mean of the weighted summation result is multiplied by the preset ratio as the first preset threshold. For example, for young or old native speakers, the weight of the corresponding rhythm score is smaller, while the weight of the rhythm score for young and middle-aged native speakers is higher.

[0135] In order to verify whether the rhythm of spoken audio is evaluated by scoring, the present embodiment of the present invention is to Figure 4-7 The values ​​of nPVI_V, VarcoV, nS and sumS are calculated for the spoken audio respectively, and multiplied by the corresponding weights (the weights of the four parameters are 1, 1, -3 and -0.05 respectively) to obtain the rhythm score. At the same time, the four spoken audios are scored by experts. The final results are shown in Table 2.

[0136]

[0137] Table 2 Oral audio scoring table

[0138] Since English is a stress-rhythm language, the nPVI_V value is usually between 50 and 80. Based on the nPVI_V values ​​of the second language learners in the test set, the expert scores can be roughly given based on the nPVI_V values, as shown in Table 3:

[0139]

[0140]

[0141] Table 3 Correspondence between nPVI_V values ​​and expert scores

[0142] According to Table 3, the expert scores of the two native speakers and the second Chinese learner speaker are both 5 points, while the expert score of the first Chinese learner speaker is 1 point, which is basically consistent with the expert's prosody score. It can be understood that the embodiment of the present application can equate the expert score with the rating, that is, the expert score of 1 point is equal to the rating of level 1.

[0143] By multiplying the rhythm scores of the two English native speakers by the preset ratio 0.6, a threshold for judging whether the rhythm is wrong can be obtained. In this embodiment, the threshold is 75. It can be seen from the relationship between the rhythm scores of the two Chinese second language learners and the threshold that the rhythm score of Chinese second language learner 1 is 11, which is less than the threshold 75, and the expert score is 1, which is the lowest rating, which is very different from the expert score of the English native speaker. The rhythm score of Chinese second language learner 2 is 86, which is greater than the threshold 75, and the expert score is 5, which is the highest rating, which is also consistent with the expert score of the English native speaker. Therefore, the result of obtaining the rhythm score and evaluating the rhythm of the spoken audio according to the rhythm score in the embodiment of the present application is consistent with the result of the expert score, and it is feasible to evaluate the rhythm of the spoken audio by obtaining the rhythm score.

[0144] Fig. 9 A flow chart of a spoken language evaluation method provided in another embodiment of the present application is shown in FIG. Fig. 9 As shown:

[0145] 1) The user first opens a language learning application in the terminal and inputs a sentence or a text to be read aloud in the application;

[0146] 2) The terminal starts recording in response to the user clicking a start recording button in the application, and then ends recording in response to the user clicking an end recording button in the application, thereby obtaining spoken audio;

[0147] 3) The terminal sends the text and spoken audio to the server;

[0148] 4) The server obtains rhythm features in the spoken audio, performs weighted summation on the rhythm features to obtain a rhythm score, and determines whether the rhythm of the spoken audio is wrong based on the relationship between the rhythm score and a first preset threshold;

[0149] 5) The server determines the target word from the text and determines whether the rhythm of the target word is wrong based on the vowel segment of the spoken audio;

[0150] 6) The server returns the rhythm score, the rhythm evaluation results of the spoken audio, and the rhythm evaluation results of the target vocabulary to the terminal for display.

[0151] Based on the above embodiments, as an optional embodiment, the spoken language evaluation method further includes the following steps:

[0152] S301, obtaining the text corresponding to the spoken audio, and determining the target vocabulary in the text.

[0153] Optionally, the embodiment of the present application does not make any specific limitation on the timing of step S301. For example, it can be executed simultaneously with step S101, or it can be executed after any one of steps S101 to S103. Preferably, considering that the purpose of this embodiment is to evaluate the reading rhythm of specific words, step S301 can be executed after determining that the rhythm of the spoken audio is wrong based on the rhythm characteristics, that is, when the rhythm of the spoken audio is wrong, the reading rhythm of the specific word is evaluated.

[0154] The target vocabulary in the embodiment of the present application can be each vocabulary in the text, or it can be part of the vocabulary in the text. For Chinese students, it is most likely to make mistakes on the syllables that should be weakly pronounced when reading English aloud, such as function words such as articles and definite articles. Therefore, the embodiment of the present application can pre-construct a vocabulary list that is easy to read aloud incorrectly, and then after obtaining the text, the text is divided into individual vocabulary by word segmentation technology, and then each vocabulary is traversed in the vocabulary list, and if a certain vocabulary appears in the vocabulary list, it is used as the target vocabulary. In addition, the vocabulary selected by the user in the text can also be received as the target vocabulary, so that the setting can meet the personalized oral evaluation needs of the user and improve the user's satisfaction.

[0155] S302: Determine a vowel segment corresponding to a target vocabulary from the spoken audio as a target segment.

[0156] It should be understood that the vowel segments corresponding to the target vocabulary can be determined from the spoken audio using the speech annotation tool. Figure 4 For example, from Figure 4 From the third and fourth layers in the figure, we can see that the first vowel segment is the vowel segment corresponding to the word what, and the second vowel segment is the vowel segment corresponding to the word do.

[0157] S303: Evaluate the rhythm of the target vocabulary according to the duration of the target segment and the vowel segment adjacent to the target segment.

[0158] The vowel segment adjacent to the target segment may be a vowel segment before the target segment, a vowel segment after the target segment, or two vowel segments on both sides of the target segment, which is not specifically limited in the embodiment of the present application. Preferably, the adjacent vowel segments are two vowel segments on both sides of the target segment, so that the evaluation of the rhythm of the target vocabulary has more vowel segments for reference and the result is more accurate.

[0159] Specifically, a vowel segment before the target segment is taken as the left vowel segment, and a vowel segment after the target segment is taken as the right vowel segment, and the difference in duration between the target segment and the left vowel segment and the difference in duration between the target segment and the right vowel segment are calculated respectively, and if at least one of the two differences is greater than a second preset threshold, it is determined that the rhythm of the target vocabulary is correct, and if both differences are less than the second preset threshold, it is determined that the rhythm of the target vocabulary is wrong. The second preset threshold of the embodiment of the present application can be obtained by statistically evaluating a large number of native speakers' spoken audio.

[0160] exist Figure 3 Based on the embodiment shown, Fig.10 This is a schematic diagram of a terminal display evaluation result interface in an embodiment of the present application, see Fig.10 , the text content marks the target words know, the, fact, and the stress rhythm and pause rhythm of know. The stress rhythm of know, fact, and know is correct and the font is italic, while the stress of the is wrong, the font of the is not italicized, and the pause of the is wrong, so a vertical line is used after the word the to indicate the pause error, while the pause of fact is correct, so two vertical lines are used after the word fact to indicate the pause is correct, and Fig.10 The intonation and the judgment result of whether the intonation is correct are also identified. For example, there is a wavy line symbol above the and fact, indicating that the intonation of the and fact is falling in the spoken audio, but the is a hollow wavy line symbol, and fact is a broken wavy line symbol, indicating that the intonation of the is wrong, while the intonation of fact is correct; there is a double waveform symbol above know, indicating that the intonation of know in the spoken audio is rising, and the double waveform symbol is a solid double waveform symbol, indicating that the intonation of know is correct. The embodiment of the present application can identify the intonation of the target vocabulary and whether the intonation is correct through a speech recognition tool. Fig.10 In the , the rhythm scores and ratings of the spoken audio are also shown. Fig.10 It can be seen that the user's rhythm rating is 4 hearts and the rhythm score is 80 points.

[0161] The present application embodiment provides a spoken language evaluation device, such as Fig.11 As shown, the spoken language evaluation device may include: an audio annotation module 101, a rhythm feature acquisition module 102 and an audio rhythm determination module 103, wherein:

[0162] The audio annotation module 101 is used to obtain spoken audio and annotate at least one vowel segment of the spoken audio.

[0163] The spoken audio is the audio generated by the user reading aloud the text for spoken language evaluation. It should be understood that the embodiment of the present application needs to evaluate the rhythm of spoken language, so the text is preferably in the form of sentences or paragraphs.

[0164] The oral evaluation device can be built into a terminal or a server. When built into a terminal, the terminal first displays the text for oral evaluation on the display interface, and the oral evaluation device collects the audio generated when the user reads the text as the oral audio, and performs subsequent processing. The text displayed by the terminal can be the text entered by the user himself by typing, taking a photo, copying and pasting, etc., or it can be the text sent by the server, which is not specifically limited in the embodiments of the present application.

[0165] When built into a terminal, after collecting spoken audio, the terminal will send the spoken audio to a spoken language evaluation device in a server, and the spoken language evaluation device will execute the technical solution.

[0166] The rhythm feature acquisition module 102 is used to obtain the rhythm feature of the spoken audio according to the duration of at least one vowel segment.

[0167] Since English is a stress rhythm language, unstressed syllables are required to be read weakly. The duration of unstressed syllables is shortened accordingly according to the interval of stress, so as to ensure that adjacent syllables are not all stressed as much as possible, and the stressed syllables are kept as close as possible in consideration of the correct expression of semantics and the shortening of the length of each unstressed syllable between stresses. Chinese is a syllable rhythm language, and the pronunciation of each syllable is in place. One of the biggest prosody problems for Chinese students speaking English is the rhythm problem. The main reason why Chinese students are affected by their mother tongue and cannot produce a rhythm similar to that of native speakers is that unstressed syllables are not weakened. Therefore, the embodiment of the present application obtains the rhythm characteristics of spoken audio according to the number and duration of vowel segments.

[0168] The audio rhythm determination module 103 is used to evaluate the rhythm of the spoken audio according to the rhythm features.

[0169] After obtaining the rhythm features, the embodiments of the present application can determine whether the rhythm of the spoken audio is wrong based on the rhythm features. For example, the duration difference between adjacent vowel segments is a rhythm feature. If the number of audio differences between adjacent vowel segments in the spoken audio that is greater than a preset threshold meets the preset conditions, then the rhythm of the spoken audio is determined to be correct. For example, there are 6 vowel segments marked in a certain spoken audio, so there are 5 audio differences. If there are more than 3 audio differences that are greater than the preset threshold, then the rhythm of the spoken audio is correct.

[0170] The oral evaluation device provided in the embodiment of the present application specifically executes the above-mentioned method embodiment process. For details, please refer to the contents of the above-mentioned oral evaluation method embodiment, which will not be repeated here. The oral evaluation device provided in the embodiment of the present application obtains spoken audio and marks the vowel segments of the spoken audio. Compared with the existing method of directly manually marking stress, it can avoid the influence of subjective perception on oral evaluation, and is also suitable for the execution of massive oral evaluation tasks. According to the characteristics that the segment lengths of stressed rhythm languages ​​are significantly different, while the segment lengths of syllable rhythm speech are almost the same, the embodiment of the present application further obtains the rhythm features of the spoken audio according to the duration of the vowel segments, and judges whether the rhythm of the spoken audio is wrong by the rhythm features, so as to more accurately judge the rhythm level of spoken language.

[0171] Based on the above embodiments, as an optional embodiment, the vowel rhythm feature includes: a normalized pairwise difference index of continuous vowel segments and a vowel segment duration variation coefficient.

[0172] On the basis of the above embodiments, as an optional embodiment, the vowel rhythm feature further includes: one or more of the vowel segment duration ratio, the standard deviation of the vowel segment duration and the vowel segment rate.

[0173] It should be noted that in addition to the rhythm of the stress affecting the quality of the spoken rhythm, the reader's control of the pauses can assist in the evaluation of the quality of the rhythm. Therefore, the audio annotation module is also used to: annotate the pause segments in the spoken audio; and the number and total duration of the pause segments are also used as rhythm features. In the embodiment of the present application, the longer the duration of the pause segments and the more the number, the less ideal the rhythm.

[0174] Based on the above embodiments, as an optional embodiment, the audio rhythm determination module includes:

[0175] The rhythm score acquisition unit is used to perform weighted summation on the rhythm features to obtain a rhythm score.

[0176] For example, if continuous vowel segments are normalized into pair difference index, vowel segment duration variation coefficient, vowel segment rate, number and duration of pause segments, a total of five parameters are used as rhythm features. By setting weights for each parameter and then summing the weighted results, a weighted summation result can be obtained.

[0177] It should be noted that the weighted value of the rhythm feature in the embodiment of the present application is related to the significance of distinguishing the rhythm. The stronger the significance, the higher the weighted value. For example, since the normalized pairwise difference index nPVI_V of continuous vowel segments and the coefficient of variation of vowel segment duration VarcoV are the most significant in Table 1, the weighted values ​​of these two rhythm features are positive numbers and have the largest values ​​among all optional rhythm features. The number and duration of pause segments have a negative impact on the rhythm, so the weighted value is negative. Considering that there are generally not too many pause segments when reading a sentence, the absolute value of the weight corresponding to the pause segment can be It is set to be greater than 1, and the duration of a speech segment is usually measured in milliseconds, so the absolute value of the weighted value can be two or three decimal places. Specifically, the weight of the normalized pairwise difference index nPVI_V of continuous vowel segments can be set to 1, the weight of the vowel segment duration variation coefficient VarcoV can be set to 1, the weight of the vowel segment rate rateV can be set to 0.2, the weight of the number of pause segments nS can be set to -3, and the weight of the total duration of the pause segments sumS can be set to -0.05. Then the rhythm score is: nPVI_V+VarcoV+rateV*0.2-nS*3-sumS*0.05.

[0178] The threshold comparison unit is used to determine the scoring interval where the rhythm score is located, and determine the rating of the spoken audio according to the predetermined correspondence between the scoring interval and the rating.

[0179] For example, 0 to 20 points are preset as level 1, 21 to 40 points are preset as level 2, 41 to 60 points are preset as level 3, 61 to 80 points are preset as level 4, and 80 points and above are preset as level 5. By determining the interval of the rhythm score, the corresponding rhythm rating can be determined. Obviously, the higher the rating (score), the better the rhythm of the reading.

[0180] right Figure 4-7 The values ​​of nPVI_V, VarcoV, nS, and sumS are calculated for the spoken audio, and multiplied by the corresponding weights (the weights of the four parameters are 1, 1, -3, and -0.05, respectively), and the rhythm score is obtained as follows:

[0181] Native English speaker 1: 131 points; Native English speaker 2: 118 points; Chinese second language learner 1: 11 points, Chinese second language learner 2: 86 points; Combining the above scoring ranges, it can be seen that the rhythm rating of the two native English speakers and Chinese second language learner 1 is level 5, while the rhythm rating of Chinese second language learner 2 is level 1.

[0182] In the embodiment of the present application, a large number of spoken audio samples of native speakers are collected and then the rhythm scores of the spoken audio samples are calculated. The mean of the rhythm scores is multiplied by a preset ratio (e.g., 0.6) as the first preset threshold. In addition, the rhythm scores can be weighted according to information such as the age of the native speakers, and the mean of the weighted summation result is multiplied by the preset ratio as the first preset threshold. For example, for young or old native speakers, the weight of the corresponding rhythm score is smaller, while the weight of the rhythm score for young and middle-aged native speakers is higher.

[0183] Based on the above embodiments, as an optional embodiment, the spoken language evaluation device further includes:

[0184] The vocabulary acquisition module is used to obtain the text corresponding to the spoken audio and determine the target vocabulary in the text.

[0185] Optionally, the embodiment of the present application does not make any specific limitation on the execution sequence of the vocabulary acquisition module. For example, it can be executed simultaneously with the audio annotation module, or it can be executed after any one of the audio annotation module, the rhythm feature acquisition module and the audio rhythm judgment module is executed. Preferably, considering that the purpose of this embodiment is to evaluate the reading rhythm of specific words, the step of the vocabulary acquisition module can be executed after judging that the rhythm of the spoken audio is wrong according to the rhythm features, that is, when the rhythm of the spoken audio is wrong, the reading rhythm of the specific word is evaluated.

[0186] The target vocabulary in the embodiment of the present application can be each vocabulary in the text, or it can be part of the vocabulary in the text. For Chinese students, it is most likely to make mistakes on the syllables that should be weakly pronounced when reading English aloud, such as function words such as articles and definite articles. Therefore, the embodiment of the present application can pre-construct a vocabulary list that is easy to read aloud incorrectly, and then after obtaining the text, the text is divided into individual vocabulary by word segmentation technology, and then each vocabulary is traversed in the vocabulary list, and if a certain vocabulary appears in the vocabulary list, it is used as the target vocabulary. In addition, the vocabulary selected by the user in the text can also be received as the target vocabulary, so that the setting can meet the personalized oral evaluation needs of the user and improve the user's satisfaction.

[0187] The target segment acquisition module is used to determine the vowel segment corresponding to the target vocabulary from the spoken audio as the target segment.

[0188] It should be understood that the vowel segments corresponding to the target vocabulary can be determined from the spoken audio using the speech annotation tool. Figure 4 For example, from Figure 4 From the third and fourth layers in the figure, we can see that the first vowel segment is the vowel segment corresponding to the word what, and the second vowel segment is the vowel segment corresponding to the word do.

[0189] The vocabulary rhythm judgment module is used to judge whether the rhythm of the target vocabulary is wrong according to the duration of the target segment and the vowel segment adjacent to the target segment.

[0190] The vowel segment adjacent to the target segment may be a vowel segment before the target segment, a vowel segment after the target segment, or two vowel segments on both sides of the target segment, which is not specifically limited in the embodiment of the present application. Preferably, the adjacent vowel segments are two vowel segments on both sides of the target segment, so that the evaluation of the rhythm of the target vocabulary has more vowel segments for reference and the result is more accurate.

[0191] Based on the above embodiments, as an optional embodiment, the vocabulary rhythm determination module includes:

[0192] A difference calculation unit, used to calculate the difference between the duration of the target segment and the two adjacent vowel segments before and after the target segment;

[0193] The rhythm judgment unit is used to determine that the rhythm of the target vocabulary is correct if at least one of the two differences is greater than a second preset threshold, and to determine that the rhythm of the target vocabulary is wrong if both differences are less than the second preset threshold.

[0194] In the embodiment of the present application, a vowel segment before the target segment is taken as the left vowel segment, and a vowel segment after the target segment is taken as the right vowel segment, and the difference between the duration of the target segment and the left vowel segment and the difference between the duration of the target segment and the right vowel segment are calculated respectively. If at least one of the two differences is greater than a second preset threshold, it is determined that the rhythm of the target vocabulary is correct. If both differences are less than the second preset threshold, it is determined that the rhythm of the target vocabulary is wrong. The second preset threshold of the embodiment of the present application can be obtained by statistically evaluating a large number of native speakers' spoken audio.

[0195] An electronic device is provided in an embodiment of the present application, and the electronic device includes: a memory and a processor; at least one program, stored in the memory, for being executed by the processor, which can achieve the following compared to the prior art: by acquiring spoken audio and marking the vowel segments of the spoken audio, compared to the prior art method of directly marking stress manually, it can avoid the influence of subjective perception on the oral evaluation, and is also suitable for the execution of massive oral evaluation tasks. According to the characteristics that the segment lengths of stressed rhythmic languages ​​are significantly different, while the segment lengths of syllable rhythmic speech are almost the same, the embodiment of the present application further obtains the rhythm features of the spoken audio according to the number and duration of the vowel segments, and judges whether the rhythm of the spoken audio is wrong by the rhythm features, so as to more accurately judge the rhythm level of the spoken language.

[0196] In an alternative embodiment, an electronic device is provided, such as Fig.12 As shown, Fig.12The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0197] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0198] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0199] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0200] The memory 4003 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the contents shown in the above method embodiment.

[0201] The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment. Compared with the prior art, by acquiring spoken audio and marking the vowel segments of the spoken audio, compared with the existing method of directly manually marking stress, it can avoid the influence of subjective perception on oral evaluation, and is also suitable for the execution of massive oral evaluation tasks. According to the characteristics that the segment lengths of stressed-rhythm languages ​​are significantly different, while the segment lengths of syllable-rhythm speech are almost the same, the embodiment of the present application further obtains the rhythm features of the spoken audio according to the number and duration of the vowel segments, and judges whether the rhythm of the spoken audio is wrong by the rhythm features, so as to more accurately judge the rhythm level of the spoken language.

[0202] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0203] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A spoken language assessment method, characterized in that: include: Acquire spoken audio, and mark at least one vowel segment of the spoken audio; According to the duration of the at least one vowel segment, and with the goal of distinguishing the prominence of the rhythm of a second language speaker from that of a native speaker, a rhythm feature of the spoken audio is obtained; the rhythm feature includes: a coefficient of variation of the duration of the vowel segment; evaluating the rhythm of the spoken audio according to the rhythm feature; If it is determined that the rhythm of the spoken audio is wrong, obtaining a text corresponding to the spoken audio, and determining a target vocabulary in the text that appears in a pre-created vocabulary list that is easy to mispronounce; Determining a vowel segment corresponding to the target vocabulary from the spoken audio as a target segment; respectively calculating the difference in duration between the target segment and two adjacent vowel segments before and after the target segment; If at least one of the two differences is greater than a second preset threshold, it is determined that the rhythm of the target vocabulary is correct; if both of the two differences are less than the second preset threshold, it is determined that the rhythm of the target vocabulary is wrong.

2. The oral language evaluation method according to claim 1, characterized in that: The rhythm features include: a vowel segment duration variation coefficient.

3. The oral language evaluation method according to claim 2, characterized in that: The vowel rhythm feature also includes: one or more of the vowel segment duration ratio, the standard deviation of the vowel segment duration, and the vowel segment rate.

4. The oral language evaluation method according to claim 2, characterized in that: The step of obtaining the spoken audio further includes: marking pause segments in the spoken audio; The rhythm feature also includes the number and total duration of the pause segments.

5. The oral language evaluation method according to any one of claims 2 to 4, characterized in that: The evaluating the rhythm of the spoken audio according to the rhythm feature includes: Performing weighted summation on the rhythm features to obtain a rhythm score; Determine a scoring interval in which the rhythm score is located, and determine the rating of the spoken audio according to a predetermined correspondence between the scoring interval and the rating.

6. A spoken language evaluation device, characterized in that: include: An audio annotation module, used to obtain spoken audio and annotate at least one vowel segment of the spoken audio; A rhythm feature acquisition module is used to obtain the rhythm feature of the spoken audio according to the duration of the at least one vowel segment and with the goal of distinguishing the significance of the rhythm between the second language and the native speaker; wherein the rhythm feature includes: a vowel segment duration variation coefficient; An audio rhythm determination module, used for evaluating the rhythm of the spoken audio according to the rhythm features; A vocabulary acquisition module, for acquiring a text corresponding to the spoken audio if it is determined that the rhythm of the spoken audio is wrong, and determining a target vocabulary in the text that appears in a pre-created vocabulary list that is easy to mispronounce; A target segment acquisition module, used to determine a vowel segment corresponding to the target vocabulary from the spoken audio as a target segment; A difference calculation unit, used to respectively calculate the difference in duration between the target segment and two adjacent vowel segments before and after the target segment; The rhythm judgment unit is used to determine that the rhythm of the target vocabulary is correct if at least one of the two differences is greater than a second preset threshold, and to determine that the rhythm of the target vocabulary is wrong if both of the two differences are less than the second preset threshold.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the spoken language evaluation method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the steps of the spoken language evaluation method according to any one of claims 1 to 5.

Images