Voice evaluation method, related equipment and readable storage medium

Abstract

The present application discloses a speech evaluation method, related equipment and readable storage medium. After obtaining the speech to be evaluated, the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are determined, and the evaluation result of the speech to be evaluated is determined according to the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated. Since the above-mentioned multiple aspects of features can characterize the various abilities of the speaker of the speech to be evaluated, the method can be used for oral evaluation of the shadow reading method.

Description

Technical Field

[0001] The present application relates to the technical field of spoken language evaluation, and more specifically, to a speech evaluation method, related equipment and a readable storage medium. Background Art

[0002] Oral assessment technology can evaluate the speaker's oral proficiency and guide the speaker to learn based on the evaluation results. The basic principle of the assessment is to measure the difference between standard pronunciation and actual pronunciation. Compared with manual scoring methods, it can significantly improve the objectivity and fairness of the assessment test, while greatly reducing the cost of manpower and material resources, making it possible to evaluate the oral ability of a large number of people. Therefore, it has great development potential and application prospects, and has also received widespread attention from researchers in recent years.

[0003] Shadowing is a commonly used method in oral practice. This method provides learners with demonstration sounds and requires them to wear headphones and repeat the sounds they hear in the same voice, intonation and speed while listening to the demonstration sounds. This method tests learners' various abilities (such as target language pronunciation level, listening, language organization ability, reaction ability, memory and reading ability, etc.). Currently, most commonly used oral assessment technologies only assess learners' target language pronunciation level based on the completeness, fluency and standardization of the learners' reading. The assessment ability is single and cannot achieve oral assessment of shadowing.

[0004] Therefore, how to provide a spoken language assessment technology suitable for the shadow reading method has become a technical problem that needs to be solved urgently by technical personnel in this field. Summary of the invention

[0005] In view of the above problems, this application proposes a speech evaluation method, related equipment and readable storage medium. The specific scheme is as follows:

[0006] A speech evaluation method, the method comprising:

[0007] Get the voice to be evaluated;

[0008] Determining language organization assessment features, segment assessment features, and follow-up reading rhythm assessment features of the speech to be assessed;

[0009] An evaluation result of the speech to be evaluated is determined according to the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0010] Optionally, the determining of the language organization assessment feature, the segment assessment feature, and the follow-up rhythm assessment feature of the speech to be assessed includes:

[0011] Recognize the speech to be evaluated and obtain recognized text information;

[0012] Determining language organization evaluation features of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network;

[0013] Based on a pre-constructed limited text decoding network with a timestamp, the speech to be evaluated is decoded to determine the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0014] Optionally, the semantic network is constructed as follows:

[0015] Obtaining a sample sound as an evaluation standard, an original text corresponding to the sample sound, and a translation text in a target language corresponding to the original text;

[0016] A semantic network is constructed based on the original text and the translated text.

[0017] Optionally, constructing a semantic network based on the original text and the translated text includes:

[0018] Performing reverse translation on the translation text to determine a candidate original text;

[0019] Determine a text set based on the original text and the candidate original text;

[0020] A semantic network is constructed based on the text collection, wherein the semantic network includes a plurality of parallel semantic paths, and each semantic path corresponds to a text in the text collection.

[0021] Optionally, determining the language organization assessment features of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network includes:

[0022] Calculating the content coverage of each semantic path of the recognized text information in the semantic network;

[0023] Based on the content coverage of each semantic path in the semantic network of the recognized text information, the language organization evaluation features of the speech to be evaluated are determined.

[0024] Optionally, the timestamped limited text decoding network is constructed as follows:

[0025] Based on the demonstration sound and the original text, a time-stamped limited text decoding network is established.

[0026] Optionally, the establishing a time-stamped limited text decoding network based on the demonstration sound and the original text includes:

[0027] Establishing a limited text decoding network based on the original text;

[0028] Determining a speech phoneme-based timestamp of the demonstration sound;

[0029] The speech phoneme-based timestamp of the demonstration sound is integrated into the limited text decoding network to construct a limited text decoding network with a timestamp.

[0030] Optionally, the time-stamped limited text decoding network includes a start layer, an end layer, a speech phoneme unit layer, and an absorption layer between all the aforementioned layers;

[0031] Each of the phonetic phoneme unit layers is constructed according to the phonetic phonemes of each single character in the original text; each of the phonetic phoneme unit layers includes the timestamp of the phonetic phoneme in the demonstration sound;

[0032] There may be jump arcs between each of the speech phoneme unit layers and each absorption layer, each speech phoneme unit layer and the end layer.

[0033] Optionally, the training method of the time-stamped limited text decoding network includes:

[0034] Acquiring a training voice corresponding to the demonstration sound;

[0035] For each training speech, determine a follow-up rhythm evaluation feature label and a segment evaluation feature label of each speech phoneme in the training speech, wherein the follow-up rhythm evaluation feature label is used to indicate whether the timestamp of the speech phoneme in the training speech is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound; and the segment evaluation feature label is used to indicate the score of each jump arc corresponding to the speech phoneme in the training speech;

[0036] The training speech and the phoneme-based timestamp of the training speech are used as training samples, and the reading rhythm evaluation feature label and the segment evaluation feature label are used as training labels to train and obtain the timestamp-containing limited text decoding network.

[0037] Optionally, determining the evaluation result of the speech to be evaluated according to the language organization evaluation feature, the segment evaluation feature and the follow-up rhythm evaluation feature of the speech to be evaluated includes:

[0038] The language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are input into a speech evaluation model, and the speech evaluation model outputs the evaluation results of the speech to be evaluated. The speech evaluation model is trained using the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the training speech as training samples, and using the evaluation results annotated by the training speech as sample labels.

[0039] Optionally, after determining the reading rhythm assessment feature of the speech to be evaluated, the method further includes:

[0040] Based on the follow-up rhythm evaluation characteristics of the speech to be evaluated, follow-up rhythm feedback information is generated, and the follow-up rhythm feedback information is provided to the speaker of the speech to be evaluated to prompt the speaker of the current follow-up rhythm.

[0041] A speech evaluation device, comprising:

[0042] An acquisition unit, used for acquiring the speech to be evaluated;

[0043] A feature determination unit, used to determine the language organization evaluation feature, the segment evaluation feature and the follow-up rhythm evaluation feature of the speech to be evaluated;

[0044] The evaluation result determination unit is used to determine the evaluation result of the speech to be evaluated according to the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0045] Optionally, the feature determination unit includes:

[0046] A recognition unit, used for recognizing the speech to be evaluated and obtaining recognized text information;

[0047] A language organization evaluation feature determination unit, used to determine the language organization evaluation feature of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network;

[0048] The segment evaluation feature and follow-up rhythm evaluation feature determination unit is used to decode the speech to be evaluated based on a pre-constructed limited text decoding network with timestamp, and determine the segment evaluation feature and follow-up rhythm evaluation feature of the speech to be evaluated.

[0049] Optionally, the device further comprises: a semantic network construction unit; the semantic network construction unit comprises:

[0050] An acquisition subunit, used to acquire a demonstration sound as an evaluation standard, an original text corresponding to the demonstration sound, and a translation text in a target language corresponding to the original text;

[0051] The construction subunit is used to construct a semantic network based on the original text and the translated text.

[0052] Optionally, the construction subunit is specifically used for:

[0053] Performing reverse translation on the translation text to determine a candidate original text;

[0054] Determine a text set based on the original text and the candidate original text;

[0055] A semantic network is constructed based on the text collection, wherein the semantic network includes a plurality of parallel semantic paths, and each semantic path corresponds to a text in the text collection.

[0056] Optionally, the language organization assessment feature determination unit includes:

[0057] A calculation unit, used for calculating the content coverage of each semantic path of the recognized text information in the semantic network;

[0058] The language organization evaluation feature determination subunit is used to determine the language organization evaluation feature of the speech to be evaluated based on the content coverage of each semantic path in the semantic network of the recognized text information.

[0059] Optionally, the device further comprises: a time-stamped limited text decoding network construction unit; the time-stamped limited text decoding network construction unit is used to:

[0060] Based on the demonstration sound and the original text, a time-stamped limited text decoding network is established.

[0061] Optionally, the time-stamped limited text decoding network construction unit includes:

[0062] A limited text decoding network establishing unit, used for establishing a limited text decoding network based on the original text;

[0063] A timestamp determination unit, used to determine a timestamp of the demonstration sound based on speech phonemes;

[0064] The incorporation unit is used to incorporate the speech phoneme-based timestamp of the demonstration sound into the limited text decoding network to construct a limited text decoding network with a timestamp.

[0065] Optionally, the time-stamped limited text decoding network includes a start layer, an end layer, a speech phoneme unit layer, and an absorption layer between all the aforementioned layers;

[0066] Each of the phonetic phoneme unit layers is constructed according to the phonetic phonemes of each single character in the original text; each of the phonetic phoneme unit layers includes the timestamp of the phonetic phoneme in the demonstration sound;

[0067] There may be jump arcs between each of the speech phoneme unit layers and each absorption layer, each speech phoneme unit layer and the end layer.

[0068] Optionally, the training method of the time-stamped limited text decoding network includes:

[0069] Acquiring a training voice corresponding to the demonstration sound;

[0070] For each training speech, determine a follow-up rhythm evaluation feature label and a segment evaluation feature label of each speech phoneme in the training speech, wherein the follow-up rhythm evaluation feature label is used to indicate whether the timestamp of the speech phoneme in the training speech is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound; and the segment evaluation feature label is used to indicate the score of each jump arc corresponding to the speech phoneme in the training speech;

[0071] The training speech and the phoneme-based timestamp of the training speech are used as training samples, and the reading rhythm evaluation feature label and the segment evaluation feature label are used as training labels to train and obtain the timestamp-containing limited text decoding network.

[0072] Optionally, the evaluation result determination unit is used to:

[0073] The language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are input into a speech evaluation model, and the speech evaluation model outputs the evaluation results of the speech to be evaluated. The speech evaluation model is trained using the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the training speech as training samples, and using the evaluation results annotated by the training speech as sample labels.

[0074] Optionally, the device further comprises:

[0075] A feedback unit is used to generate follow-up rhythm feedback information based on the follow-up rhythm evaluation characteristics of the speech to be evaluated after determining the follow-up rhythm evaluation characteristics of the speech to be evaluated, and provide the follow-up rhythm feedback information to the speaker of the speech to be evaluated to prompt the speaker of the current follow-up rhythm.

[0076] A speech evaluation device, comprising a memory and a processor;

[0077] The memory is used to store programs;

[0078] The processor is used to execute the program to implement the various steps of the speech evaluation method described above.

[0079] A readable storage medium stores a computer program, and when the computer program is executed by a processor, each step of the speech evaluation method described above is implemented.

[0080] By means of the above technical scheme, the present application discloses a speech evaluation method, related equipment and readable storage medium. After obtaining the speech to be evaluated, the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are determined, and the evaluation result of the speech to be evaluated is determined according to the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated. Since the above-mentioned multiple aspects of features can characterize the various abilities of the speaker of the speech to be evaluated, this method can be used for oral evaluation of the shadow reading method. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0082] Figure 1 A flowchart of the speech evaluation method disclosed in the embodiment of the present application;

[0083] Figure 2 A schematic diagram of a semantic network disclosed in an embodiment of the present application;

[0084] Figure 3 A schematic diagram of a limited text decoding network disclosed in an embodiment of the present application;

[0085] Figure 4 A schematic diagram of the structure of a time-stamped limited text decoding network disclosed in an embodiment of the present application;

[0086] Figure 5 A schematic diagram of the structure of a speech evaluation device disclosed in an embodiment of the present application;

[0087] Figure 6 This is a hardware structure block diagram of a ** device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0088] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0089] Next, the speech evaluation method provided by the present application is introduced through the following embodiments.

[0090] Reference Figure 1 , Figure 1This is a flow chart of a speech evaluation method disclosed in an embodiment of the present application. The method may include:

[0091] Step S101: Acquire speech to be evaluated.

[0092] Specifically, taking the shadow learning method as an example, the voice to be evaluated can be a recording of the learner following the demonstration voice. In this step, the voice to be evaluated can be obtained by receiving it through a recording device, which can include a microphone, such as a head-mounted microphone.

[0093] Step S102: determining the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0094] In this application, the language organization assessment feature refers to a feature that can assess the language organization ability of the speaker of the speech to be assessed. The segment assessment feature and the follow-up rhythm assessment feature refer to features that can assess the listening, language organization ability, reaction ability, memory, and reading ability of the speaker of the speech to be assessed.

[0095] Step S103: determining the evaluation result of the speech to be evaluated according to the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0096] After determining the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated, the evaluation result of the speech to be evaluated can be determined according to the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated. There are many ways to implement it. For example, the relationship between the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated and the evaluation result of the speech to be evaluated can be learned based on a neural network model, and the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are input into the neural network model to determine the evaluation result of the speech to be evaluated. The relationship between the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated and the evaluation result of the speech to be evaluated can also be calculated based on a mathematical algorithm. This application does not make any limitation.

[0097] This embodiment discloses a speech evaluation method. After obtaining the speech to be evaluated, the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated are determined, and the evaluation result of the speech to be evaluated is determined according to the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated. Since the above-mentioned multiple aspects of features can characterize the various abilities of the speaker of the speech to be evaluated, the method can be used for oral evaluation of the shadow reading method.

[0098] In one embodiment of the present application, a specific implementation method of step S102, determining the language organization evaluation features, the segment evaluation features, and the follow-up rhythm evaluation features of the speech to be evaluated, is introduced. The method may include the following steps:

[0099] Step S201: Recognize the speech to be evaluated and obtain recognized text information.

[0100] Specifically, speech recognition can be performed on the speech to be evaluated to obtain the recognized text information. The process may include: extracting the acoustic features of the speech to be evaluated; inputting the acoustic features into a preset acoustic recognition model to obtain the text information corresponding to the speech to be evaluated output by the acoustic recognition model. Among them, the acoustic features are used for speech recognition, and the acoustic features are generally spectral features of speech data, such as Mel-Frequency Cepstral Coefficients (MFCC) features or Perceptual Linear Prediction (PLP) features. During the specific extraction, the speech to be evaluated can be framed in advance. Furthermore, the speech to be evaluated after framing is pre-emphasized. Finally, the spectral features of each frame of the speech to be evaluated are extracted in turn.

[0101] The acoustic recognition model may be an acoustic recognition model in the form of a neural network trained using a training corpus. For example, the acoustic recognition model may be a universal acoustic recognition model, that is, a universal acoustic recognition model trained using an existing training corpus.

[0102] Step S202: Determine the language organization assessment features of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network.

[0103] In the present application, a demonstration sound as an evaluation standard, an original text corresponding to the demonstration sound, and a translation text in a target language corresponding to the original text can be obtained; a semantic network is constructed based on the original text and the translation text.

[0104] As an implementable method, the process of constructing a semantic network based on the original text and the translated text may include the following process:

[0105] Step S301: performing reverse translation on the translation text to determine candidate original texts.

[0106] In this step, based on the Encode-decode bidirectional network, the translation text can be input through traditional translation technology to obtain the TopN machine reverse-translated original texts as candidate original texts.

[0107] Step S302: determining a text set based on the original text and the candidate original text.

[0108] In this step, if the candidate original texts do not include the original text, the two are combined to obtain a text set; if the candidate original texts include the original text, all candidate original texts are combined to obtain a text set.

[0109] Step S303: constructing a semantic network based on the text collection, wherein the semantic network includes a plurality of parallel semantic paths, and each semantic path corresponds to a text in the text collection.

[0110] For ease of understanding, the following table shows an example. The semantic network corresponding to this example can be found in Figure 2 shown.

[0111] Original text: I like to be your good friend forever Translated text: I think forever Do your good friend Semantic Web 1 I like to be your good friend forever Semantic Web 2 I like to make good friend with you forever

[0112] As an implementable method, the process of determining the language organization assessment features of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network may include the following steps:

[0113] Step S401: Calculate the content coverage of each semantic path of the recognized text information in the semantic network.

[0114] In the present application, the content coverage of each semantic path in the semantic network of the recognized text information is used to characterize the degree of overlap between the recognized text information and the content of each semantic path in the semantic network. LCS (Longest Common Subsequence) can usually be used to describe the similarity between two texts. In the present application, the LSC value of the text corresponding to each semantic path in the semantic network and the recognized text information can be calculated to characterize the content coverage of each semantic path in the semantic network of the recognized text information.

[0115] Step S402: Determine the language organization evaluation features of the speech to be evaluated based on the content coverage of each semantic path in the semantic network of the recognized text information.

[0116] In the present application, as an implementable method, the language organization evaluation feature of the speech to be evaluated can be determined based on the maximum content coverage. For example, the maximum content coverage can be determined as the language organization evaluation feature of the speech to be evaluated.

[0117] Step S203: Based on the pre-constructed limited text decoding network with timestamp, the speech to be evaluated is decoded to determine the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0118] In this step, the acoustic features of the speech to be evaluated can be extracted; the acoustic features can be input into a pre-constructed limited text decoding network with a timestamp, and the speech to be evaluated can be decoded by the pre-constructed limited text decoding network with a timestamp to determine the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0119] The following is a detailed introduction to the relevant content of the limited text decoding network with timestamp.

[0120] In this application, the construction method of the time-stamped limited text decoding network is as follows:

[0121] Based on the demonstration sound and the original text, a time-stamped limited text decoding network is established.

[0122] As an implementable method, the specific implementation method of establishing a time-stamped limited text decoding network based on the demonstration sound and the original text may include the following steps:

[0123] Step S501: establishing a limited text decoding network based on the original text.

[0124] See also Figure 3 , Figure 3 A schematic diagram of a limited text decoding network disclosed in an embodiment of the present application, comprising Figure 3 As shown, the text decoding network includes a starting layer S, an ending layer E, and a speech phoneme unit layer a i And the absorption layer FS between all the above layers, the starting layer S is used to receive the acoustic features of the speech frame to be decoded, each speech phoneme unit layer is constructed according to the speech phonemes of each single character in the original text; it is used to decode the acoustic features of the received speech frame, and the ending layer is used to output the decoding result. There may be jump arcs between each of the speech phoneme unit layers and each absorption layer, each speech phoneme unit layer and the ending layer, wherein the jump arc includes a front jump arc and a back jump arc.

[0125] It should be noted that for each speech phoneme unit layer, the corresponding jump arc has a score, and the scores of different jump arcs may be different. For example, if the jump arc jumps to the absorption layer, the jump arc has a penalty score, and if the jump arc jumps to other speech phoneme unit layers, the jump arc has a matching score. For each speech phoneme unit layer, the score of the corresponding jump arc may be a fixed value calculated by training or manual tuning.

[0126] Step S502: Determine the speech phoneme-based timestamp of the demonstration sound.

[0127] In the present application, a limited text decoding network can be used to decode the demonstration sound. During the decoding process, the sound segments of the absorption layer FS are forced to be aligned to obtain the speech phoneme-based timestamp of the demonstration sound.

[0128] Step S503: Integrate the speech phoneme-based timestamp of the demonstration sound into the limited text decoding network to construct a limited text decoding network with a timestamp.

[0129] In the present application, the timestamped limited text decoding network includes a starting layer, an ending layer, a speech phoneme unit layer, and an absorption layer between all the aforementioned layers; each speech phoneme unit layer is constructed according to the speech phonemes of each single character in the original text; each speech phoneme unit layer includes the timestamp of the speech phoneme in the demonstration sound; there may be jump arcs between each speech phoneme unit layer and each absorption layer, each speech phoneme unit layer and the ending layer, and each jump arc corresponds to a score. In addition, if the speech to be evaluated is in this application, whether the timestamp of each speech phoneme in the speech to be evaluated is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound can characterize the speaker's follow-up rhythm. Therefore, in this application, a follow-up rhythm score can be set between different layers in the limited text decoding network with timestamps. For example, if the timestamp of a speech phoneme in the speech to be evaluated is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound, it means that the speaker is following the demonstration sound at this moment, and the follow-up rhythm score can be set to 1. If the timestamp of a speech phoneme in the speech to be evaluated is inconsistent with the timestamp of the corresponding speech phoneme in the demonstration sound, it means that the speaker is not following the demonstration sound at this moment, and the follow-up rhythm score can be set to 0.

[0130] For easier understanding, please refer to the attached Figure 4 , Figure 4 The schematic diagram of the structure of a time-stamped limited text decoding network disclosed in the embodiment of the present application is shown in FIG.

[0131] Since in the present application, the timestamped limited text decoding network includes a speech phoneme-based timestamp of the demonstration sound, whether the timestamp is consistent with the speech phoneme-based timestamp in the actual speech of the speaker has an impact on the jump arc score corresponding to each speech phoneme unit layer and the follow-up rhythm score between different layers. Therefore, it is necessary to determine the impact of the consistency between the speech phoneme-based timestamp of the demonstration sound and the speech phoneme-based timestamp in the actual speech of the speaker on the jump arc score corresponding to each speech phoneme unit layer and the follow-up rhythm score between different layers. Therefore, in the present application, it is necessary to train the timestamped limited text decoding network, and based on the trained timestamped limited text decoding network, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated can be determined.

[0132] As an implementable method, the training method of a time-stamped limited text decoding network may include the following steps:

[0133] Step S601: Acquire training speech corresponding to the demonstration sound.

[0134] In the present application, the training speech corresponding to the demonstration sound may be a recording of a certain number of learners following the demonstration sound.

[0135] Step S602: for each training speech, determining a reading rhythm evaluation feature label and a segment evaluation feature label of each speech phoneme in the training speech.

[0136] In the present application, the follow-up rhythm evaluation feature label is used to indicate whether the timestamp of the speech phoneme in the training speech is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound; for example, if the timestamp of the speech phoneme in the training speech is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound, then the follow-up rhythm evaluation feature label is 1; if the timestamp of the speech phoneme in the training speech is inconsistent with the timestamp of the corresponding speech phoneme in the demonstration sound, then the follow-up rhythm evaluation feature label is 0.

[0137] In the present application, the segment evaluation feature label is used to represent the score of each jump arc corresponding to the speech phoneme in the training speech.

[0138] Step S603: using the training speech as a training sample, and using the reading rhythm evaluation feature label and the segment evaluation feature label as training labels, training to obtain the limited text decoding network with a timestamp.

[0139] The training goal is to make the reading rhythm scores between different layers in the limited text decoding network to be timestamped approach the reading rhythm evaluation feature label, and the score of the jump arc corresponding to each speech phoneme unit layer approach the segment evaluation feature label.

[0140] In one embodiment of the present application, a method for determining the evaluation result of the speech to be evaluated based on the language organization evaluation feature, the segment evaluation feature, and the follow-up rhythm evaluation feature of the speech to be evaluated is introduced, and the method may include:

[0141] The language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are input into a speech evaluation model, and the speech evaluation model outputs an evaluation result of the speech to be evaluated.

[0142] In the present application, the evaluation result of the speech to be evaluated may be a specific score, and the speaker's follow-up comments and improvement suggestions corresponding to the score. The follow-up comments and improvement suggestions corresponding to each score may be preset by professionals.

[0143] It should be noted that the speech evaluation model is obtained by training with the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the training speech as training samples, and the evaluation results of the training speech annotations as sample labels.

[0144] In one embodiment of the present application, after determining the follow-up rhythm evaluation characteristics of the speech to be evaluated, the method may also include: generating follow-up rhythm feedback information based on the follow-up rhythm evaluation characteristics of the speech to be evaluated, and providing the follow-up rhythm feedback information to the speaker of the speech to be evaluated to prompt the speaker of the current follow-up rhythm, so that the speaker can adjust the follow-up rhythm according to the prompt.

[0145] The speech evaluation device disclosed in the embodiment of the present application is described below. The speech evaluation device described below and the speech evaluation method described above can be referred to each other.

[0146] Reference Figure 5 , Figure 5 Schematic diagram of the structure of a speech evaluation device disclosed in the embodiment of the present application. Figure 5 As shown, the speech evaluation device may include:

[0147] An acquisition unit 51 is used to acquire the speech to be evaluated;

[0148] A feature determination unit 52, used to determine the language organization evaluation feature, the segment evaluation feature and the follow-up rhythm evaluation feature of the speech to be evaluated;

[0149] The evaluation result determination unit 53 is used to determine the evaluation result of the speech to be evaluated according to the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0150] As an implementable embodiment, the feature determination unit includes:

[0151] A recognition unit, used for recognizing the speech to be evaluated and obtaining recognized text information;

[0152] A language organization evaluation feature determination unit, used to determine the language organization evaluation feature of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network;

[0153] The segment evaluation feature and follow-up rhythm evaluation feature determination unit is used to decode the speech to be evaluated based on a pre-constructed limited text decoding network with timestamp, and determine the segment evaluation feature and follow-up rhythm evaluation feature of the speech to be evaluated.

[0154] As an implementable embodiment, the device further includes: a semantic network construction unit; the semantic network construction unit includes:

[0155] An acquisition subunit, used to acquire a demonstration sound as an evaluation standard, an original text corresponding to the demonstration sound, and a translation text in a target language corresponding to the original text;

[0156] The construction subunit is used to construct a semantic network based on the original text and the translated text.

[0157] As an implementable embodiment, the construction subunit is specifically used for:

[0158] Performing reverse translation on the translation text to determine a candidate original text;

[0159] Determine a text set based on the original text and the candidate original text;

[0160] A semantic network is constructed based on the text collection, wherein the semantic network includes a plurality of parallel semantic paths, and each semantic path corresponds to a text in the text collection.

[0161] As an implementable embodiment, the language organization assessment feature determination unit includes:

[0162] A calculation unit, used for calculating the content coverage of each semantic path of the recognized text information in the semantic network;

[0163] The language organization evaluation feature determination subunit is used to determine the language organization evaluation feature of the speech to be evaluated based on the content coverage of each semantic path in the semantic network of the recognized text information.

[0164] As an implementable embodiment, the device further includes: a time-stamped limited text decoding network construction unit; the time-stamped limited text decoding network construction unit is used to:

[0165] Based on the demonstration sound and the original text, a time-stamped limited text decoding network is established.

[0166] As an implementable embodiment, the time-stamped limited text decoding network construction unit includes:

[0167] A limited text decoding network establishing unit, used for establishing a limited text decoding network based on the original text;

[0168] A timestamp determination unit, used to determine a timestamp of the demonstration sound based on speech phonemes;

[0169] The incorporation unit is used to incorporate the speech phoneme-based timestamp of the demonstration sound into the limited text decoding network to construct a limited text decoding network with a timestamp.

[0170] As an implementable embodiment, the time-stamped limited text decoding network includes a start layer, an end layer, a speech phoneme unit layer, and an absorption layer between all the aforementioned layers;

[0171] Each of the phonetic phoneme unit layers is constructed according to the phonetic phonemes of each single character in the original text; each of the phonetic phoneme unit layers includes the timestamp of the phonetic phoneme in the demonstration sound;

[0172] There may be jump arcs between each of the speech phoneme unit layers and each absorption layer, each speech phoneme unit layer and the end layer.

[0173] As an implementable embodiment, the training method of the time-stamped limited text decoding network includes:

[0174] Acquiring a training voice corresponding to the demonstration sound;

[0175] For each training speech, determine a follow-up rhythm evaluation feature label and a segment evaluation feature label of each speech phoneme in the training speech, wherein the follow-up rhythm evaluation feature label is used to indicate whether the timestamp of the speech phoneme in the training speech is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound; and the segment evaluation feature label is used to indicate the score of each jump arc corresponding to the speech phoneme in the training speech;

[0176] The training speech and the phoneme-based timestamp of the training speech are used as training samples, and the reading rhythm evaluation feature label and the segment evaluation feature label are used as training labels to train and obtain the timestamp-containing limited text decoding network.

[0177] As an implementable embodiment, the evaluation result determination unit is used to:

[0178] The language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are input into a speech evaluation model, and the speech evaluation model outputs the evaluation results of the speech to be evaluated. The speech evaluation model is trained using the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the training speech as training samples, and using the evaluation results annotated by the training speech as sample labels.

[0179] As an implementable embodiment, the device further includes:

[0180] A feedback unit is used to generate follow-up rhythm feedback information based on the follow-up rhythm evaluation characteristics of the speech to be evaluated after determining the follow-up rhythm evaluation characteristics of the speech to be evaluated, and provide the follow-up rhythm feedback information to the speaker of the speech to be evaluated to prompt the speaker of the current follow-up rhythm.

[0181] Reference Figure 6 , Figure 6 The hardware structure diagram of the voice evaluation device provided in the embodiment of the present application is shown in FIG. Figure 6 ,The hardware structure of the speech evaluation device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0182] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0183] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0184] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0185] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0186] Get the voice to be evaluated;

[0187] Determining language organization assessment features, segment assessment features, and follow-up reading rhythm assessment features of the speech to be assessed;

[0188] An evaluation result of the speech to be evaluated is determined according to the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0189] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0190] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0191] Get the voice to be evaluated;

[0192] Determining language organization assessment features, segment assessment features, and follow-up reading rhythm assessment features of the speech to be assessed;

[0193] An evaluation result of the speech to be evaluated is determined according to the language organization evaluation features, the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

[0194] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0195] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0196] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0197] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech evaluation method, characterized in that: The method comprises: Get the voice to be evaluated; Determine the language organization assessment feature, the segment assessment feature and the follow-up rhythm assessment feature of the speech to be assessed; the language organization assessment feature can at least assess the language organization ability of the speaker of the speech to be assessed; the segment assessment feature and the follow-up rhythm assessment feature can at least assess one of the hearing, language organization ability, reaction ability, memory and reading ability of the speaker of the speech to be assessed; Determining an evaluation result of the speech to be evaluated according to the language organization evaluation features, the segment evaluation features, and the follow-up rhythm evaluation features of the speech to be evaluated; The step of determining the language organization assessment features, the segment assessment features, and the follow-up rhythm assessment features of the speech to be assessed includes: Recognize the speech to be evaluated and obtain recognized text information; Determining language organization evaluation features of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network; Based on a pre-constructed limited text decoding network with a timestamp, the speech to be evaluated is decoded to determine the segment evaluation features and the follow-up rhythm evaluation features of the speech to be evaluated.

2. The method according to claim 1, characterized in that The semantic network is constructed as follows: Obtaining a sample sound as an evaluation standard, an original text corresponding to the sample sound, and a translation text in a target language corresponding to the original text; A semantic network is constructed based on the original text and the translated text.

3. The method according to claim 2, characterized in that The step of constructing a semantic network based on the original text and the translated text includes: Performing reverse translation on the translation text to determine candidate original texts; Determine a text set based on the original text and the candidate original text; A semantic network is constructed based on the text collection, wherein the semantic network includes a plurality of parallel semantic paths, and each semantic path corresponds to a text in the text collection.

4. The method according to claim 3, characterized in that The step of determining the language organization assessment features of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network includes: Calculating the content coverage of each semantic path of the recognized text information in the semantic network; Based on the content coverage of each semantic path in the semantic network of the recognized text information, the language organization evaluation features of the speech to be evaluated are determined.

5. The method according to claim 2, characterized in that: The construction method of the time-stamped limited text decoding network is as follows: Based on the demonstration sound and the original text, a time-stamped limited text decoding network is established.

6. The method according to claim 5, characterized in that The step of establishing a time-stamped limited text decoding network based on the demonstration sound and the original text includes: Establishing a limited text decoding network based on the original text; Determining a speech phoneme-based timestamp of the demonstration sound; The speech phoneme-based timestamp of the demonstration sound is integrated into the limited text decoding network to construct a limited text decoding network with a timestamp.

7. The method according to claim 6, characterized in that The time-stamped limited text decoding network includes a starting layer, an ending layer, a speech phoneme unit layer, and an absorption layer between all the aforementioned layers; Each of the phonetic phoneme unit layers is constructed according to the phonetic phonemes of each single character in the original text; each of the phonetic phoneme unit layers includes the timestamp of the phonetic phoneme in the demonstration sound; There may be jump arcs between each of the speech phoneme unit layers and each absorption layer, each speech phoneme unit layer and the end layer.

8. The method according to claim 7, characterized in that The training method of the time-stamped limited text decoding network comprises: Acquiring a training voice corresponding to the demonstration sound; For each training speech, determine a follow-up rhythm evaluation feature label and a segment evaluation feature label of each speech phoneme in the training speech, wherein the follow-up rhythm evaluation feature label is used to indicate whether the timestamp of the speech phoneme in the training speech is consistent with the timestamp of the corresponding speech phoneme in the demonstration sound; and the segment evaluation feature label is used to indicate the score of each jump arc corresponding to the speech phoneme in the training speech; The training speech and the phoneme-based timestamp of the training speech are used as training samples, and the reading rhythm evaluation feature label and the segment evaluation feature label are used as training labels to train and obtain the limited text decoding network with timestamp.

9. The method according to claim 1, characterized in that: Determining the evaluation result of the speech to be evaluated according to the language organization evaluation feature, the segment evaluation feature and the follow-up rhythm evaluation feature of the speech to be evaluated includes: The language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the speech to be evaluated are input into a speech evaluation model, and the speech evaluation model outputs the evaluation results of the speech to be evaluated. The speech evaluation model is trained using the language organization evaluation features, segment evaluation features and follow-up rhythm evaluation features of the training speech as training samples, and using the evaluation results annotated by the training speech as sample labels.

10. The method according to claim 1, characterized in that After determining the evaluation characteristics of the follow-up rhythm of the speech to be evaluated, the method further includes: Based on the follow-up rhythm evaluation characteristics of the speech to be evaluated, follow-up rhythm feedback information is generated, and the follow-up rhythm feedback information is provided to the speaker of the speech to be evaluated to prompt the speaker of the current follow-up rhythm.

11. A speech evaluation device, characterized in that: The device comprises: An acquisition unit, used for acquiring the speech to be evaluated; A feature determination unit, used to determine the language organization assessment feature, the segment assessment feature and the follow-up rhythm assessment feature of the speech to be evaluated; the language organization assessment feature can at least assess the language organization ability of the speaker of the speech to be evaluated; the segment assessment feature and the follow-up rhythm assessment feature can at least assess one of the hearing, language organization ability, reaction ability, memory and reading ability of the speaker of the speech to be evaluated; An evaluation result determination unit, configured to determine an evaluation result of the speech to be evaluated according to the language organization evaluation features, the segment evaluation features, and the follow-up rhythm evaluation features of the speech to be evaluated; Wherein, the feature determination unit includes: A recognition unit, used for recognizing the speech to be evaluated and obtaining recognized text information; A language organization evaluation feature determination unit, used to determine the language organization evaluation feature of the speech to be evaluated based on the recognized text information and the pre-constructed semantic network; The segment evaluation feature and follow-up rhythm evaluation feature determination unit is used to decode the speech to be evaluated based on a pre-constructed limited text decoding network with timestamp, and determine the segment evaluation feature and follow-up rhythm evaluation feature of the speech to be evaluated.

12. A speech evaluation device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the speech evaluation method according to any one of claims 1 to 10.

13. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the speech evaluation method according to any one of claims 1 to 10 is implemented.

Images