A method, device, electronic device and storage medium for evaluating the quality of passage recitation

Abstract

The embodiment of the present application discloses a method, device, electronic device and storage medium for evaluating the quality of recitation of a passage. The technical solution provided by the embodiment of the present application is to input the recitation audio into a pre-constructed sentence parallel decoding network, determine the boundary information in the recitation audio based on the sentence parallel decoding network; divide the recitation audio into multiple audio segments based on the boundary information, and determine the correspondence between each audio segment and each sentence of the recitation text; input each audio segment into the corresponding fault-tolerant alignment network based on the correspondence, and decode to obtain the corresponding target text sequence; extract the reference text sequence corresponding to each sentence of the recitation text, compare the target text sequence with the reference text sequence one by one, and output the corresponding recitation quality evaluation result. By adopting the above-mentioned technical means, the boundary position corresponding to each sentence in the recitation audio can be accurately detected, the accuracy of the recitation audio detection can be improved, and then the evaluation accuracy of the passage recitation quality can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent education technology, and in particular to a method, device, electronic device and storage medium for evaluating the quality of passage recitation. Background Art

[0002] In language learning practice, passage memorization is an important and effective learning method. Through passage memorization, students can not only accumulate a large number of vocabulary, phrases and sentence patterns, but also become more proficient in oral expression and develop a richer sense of language and expression logic.

[0003] At present, in order to improve learning efficiency and recitation quality during the process of reciting a passage, oral reading error analysis technology is used to assist in analyzing and evaluating the quality of passage recitation. Oral reading error analysis technology is a subdivision of computer-assisted language learning. Oral reading error analysis technology requires efficient and accurate pointing out errors such as substitutions, pauses, omissions, insertions, and their error locations in learners' reading, helping learners correct pronunciation errors and optimize language learning effects.

[0004] However, when evaluating the quality of passage recitation, learners are prone to miss whole sentences and other unfluent and incomplete recitation when the text is long. Traditional oral reading error analysis technology is difficult to accurately detect the boundaries of each sentence in the recitation audio, and there is a lack of detection for missed sentences, which in turn affects the overall recitation quality evaluation results. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device and storage medium for evaluating the quality of recitation of a passage, which can accurately detect the boundary positions of corresponding sentences in the recitation audio and improve the evaluation accuracy of the quality of recitation of the passage.

[0006] In a first aspect, an embodiment of the present application provides a method for evaluating the quality of recitation of a passage, comprising:

[0007] Input the recitation audio into a pre-constructed sentence parallel decoding network, and determine boundary information in the recitation audio based on the sentence parallel decoding network, wherein the boundary information corresponds to the boundary between sentences in the recitation text;

[0008] Based on the boundary information, the recitation audio is divided into a plurality of audio segments, and a corresponding relationship between each audio segment and each sentence of the recitation text is determined;

[0009] Building an error-tolerant alignment network based on a reference text sequence of a corresponding sentence, inputting each of the audio segments into the corresponding error-tolerant alignment network based on the corresponding relationship, and decoding to obtain a corresponding target text sequence, wherein the target text sequence is used to identify an actual recitation detection result of the corresponding audio segment;

[0010] Extract the reference text sequence corresponding to each sentence of the recitation text, the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, compare the target text sequence with the reference text sequence one by one, and output the corresponding recitation quality evaluation result.

[0011] In a second aspect, an embodiment of the present application provides a device for evaluating the quality of recitation of a passage, comprising:

[0012] A boundary determination module inputs the recitation audio into a pre-constructed sentence parallel decoding network, and determines boundary information in the recitation audio based on the sentence parallel decoding network, wherein the boundary information corresponds to the boundary between sentences in the recitation text;

[0013] A segmentation module, used for segmenting the recitation audio into a plurality of audio segments based on the boundary information, and determining a correspondence between each audio segment and each sentence of the recitation text;

[0014] A decoding module, used to construct a fault-tolerant alignment network based on a reference text sequence of a corresponding sentence, input each of the audio segments into the corresponding fault-tolerant alignment network based on the corresponding relationship, and decode to obtain a corresponding target text sequence, wherein the target text sequence is used to identify an actual recitation detection result of the corresponding audio segment;

[0015] The output module is used to extract the reference text sequence corresponding to each sentence of the recitation text, the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, compare the target text sequence with the reference text sequence one by one, and output the corresponding recitation quality evaluation result.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0017] memory and one or more processors;

[0018] The memory is used to store one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for evaluating the quality of recitation of a passage as described in the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the method for assessing the quality of recitation of a passage as described in the first aspect.

[0021] The embodiment of the present application is to input the recitation audio into a pre-constructed sentence parallel decoding network, determine the boundary information in the recitation audio based on the sentence parallel decoding network, and the boundary information corresponds to the boundary between the sentences of the recitation text; divide the recitation audio into multiple audio segments based on the boundary information, and determine the corresponding relationship between each audio segment and each sentence of the recitation text; construct a fault-tolerant alignment network based on the reference text sequence of the corresponding sentence, input each audio segment into the corresponding fault-tolerant alignment network based on the corresponding relationship, decode to obtain the corresponding target text sequence, and the target text sequence is used to identify the actual recitation detection result of the corresponding audio segment; extract the reference text sequence of each sentence of the corresponding recitation text, and the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, compare the target text sequence with the reference text sequence one by one, and output the corresponding recitation quality evaluation result. Adopting the above-mentioned technical means, the boundary position of each sentence corresponding to the recitation audio can be accurately detected by the sentence parallel decoding network, avoid the omission detection situation of the whole sentence missing, improve the accuracy of the recitation audio detection, and then improve the evaluation accuracy of the chapter recitation quality.

[0022] In addition, the embodiment of the present application can further improve the evaluation accuracy of the quality of passage recitation by accurately detecting missed words, repeated pronunciations, confused pronunciations, etc. in each sentence through a fault-tolerant alignment network. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for evaluating the quality of reciting a passage provided in Example 1 of the present application;

[0024] Figure 2 This is a flow chart of boundary information determination based on a sentence parallel decoding network in Embodiment 1 of the present application;

[0025] Figure 3 is a flow chart of determining boundary information for the corresponding parallel structure in Embodiment 1 of the present application;

[0026] Figure 4 is a schematic diagram of a sentence parallel decoding network in Example 1 of the present application;

[0027] Figure 5 This is a schematic diagram of adding a garbage state path and a confusing sound path in the first embodiment of the present application;

[0028] Figure 6 is a flowchart of audio segment decoding in Embodiment 1 of the present application;

[0029] Figure 7 is a schematic diagram of the comparison between the target text sequence and the reference text sequence in Example 1 of the present application;

[0030] Figure 8 It is a structural schematic diagram of a device for evaluating the quality of recitation of a passage provided in the second embodiment of the present application;

[0031] Fig. 9 It is a structural schematic diagram of an electronic device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0033] A kind of chapter recitation quality evaluation method provided by the present application is intended to realize the accurate positioning of the boundary position of the recitation audio through the sentence parallel decoding network, so as to avoid the situation of missing the whole sentence and causing the omission detection, and improve the evaluation accuracy of the chapter recitation quality. And further realize the accurate detection of the audio segment corresponding to each sentence through the fault-tolerant alignment network, avoid the omission detection of the situation such as missing words in the sentence, and further improve the evaluation accuracy of the chapter recitation quality. Relative to the traditional oral reading error analysis technology, when it is reciting the chapter, although it can detect the situation of mispronunciation in the sentence, it cannot accurately detect the situation of missing the whole sentence, which easily affects the evaluation result of the whole recitation quality. In addition, the recitation quality detection corresponding to each sentence cannot be accurately detected to each word, and lacks accurate detection means for pronunciation replacement, missing reading, repetition, pause and misreading. Based on this, a kind of chapter recitation quality evaluation method of the present application embodiment is provided to solve the problem of missing detection of the whole sentence missing reading in the existing chapter recitation quality evaluation technology.

[0034] Embodiment 1:

[0035] Figure 1A flow chart of a method for evaluating the quality of reciting a passage provided in the first embodiment of the present application is provided. The method for evaluating the quality of reciting a passage provided in the present embodiment can be performed by a device for evaluating the quality of reciting a passage. The device for evaluating the quality of reciting a passage can be implemented by software and / or hardware. The device for evaluating the quality of reciting a passage can be composed of two or more physical entities or one physical entity. Generally speaking, the device for evaluating the quality of reciting a passage can be a terminal device such as a computer, a mobile phone, a tablet computer, or a learning machine.

[0036] The following description is made by taking the passage recitation quality evaluation device as an example of the main body of the passage recitation quality evaluation method. Figure 1 , the chapter recitation quality assessment method specifically includes:

[0037] S110, inputting the recitation audio into a pre-constructed sentence parallel decoding network, and determining boundary information in the recitation audio based on the sentence parallel decoding network, wherein the boundary information corresponds to the boundary between sentences in the recitation text.

[0038] Exemplarily, in the chapter recitation quality evaluation scenario, the reciter recites the chapter based on the recitation text, and the reciter's recitation audio is collected in real time during the recitation process. Then the recitation quality evaluation device based on the chapter recitation quality of the embodiment of the application evaluates the recitation quality of the recitation audio and outputs the corresponding recitation quality evaluation result. Based on the recitation quality evaluation result, the reciter's chapter recitation quality of the recitation text can be intuitively and accurately reflected.

[0039] When evaluating the quality of reciting a passage, the embodiment of the present application determines the boundary information in the reciting audio so as to segment the reciting audio, and then performs voice decoding comparison on the corresponding segmented audio segments one by one to determine the reciting quality one by one. It is understandable that each audio segment should correspond to the corresponding sentence in the reciting text, indicating the actual reciting pronunciation of the sentence.

[0040] Wherein, when determining the boundary information of reciting audio, use the sentence parallel decoding network based on WFST (Weighted Finite-State Transducers, weighted finite state transfer device) to detect and identify the boundary position of corresponding each sentence in the reciting audio.It is understandable that, because the reciting audio is corresponding to the reciting text.And in the reciting text, there must be a boundary between each sentence.Therefore, in reciting audio, there will naturally be the boundary information corresponding to the boundary between the sentences, by determining this boundary information, the reciting audio can be cut into the audio segment of corresponding each sentence, so that the reciting quality evaluation of the audio is recited based on the audio segment.

[0041] Specifically, refer to Figure 2, the boundary information determination process based on the sentence parallel decoding network in the embodiment of the present application includes:

[0042] S1101, determining the separators of each sentence in the recitation text based on the sentence parallel decoding network, and constructing a parallel structure between the sentences;

[0043] S1102, adding a silent path to the parallel structure, and adding a silent spinning path to the first and last states of the parallel structure, wherein the silent path is used to detect missed sentences, and the silent spinning path is used to detect silence, background sound, onomatopoeia and modal particles at the first and last states of a sentence;

[0044] S1103, decoding the recitation audio corresponding to the parallel structure, and determining boundary information in the recitation audio

[0045] When the embodiment of the present application uses the sentence parallel decoding network to detect the boundary information of the recitation audio, the recitation text is first split into multiple sentences by the sentence parallel decoding network to construct a parallel structure between sentences. Different from the traditional method of using a series structure to perform recognition detection between sentences, the embodiment of the present application constructs a parallel structure between sentences by corresponding to the recitation text, and can detect the audio segment corresponding to each sentence based on the parallel structure. The audio segment corresponding to each sentence is detected in a parallel manner, so the situation of missing the whole sentence of the sentence can be detected, and the alignment result of the audio segment and the corresponding sentence can be improved.

[0046] Among them, since the sentences in the long chapter recitation text all use periods, question marks or exclamation marks as sentence separators, the embodiment of the present application detects the above separators corresponding to the recitation text, determines the boundary positions of each sentence in the recitation text based on the separators, and then splits the recitation text into multiple sentences, and constructs a parallel structure corresponding to these sentences.

[0047] Further, the embodiment of the present application adds a silent spin path in the first and last states of the parallel structure, and adds a silent path in parallel with the sentence in the parallel structure. Wherein, the silent path is used to absorb the situation of skipping the whole sentence. The silent spin path can detect the silence, external noise, onomatopoeia and subconsciously emitted "oh", "um" and other modal particles corresponding to the beginning and end of a sentence in the recitation audio. On the other hand, it should be noted that the essence of the silent spin path is a recognition model for identifying the above-mentioned silence, background sound, onomatopoeia and modal particles, which is pre-trained by corresponding silence, background sound, onomatopoeia and modal particles to achieve the detection of silence, background sound, onomatopoeia and modal particles. The subsequent recitation audio based on the input model can identify the positions of the above-mentioned silence, background sound, onomatopoeia and modal particles by detection and identification. The recitation audio can be accurately detected and identified by the silent path and the silent spin path, so as to facilitate the accurate determination of the boundary information of the subsequent recitation audio.

[0048] Furthermore, based on the above-mentioned sentence parallel decoding network, the corresponding decoding structure can be used to decode the recitation audio to determine the boundary information in the recitation audio.

[0049] Reference Figure 3 The process of determining boundary information corresponding to the parallel structure in the embodiment of the present application includes:

[0050] S11031, determining a second posterior probability score of each audio frame in the recitation audio based on the DNN acoustic model;

[0051] S11032. Based on the second posterior probability score and corresponding to the parallel structure, use the Viterbi algorithm to search for the optimal path of each audio frame, and determine the audio frames corresponding to the boundaries of each sentence in the recitation text as the boundary information in the recitation audio.

[0052] Among them, in the embodiment of the present application, the recitation audio is decoded by a DNN (deep neural network) acoustic model in the sentence parallel decoding network. Based on the posterior probability score of each frame of speech given by the DNN acoustic model, the Viterbi algorithm is used to search for the optimal path in the parallel decoding network to obtain the boundary information at the sentence level.

[0053] When decoding the recitation audio, the posterior probability score (i.e., the second posterior probability score) of each audio frame in the recitation audio is determined in sequence. It can be understood that, corresponding to each audio frame, the DNN acoustic model is used to perform speech recognition decoding, and the posterior probability score of the audio frame corresponding to a certain phoneme can be obtained. Based on the posterior probability scores of each possible phoneme corresponding to the audio frame, the optimal path of the audio frame is searched by the Viterbi algorithm as the speech decoding result of the audio frame.

[0054] Furthermore, the optimal path is searched through the posterior probability scores of the continuous multi-frame audio frames to determine the speech decoding result. The phoneme sequence determined by the speech decoding of the continuous multi-frame audio frames is compared with the corresponding phoneme sequence of each sentence in the recitation text to determine that the continuous multi-frame audio frames correspond to the corresponding sentences. Then the audio frame corresponding to the phoneme at the sentence boundary should correspond to the boundary of the sentence, that is, the boundary information.

[0055] By sequentially voice decoding the recitation audio in accordance with the above method, corresponding to each sentence of the parallel structure, the voice decoding result of the recitation audio is compared with each sentence, and the boundary information in the recitation audio can be determined.

[0056] It should be noted that there are many ways to determine the posterior probability score of the audio frame based on the speech decoding of the recitation audio, and search the optimal path according to the Viterbi algorithm to determine the decoding result corresponding to the audio frame. The embodiment of the present application does not impose fixed restrictions on the specific implementation of the algorithm, and will not be elaborated here.

[0057] For example, refer to Figure 4 , providing a schematic diagram of a sentence parallel decoding network according to an embodiment of the present application. Figure 4 In the example, the two sentences in the recitation text are "what day is it?" and "I want to read books.", which are parallel structures in the sentence parallel decoding network. At the head and tail states of the parallel structure, a silent spin path is set correspondingly, and the silent spin path is used to detect the silent label "SIL" of the head and tail states. A silent path is set in the parallel structure, and the silent path is used to detect the silent label "SIL" of the whole sentence that is missed. When performing boundary information detection, the recitation audio is correspondingly identified based on the parallel structure between sentences provided by the sentence parallel decoding network. In the parallel decoding network, based on the posterior probability score of each frame of speech given by the DNN acoustic model, the Viterbi algorithm is used to search for the optimal path in the parallel decoding network to obtain the boundary information at the sentence level.

[0058] S120: dividing the recitation audio into a plurality of audio segments based on the boundary information, and determining a correspondence between each audio segment and each sentence of the recitation text.

[0059] Further, after determining to recite each boundary information of audio, the embodiment of the present application recites audio corresponding to each boundary information segmentation, obtains corresponding multiple audio segments.It is understandable that each audio segment corresponds to each sentence in the text of recitation.Carry out speech recognition and error detection based on determined audio segment, can realize the chapter recitation evaluation of the application's sentence level, determine to recite the recitation quality of each sentence in the text.

[0060] It can be understood that each boundary information corresponds to a sentence boundary in the recitation text. Then, according to the timestamp of the audio frame corresponding to the boundary information, the sentence boundary position in the recitation text corresponding to the boundary information can be determined one by one in sequence, and then the sentence in the recitation text corresponding to the corresponding audio segment can be determined, and the correspondence between each audio segment and each sentence of the recitation text can be determined.

[0061] S130, constructing a fault-tolerant alignment network based on a reference text sequence of the corresponding sentence, inputting each of the audio segments into the corresponding fault-tolerant alignment network based on the corresponding relationship, decoding to obtain a corresponding target text sequence, and the target text sequence is used to identify an actual recitation detection result of the corresponding audio segment.

[0062] Based on the above-mentioned alignment of the corresponding audio segments and sentences, the embodiment of the present application decodes the corresponding audio segments based on the voice of each sentence in the recitation text, and determines the voice decoding result of each audio segment, that is, the target text sequence. Each sentence in the recitation text is a reference text sequence, and the recitation quality of the sentence can be determined by comparing the reference text sequence with the identified target text sequence.

[0063] It should be noted that since the target text sequence needs to be compared with the reference text sequence later, the sequence labels need to be compared one by one during the comparison. Therefore, it is necessary to forcibly align the corresponding audio segment with the fault-tolerant network constructed with the reference text, and the final speech recognition result of the corresponding audio segment (i.e., the target text sequence) should measure the degree of difference between the sequence labels contained in it and the labels of the reference text sequence, so as to facilitate the determination of the positions of replacement, omission, repetition, pause and / or misreading on the target text sequence, and achieve a more accurate evaluation of the quality of recitation of the chapter.

[0064] Specifically, the embodiment of the present application constructs a fault-tolerant alignment network based on a reference text sequence, uses the fault-tolerant alignment network to force alignment of each audio segment with the corresponding sentence, and obtains the corresponding target text sequence through decoding of the fault-tolerant alignment network. Wherein, when constructing a fault-tolerant alignment network based on a reference text sequence of a corresponding sentence, an optional garbage state path and a confused sound path are added to the corresponding position in the reference text sequence to construct a corresponding fault-tolerant alignment network, wherein the garbage state path contains sequence labels of background sound, silence, and onomatopoeia, and the confused sound path contains corresponding confused word labels.

[0065] Since the text is known, the forced alignment technology can be used to decode the audio segment. Forced alignment aligns the speech to each phoneme of the sentence corresponding to the recitation text, and gives the start and end time of each phoneme. Among them, the error-tolerant alignment network is used to force the alignment of the audio segment. In the general alignment network, the speech decoding result of the standard pronunciation of the audio segment is generally described by the reference text sequence of the sentence corresponding to the recitation text, but in the actual chapter recitation evaluation, the reader often adds some unpredictable wrong pronunciations while reading aloud, or replaces a word with a wrong pronunciation because he does not know it, and there is also the case of missing one or more words. In order to match these pronunciation phenomena, a more complex network structure description is needed.

[0066] The present application embodiment uses a fault-tolerant alignment network to describe these pronunciation phenomena. Figure 5, there is an optional junk state path in the fault-tolerant alignment network, and the junk state path is obtained by training with background sound, silence, and onomatopoeia audio. Among them, the junk state can reach other states, and other states can also reach the junk state. In this way, possible paths for normal reading of words, skipped words, repeated reading of words, and pauses corresponding to the pronunciation can be constructed. In addition, the fault-tolerant alignment network also includes an optional confused sound path, which records the confused sound words of the corresponding words in each sentence in the recitation text. The confused sound words of each word can be set manually, or they can be obtained by counting the degree of confusion between the recognition results and the original words through speech decoding technology. For example, the confused sound words of the word "park" are "bark", "par", and "puck"; the confused sound words of the word "watermelon" are the word string "watermain land", and "water_main_land" is recorded as a word as the confused sound word of "watermelon".

[0067] The garbage state path and the confused sound path can be used to detect background sound, silence, onomatopoeia and confused sound. When the fault-tolerant alignment network is used for speech decoding, an optional decoding path is provided in the speech decoding process, thereby improving the accuracy of speech decoding and optimizing the speech decoding results.

[0068] Further, see Figure 6 , the process of decoding the audio segment in the embodiment of the present application includes:

[0069] S1301, inputting the audio segment into a pre-trained acoustic model, and calculating a first posterior probability score corresponding to the audio segment;

[0070] S1302: Based on the first posterior probability score of the audio segment, search for an optimal path in the fault-tolerant alignment network according to a Viterbi algorithm as a target text sequence for the audio segment.

[0071] It is understandable that based on the junk state path and confused sound path of the fault-tolerant alignment network, it is possible to determine the junk state and confused sound recognition in the speech decoding process of the corresponding audio segment, and determine the optional path corresponding to the junk state and confused sound. Among them, when determining the optional path of the junk state and confused sound, the reference text sequence of the corresponding sentence is used as the basic text sequence, and the junk state path and the confused sound path are added or replaced at the corresponding position in the reference text sequence. The junk state path contains sequence labels of background sound, silence and onomatopoeia, and the confused sound path contains corresponding confused word labels. It is understandable that through the optional junk state path and confused sound path, the sequence labels of pronunciation conditions such as skipped words, repeated words, pauses and confused words can be set based on the reference text sequence.

[0072] Specifically, when performing speech decoding on an audio frame, the posterior probability score (i.e., the first posterior probability score) of the sequence label corresponding to each phoneme in the audio segment is calculated through a pre-trained acoustic model, and then the Viterbi algorithm is used to search for the optimal path in a fault-tolerant alignment network with optional junk state paths and confused sound paths to determine the sequence labels that ultimately correspond to each phoneme in the audio segment. These sequence labels are the target text sequences corresponding to the audio segment.

[0073] It should be noted that for each independent audio segment, it is input into a fault-tolerant alignment network with a confused sound recognition model and a junk state recognition model. The fault-tolerant alignment network can identify the pronunciation of confused sounds in the audio segment through the confused sound recognition model. Similarly, the junk state recognition model can identify the pronunciation of skipped words, repeated words, and pauses in the audio segment, so as to determine the optional junk state path and confused sound path for the audio segment. Prior to this, the confused sound recognition model and the junk state recognition model are trained corresponding to the pronunciation of confused sounds and skipped words, repeated words, and pauses, respectively, so as to realize the recognition of the pronunciation of confused pronunciations, skipped words, repeated words, and pauses in the audio segment.

[0074] Further, after obtaining the posterior probability score (i.e., the first posterior probability score) of each audio segment through speech decoding, similarly, the optimal path is searched using the Viterbi algorithm to obtain the target text sequence of the audio segment. It can be understood that the posterior probability score of the sequence label corresponding to each phoneme in the phoneme sequence corresponding to each audio segment is obtained through speech decoding, and the final target text sequence can be determined through Viterbi path search based on the posterior probability score of the sequence label corresponding to each phoneme.

[0075] S140, extracting a reference text sequence corresponding to each sentence of the recitation text, wherein the reference text sequence is used to identify a standard recitation detection result of the corresponding sentence, comparing the target text sequence with the reference text sequence one by one, and outputting a corresponding recitation quality evaluation result.

[0076] Finally, according to the target text sequence obtained by identifying each audio segment, the target text sequence is compared with the reference text sequence of the sentence corresponding to the audio segment, and the recitation quality evaluation can be performed through the difference between the two text sequences. It can be understood that the target text sequence identifies the actual recitation test result of the corresponding audio segment, which records the sequence labels of each phoneme in the actual pronunciation process of the corresponding sentence. The reference text sequence identifies the standard recitation test result of the corresponding sentence, which records the sequence labels of each phoneme in the standard pronunciation process of the corresponding sentence. Then, by comparing the sequence labels, the difference between the two text sequences can be determined, and the corresponding recitation quality evaluation result can be obtained.

[0077] Specifically, the embodiment of the present application compares the target text sequence with the reference text sequence one by one and outputs corresponding recitation quality evaluation results, including:

[0078] Align the target text sequence with each sequence tag of the reference text sequence, detect corresponding replacement, omission, repetition, pause and / or misreading positions, and output corresponding recitation quality evaluation results.

[0079] By aligning the corresponding sequence labels of two text sequences, and then comparing each group of sequence labels one by one, it is detected whether there are replacements, omissions, repetitions, pauses or misreadings, and then the positions of replacements, omissions, repetitions, pauses and / or misreadings in the target text sequence are determined. By marking these positions, these error situations are output as recitation quality assessment results, thereby completing the chapter recitation quality assessment process of the embodiment of the present application.

[0080] For example, refer to Figure 7 , the audio segment corresponding to the sentence "i like the park" is recited, and the target text sequence "ii, likes bark" of the audio segment is identified through the above steps S110-S130, and the target text sequence is aligned with the reference text sequence of the sentence and then compared, such as Figure 7 As shown, determine the correct, replacement, omission, repetition, pause and / or misread position existing on each sequence tag one by one, and then after the pronunciation of each sequence tag is marked, output and display as the recitation quality evaluation result. At this moment, evaluate this recitation quality evaluation result based on the output display, can know the recitation quality situation of the current corresponding audio segment, and determine the corresponding pronunciation error position. Similarly, all determine the recitation quality evaluation result and output display with reference to the above-mentioned method for each audio segment, with this, the evaluator can determine the chapter recitation quality of the current entire recitation text.

[0081] In one embodiment, the apparatus for evaluating the quality of recitation of a passage further comprises: comparing the target text sequence with the reference text sequence one by one and outputting corresponding evaluation results of the quality of recitation;

[0082] Align the sequence labels of the target text sequence and the reference text sequence, calculate the edit distance between the target text sequence and the reference text sequence, and output a corresponding recitation quality evaluation result according to the edit distance.

[0083] Specifically, Figure 7As shown, by aligning the target text sequence with the sentence reference text sequence and comparing them, the correct, replaced, omitted, repeated, paused and / or misread positions on each sequence tag are determined one by one, and the corresponding position of replacement, omitted, repeated, paused and / or misread is recorded as 1 unit of edit distance. In this way, the edit distance between the target text sequence and the sentence reference text sequence can be obtained. It can be understood that the edit distance describes the degree of difference between the target text sequence and the sentence reference text sequence, that is, the degree of difference between the current recitation pronunciation of the sentence and the standard pronunciation. The larger the edit distance, the greater the degree of difference between the current recitation pronunciation of the sentence and the standard pronunciation. When the edit distance is 0, it means that the current recitation pronunciation of the sentence is the same as the standard pronunciation, and its recitation quality is optimal.

[0084] Further, referring to the above method, for each sentence of the recitation text, by counting the edit distance between the target text sequence of the audio segment and the reference text sequence of the corresponding sentence, the edit distance between the entire recitation audio and the standard pronunciation of the recitation text can be obtained, and the edit distance is used as the chapter recitation quality evaluation result of the current recitation text. Based on the edit distance, the recitation quality of the current reciter can be intuitively determined to complete the chapter recitation evaluation result.

[0085] Optionally, based on the statistical edit distance between the entire recitation audio and the standard pronunciation of the recitation text, it is normalized to a score of 1-100. It is understandable that the larger the edit distance, the lower the chapter recitation quality score, and the worse the quality of the marked chapter recitation. On the contrary, the smaller the edit distance, the higher the chapter recitation quality score, and the better the quality of the marked chapter recitation. This score is output to the evaluator, so that the chapter recitation quality score of the current recitation audio can be more intuitively reflected.

[0086] In the above, by inputting the recitation audio into the pre-constructed sentence parallel decoding network, the boundary information in the recitation audio is determined based on the sentence parallel decoding network, and the boundary information corresponds to the boundary between the sentences of the recitation text; the recitation audio is divided into multiple audio segments based on the boundary information, and the corresponding relationship between each audio segment and each sentence of the recitation text is determined; a fault-tolerant alignment network is constructed based on the reference text sequence of the corresponding sentence, and each audio segment is input into the corresponding fault-tolerant alignment network based on the corresponding relationship, and the corresponding target text sequence is decoded to obtain the corresponding target text sequence, and the target text sequence is used to identify the actual recitation detection result of the corresponding audio segment; the reference text sequence of each sentence of the corresponding recitation text is extracted, and the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, and the target text sequence is compared with the reference text sequence one by one, and the corresponding recitation quality evaluation result is output. Using the above technical means, the boundary position of each sentence in the recitation audio can be accurately detected through the sentence parallel decoding network, avoiding the omission detection of the whole sentence, improving the accuracy of the recitation audio detection, and then improving the evaluation accuracy of the chapter recitation quality.

[0087] In addition, the embodiment of the present application can further improve the evaluation accuracy of the quality of passage recitation by accurately detecting missed words, repeated pronunciations, confused pronunciations, etc. in each sentence through a fault-tolerant alignment network.

[0088] Embodiment 2:

[0089] Based on the above embodiments, Figure 8 This is a schematic diagram of the structure of a device for evaluating the quality of recitation of a passage provided in Example 2 of the present application. Figure 8 The apparatus for evaluating the quality of reciting a passage provided in this embodiment specifically includes: a boundary determination module 21, a segmentation module 22, a decoding module 23 and an output module 24.

[0090] The boundary determination module 21 inputs the recitation audio into a pre-constructed sentence parallel decoding network, and determines boundary information in the recitation audio based on the sentence parallel decoding network, wherein the boundary information corresponds to the boundary between sentences in the recitation text;

[0091] The segmentation module 22 is used to segment the recitation audio into multiple audio segments based on the boundary information, and determine the corresponding relationship between each audio segment and each sentence of the recitation text;

[0092] The decoding module 23 is used to construct a fault-tolerant alignment network based on the reference text sequence of the corresponding sentence, input each of the audio segments into the corresponding fault-tolerant alignment network based on the corresponding relationship, and decode to obtain a corresponding target text sequence, wherein the target text sequence is used to identify the actual recitation detection result of the corresponding audio segment;

[0093] The output module 24 is used to extract the reference text sequence corresponding to each sentence of the recitation text, the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, compare the target text sequence with the reference text sequence one by one, and output the corresponding recitation quality evaluation result.

[0094] Specifically, the decoding module 23 includes:

[0095] A construction unit, configured to add an optional junk state path and a confused sound path at a corresponding position in the reference text sequence to construct a corresponding fault-tolerant alignment network, wherein the junk state path includes sequence labels of background sound, silence, and onomatopoeia, and the confused sound path includes corresponding confused word labels;

[0096] A recognition unit, configured to input the audio segment into a pre-trained DNN acoustic model and calculate a first posterior probability score corresponding to the audio segment;

[0097] A search unit is used to search for an optimal path in the fault-tolerant alignment network based on the first posterior probability score of the audio segment according to the Viterbi algorithm as a target text sequence of the audio segment.

[0098] Specifically, the boundary determination module 21 includes:

[0099] A parallel unit, used to determine the separators of each sentence in the recitation text based on the sentence parallel decoding network, and construct a parallel structure between the sentences;

[0100] An adding unit, used for adding a silent path to the parallel structure, and adding a silent spin path in the first and last states of the parallel structure, wherein the silent path is used for detecting missed sentences, and the silent spin path is used for detecting silence, background sound, onomatopoeia and modal particles in the first and last states of a sentence;

[0101] A determination unit is used to decode the recitation audio corresponding to the parallel structure and determine boundary information in the recitation audio.

[0102] Specifically, the determination unit includes:

[0103] Determine a second posterior probability score for each audio frame in the recitation audio based on the DNN acoustic model;

[0104] According to the second posterior probability score, and corresponding to the parallel structure, the optimal path of each audio frame is searched using the Viterbi algorithm to determine the audio frames corresponding to the boundaries of each sentence in the recitation text as the boundary information in the recitation audio.

[0105] Specifically, the output module 24 includes:

[0106] Align the target text sequence with each sequence tag of the reference text sequence, detect corresponding replacement, omission, repetition, pause and / or misreading positions, and output corresponding recitation quality evaluation results.

[0107] Optionally, the output module 24 further includes:

[0108] Align the sequence labels of the target text sequence and the reference text sequence, calculate the edit distance between the target text sequence and the reference text sequence, and output a corresponding recitation quality evaluation result according to the edit distance.

[0109] In the above, by inputting the recitation audio into the pre-constructed sentence parallel decoding network, the boundary information in the recitation audio is determined based on the sentence parallel decoding network, and the boundary information corresponds to the boundary between the sentences of the recitation text; the recitation audio is divided into multiple audio segments based on the boundary information, and the corresponding relationship between each audio segment and each sentence of the recitation text is determined; a fault-tolerant alignment network is constructed based on the reference text sequence of the corresponding sentence, and each audio segment is input into the corresponding fault-tolerant alignment network based on the corresponding relationship, and the corresponding target text sequence is decoded to obtain the corresponding target text sequence, and the target text sequence is used to identify the actual recitation detection result of the corresponding audio segment; the reference text sequence of each sentence of the corresponding recitation text is extracted, and the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, and the target text sequence is compared with the reference text sequence one by one, and the corresponding recitation quality evaluation result is output. Using the above technical means, the boundary position of each sentence in the recitation audio can be accurately detected through the sentence parallel decoding network, avoiding the omission detection of the whole sentence, improving the accuracy of the recitation audio detection, and then improving the evaluation accuracy of the chapter recitation quality.

[0110] In addition, the embodiment of the present application can further improve the evaluation accuracy of the quality of passage recitation by accurately detecting missed words, repeated pronunciations, confused pronunciations, etc. in each sentence through a fault-tolerant alignment network.

[0111] The apparatus for evaluating the quality of reciting a passage provided in the second embodiment of the present application can be used to execute the method for evaluating the quality of reciting a passage provided in the first embodiment above, and has corresponding functions and beneficial effects.

[0112] Embodiment three:

[0113] Embodiment 3 of the present application provides an electronic device, referring to Fig. 9 The electronic device includes: a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The number of processors 31 in the electronic device can be one or more, and the number of memories 32 in the electronic device can be one or more. The processor 31, memory 32, communication module 33, input device 34, and output device 35 of the electronic device can be connected via a bus or other means.

[0114] As a computer-readable storage medium, the memory can be used to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the chapter recitation quality evaluation method described in any embodiment of the present application (for example, the boundary determination module, alignment module, recognition module and output module in the chapter recitation quality evaluation device). The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0115] The communication module is used for data transmission.

[0116] The processor executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory, thereby realizing the above-mentioned chapter recitation quality assessment method.

[0117] The input device can be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device can include display devices such as display screens.

[0118] The electronic device provided above can be used to execute the chapter recitation quality assessment method provided in the above embodiment 1, and has corresponding functions and beneficial effects.

[0119] Embodiment 4:

[0120] The embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to execute a method for evaluating the quality of recitation of a passage when executed by a computer processor. The method comprises: inputting the recitation audio into a pre-constructed sentence parallel decoding network, determining boundary information in the recitation audio based on the sentence parallel decoding network, wherein the boundary information corresponds to the boundary between sentences in the recitation text; dividing the recitation audio into multiple audio segments based on the boundary information, and determining the correspondence between each audio segment and each sentence in the recitation text; constructing a fault-tolerant alignment network based on a reference text sequence of the corresponding sentence, inputting each audio segment into the corresponding fault-tolerant alignment network based on the correspondence, decoding to obtain a corresponding target text sequence, wherein the target text sequence is used to identify an actual recitation detection result of the corresponding audio segment; extracting a reference text sequence corresponding to each sentence of the recitation text, wherein the reference text sequence is used to identify a standard recitation detection result of the corresponding sentence, comparing the target text sequence with the reference text sequence one by one, and outputting a corresponding recitation quality evaluation result.

[0121] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0122] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application is not limited to the chapter recitation quality assessment method described above, and the computer executable instructions can also execute the relevant operations in the chapter recitation quality assessment method provided in any embodiment of the present application.

[0123] The passage recitation quality assessment device, storage medium and electronic device provided in the above embodiments can execute the passage recitation quality assessment method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the passage recitation quality assessment method provided in any embodiment of the present application.

[0124] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for evaluating the quality of reciting a passage, characterized in that: include: Input the recitation audio into a pre-constructed sentence parallel decoding network, and determine boundary information in the recitation audio based on the sentence parallel decoding network, wherein the boundary information corresponds to the boundary between sentences in the recitation text; Based on the boundary information, the recitation audio is divided into a plurality of audio segments, and a corresponding relationship between each audio segment and each sentence of the recitation text is determined; Building a fault-tolerant alignment network based on a reference text sequence of the corresponding sentence, inputting each of the audio segments into the corresponding fault-tolerant alignment network based on the corresponding relationship, and decoding to obtain a corresponding target text sequence, wherein the target text sequence is used to identify an actual recitation detection result of the corresponding audio segment; Extract the reference text sequence corresponding to each sentence of the recitation text, the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, compare the target text sequence with the reference text sequence one by one, and output the corresponding recitation quality evaluation result.

2. The method for evaluating the quality of recitation of a passage according to claim 1, characterized in that: Construct an error-tolerant alignment network based on the reference text sequence of the corresponding sentence, including: An optional junk state path and a confused sound path are added at corresponding positions in the reference text sequence to construct a corresponding fault-tolerant alignment network, wherein the junk state path includes sequence labels of background sound, silence, and onomatopoeia, and the confused sound path includes corresponding confused word labels.

3. The method for evaluating the quality of recitation of a passage according to claim 1, characterized in that: Inputting each of the audio segments into the corresponding error-tolerant alignment network based on the corresponding relationship and decoding to obtain a corresponding target text sequence includes: Inputting the audio segment into a pre-trained DNN acoustic model, and calculating a first posterior probability score corresponding to the audio segment; Based on the first posterior probability score of the audio segment, an optimal path is searched in the error-tolerant alignment network according to the Viterbi algorithm as a target text sequence of the audio segment.

4. The method for evaluating the quality of recitation of a passage according to claim 1, characterized in that: Inputting the recitation audio into a pre-built sentence parallel decoding network, and determining boundary information in the recitation audio based on the sentence parallel decoding network, including: Determine the separators of each sentence in the recitation text based on the sentence parallel decoding network, and construct a parallel structure between the sentences; Add a silent path in the parallel structure, and add a silent spin path in the first and last states of the parallel structure, wherein the silent path is used to detect missed sentences, and the silent spin path is used to detect silence, background sound, onomatopoeia and modal particles in the first and last states of sentences; The recitation audio is decoded corresponding to the parallel structure to determine boundary information in the recitation audio.

5. The method for evaluating the quality of recitation of a passage according to claim 4, characterized in that: Decoding the recitation audio corresponding to the parallel structure and determining boundary information in the recitation audio includes: Determine a second posterior probability score for each audio frame in the recitation audio based on the DNN acoustic model; According to the second posterior probability score, and corresponding to the parallel structure, the optimal path of each audio frame is searched using the Viterbi algorithm to determine the audio frames corresponding to the boundaries of each sentence of the recitation text as the boundary information in the recitation audio.

6. The method for evaluating the quality of recitation of a passage according to claim 1, characterized in that: Compare the target text sequence with the reference text sequence one by one, and output corresponding recitation quality evaluation results, including: Align the target text sequence with each sequence tag of the reference text sequence, detect corresponding replacement, omission, repetition, pause and / or misreading positions, and output corresponding recitation quality evaluation results.

7. The method for evaluating the quality of recitation of a passage according to claim 1, characterized in that: Comparing the target text sequence with the reference text sequence one by one and outputting corresponding recitation quality evaluation results, further comprising: Align the sequence labels of the target text sequence and the reference text sequence, calculate the edit distance between the target text sequence and the reference text sequence, and output a corresponding recitation quality evaluation result according to the edit distance.

8. A device for evaluating the quality of recitation of a passage, characterized in that: include: A boundary determination module inputs the recitation audio into a pre-constructed sentence parallel decoding network, and determines boundary information in the recitation audio based on the sentence parallel decoding network, wherein the boundary information corresponds to the boundary between sentences in the recitation text; A segmentation module, used for segmenting the recitation audio into a plurality of audio segments based on the boundary information, and determining a correspondence between each audio segment and each sentence of the recitation text; A decoding module, used to construct a fault-tolerant alignment network based on a reference text sequence of a corresponding sentence, input each of the audio segments into the corresponding fault-tolerant alignment network based on the corresponding relationship, and decode to obtain a corresponding target text sequence, wherein the target text sequence is used to identify an actual recitation detection result of the corresponding audio segment; The output module is used to extract the reference text sequence corresponding to each sentence of the recitation text, the reference text sequence is used to identify the standard recitation detection result of the corresponding sentence, compare the target text sequence with the reference text sequence one by one, and output the corresponding recitation quality evaluation result.

9. An electronic device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for evaluating the quality of recitation of a passage as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the passage recitation quality assessment method as described in any one of claims 1-7.

Images