Spoken language evaluation method, spoken language evaluation device, electronic equipment and storage medium

Abstract

The invention discloses a spoken language evaluation method, a spoken language evaluation device, electronic equipment and a storage medium. The method comprises the steps of obtaining to-be-recognized voice data; encoding the to-be-recognized voice data by using an encoder to obtain audio encoding data; constructing a guidance database based on the guidance data; wherein the guidance data comprises at least one of a general dictionary and test paper resources; based on the guidance database, decoding the audio coding data by using a decoder to obtain a target voice text corresponding to the to-be-recognized voice data; wherein the encoder is obtained through the configuration of the guidance database. Through the method, the oral evaluation accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence application, and in particular to a spoken language evaluation method, a spoken language evaluation device, an electronic device and a storage medium. BACKGROUND

[0002] With the development of education technology and the increasing emphasis on language ability in society, oral language ability has become one of the important components of evaluating students' comprehensive quality. Based on this, oral language evaluation has gradually become an indispensable link. Oral language evaluation is mainly divided into two types: fixed text reading type (such as reading short texts, sentences and words, etc.) and non-fixed text free type (such as scene answering, story retelling, picture speaking, etc.). In the oral language evaluation of the non-fixed text type, the existing oral language evaluation technology has limited recognition ability for new words, proper nouns and domain terminology, which leads to a decrease in accuracy during oral language evaluation. Secondly, the existing oral language evaluation technology has low accuracy in recognizing similar or similar sounding words. Therefore, based on the above background, how to improve the accuracy of oral language evaluation is a problem to be solved. SUMMARY

[0003] The technical solution mainly solved by the present application is to provide a spoken language evaluation method, a spoken language evaluation device, an electronic device and a storage medium, which can improve the accuracy of oral language evaluation.

[0004] To solve the above technical problems, one technical solution adopted by the present application is to provide a spoken language evaluation method, which comprises: obtaining to-be-recognized voice data; using an encoder to encode the to-be-recognized voice data to obtain audio encoding data; constructing a guidance database based on guidance data; wherein the guidance data comprises at least one of a general dictionary and test paper resources; using a decoder to decode the audio encoding data based on the guidance database to obtain a target voice text corresponding to the to-be-recognized voice data; wherein the encoder is obtained by configuring the guidance database.

[0005] To solve the above technical problems, another technical solution adopted by the present application is to provide a spoken language evaluation device, which comprises: a voice data acquisition module configured to obtain to-be-recognized voice data; an audio encoding data acquisition module configured to use an encoder to encode the to-be-recognized voice data to obtain audio encoding data; a guidance database construction module configured to construct a guidance database based on guidance data; wherein the guidance data comprises at least one of a general dictionary and test paper resources; and a target voice text acquisition module configured to use a decoder to decode the audio encoding data based on the guidance database to obtain a target voice text corresponding to the to-be-recognized voice data; wherein the encoder is obtained by configuring the guidance database.

[0006] To solve the above technical problems, another technical solution adopted by the present application is to provide an electronic device, comprising a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to realize the oral test method.

[0007] To solve the above technical problems, another technical solution adopted by the present application is to provide a computer readable storage medium, the computer readable storage medium is used to store program instructions, the program instructions can be executed to realize the oral test method.

[0008] The above scheme, by obtaining the to-be-recognized voice data, using the encoder to encode the to-be-recognized voice data, obtaining the audio encoding data. Based on the guidance database is constructed, wherein the guidance data includes at least one of the general dictionary and the test paper resource. Based on the guidance database, the audio encoding data is decoded by using the decoder to obtain the target voice text corresponding to the to-be-recognized voice data, wherein the encoder is obtained by the guidance database configuration. The present application utilizes the guidance data to construct the guidance database, and the guidance data includes at least one of the general dictionary and the test paper resource. On the one hand, the new words, proper nouns, domain terms and the like existing in the guidance data can be obtained, so as to utilize the guidance database constructed based on the guidance data to assist the encoder to obtain more accurate target voice text. On the other hand, the guidance database constructed based on the guidance data can assist the encoder to obtain more domain context conforming text, so as to reduce the semantic deviation problem, and further obtain more accurate target voice text. Therefore, through the above scheme, the accuracy of oral test can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a flowchart of an embodiment of the oral test method provided by the present application;

[0010] Figure 2 is a flowchart of a specific embodiment of the guidance database construction method provided by the present application;

[0011] Figure 3 is a flowchart of a specific embodiment of the subword sequence acquisition method provided by the present application;

[0012] Figure 4 is a flowchart of a specific embodiment of the target voice text acquisition method provided by the present application;

[0013] Figure 5 is a flowchart of a specific embodiment of the target voice text acquisition method provided by the present application;

[0014] Figure 6 is a framework diagram of an embodiment of the oral test device provided by the present application;

[0015] Figure 7 is a frame diagram of an embodiment of an electronic device of the present application.

[0016] Figure 8 is a frame diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0017] For the purpose of making the purpose, technical solutions and effects of the present application more clear and explicit, the present application is further described in detail below with reference to the drawings and embodiments.

[0018] It should be noted that the term "several" herein means at least one, and the terms "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. The term "and / or" is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in this paper generally represents an "or" relationship between the front and rear associated objects. In addition, the term "at least one" herein means any one of a plurality of combinations of at least two of any one or more, for example, including at least one of A, B, and C, which can mean selecting any one or more elements from the set consisting of A, B, and C.

[0019] Please refer to Figure 1 , Figure 1 is a flow diagram of an embodiment of the spoken language evaluation method provided by the present application. It should be noted that if there is substantially the same result, the present embodiment is not limited to the order of the flow shown in Figure 1 . As shown in Figure 1 , the present embodiment includes:

[0020] Step S11: obtaining to-be-recognized voice data.

[0021] In an embodiment, the to-be-recognized voice data is a voice file in a standard waveform audio file format. Specifically, the to-be-recognized voice data is a voice file in a standard Wav format.

[0022] Among them, the present application supports multi-modal voice input. For example, the audio data of the user can be collected in real time through a physical audio interface (such as a microphone array), and the collected audio data can be converted into a voice file in a standard Wav format. For another example, a voice file in a standard Wav format transmitted by an external system can be directly received. Of course, after receiving the audio data of the user transmitted by the external system, the received audio data can be converted into a voice file in a standard Wav format. The specific way of obtaining to-be-recognized voice data is not limited here, and can be set according to the actual situation.

[0023] Step S12: encoding the to-be-recognized speech data by using an encoder to obtain audio encoded data.

[0024] In an embodiment, before the to-be-recognized speech data is encoded by using the encoder to obtain the audio encoded data, the to-be-recognized speech data can also be preprocessed, and the preprocessed to-be-recognized speech data is preprocessed to obtain corresponding filter bank features.

[0025] In a specific embodiment, the filter bank features can be encoded by using a configured encoder to obtain the audio encoded data.

[0026] For example, the to-be-recognized speech data (such as a speech file in a standard Wav format) is preprocessed, and the preprocessed to-be-recognized speech data is analog-to-digital converted to convert analog speech features into digital speech features. The digital speech features are encoded by using an encoder to obtain audio encoded data. Specifically, the to-be-recognized speech data is preprocessed to obtain corresponding filter bank features (such as FilterBank features). The filter bank features are encoded by using an encoder to obtain audio encoded data. The audio encoded data can refer to context-dependent vector representation. That is, it can be understood that the encoder can perform time-domain context modeling based on the filter bank features to output context-dependent vector representation, which integrates semantic information of adjacent frames and the like.

[0027] Step S13: constructing a guidance database based on guidance data.

[0028] The guidance data includes at least one of a general dictionary and a test paper resource.

[0029] In an embodiment, the general dictionary refers to a collated dictionary. Considering memory occupation, efficiency, and the like, most commonly used corpora (such as words and phrases) can be selected, which can be selected according to actual conditions, and no limitation is made herein. The test paper resource refers to corpora (such as words and phrases) in a test paper used in a spoken language test. The corpora are obtained from subwords and various combinations thereof.

[0030] In yet another embodiment, constructing the guidance database based on the guidance data can include traversing the guidance data to obtain subwords and various combinations thereof of the guidance data. The subwords and various combinations thereof of the guidance data are text segmented to generate a subword sequence. The guidance database is constructed by using the subword sequence.

[0031] In a specific embodiment, the step of constructing the guidance database by using the subword sequence can include: inserting the subword sequence into a pre-built initial prefix tree. An end marker is added at the end of each subword sequence, and the prefix tree after the addition is determined as the guidance database.

[0032] The pre-built initial prefix tree refers to a prefix tree defining node structure and overall data structure, used to determine how each node stores character information and a pointer to a child node (a node in the next layer). The subword sequence can be created by inserting the prefix tree nodes created layer by layer, thereby forming a complete sequence. In addition, adding an end marker at the end of each subword in each subword sequence can ensure that the prefix tree can correctly represent the corpus and components in the guidance data. For example, adding a pre-sequence and / or end of each subword in each subword sequence can ensure that the prefix tree can correctly represent the words and components of the words in the guidance data. The end marker is not limited, and can be set according to actual conditions.

[0033] In yet another specific embodiment, the step of performing text segmentation on the subwords and various combinations of the guidance data to generate the subword sequence can include: querying the subword sequence dictionary for the subwords and various combinations of the guidance data. In response to querying the subword sequence dictionary for at least one of the subwords and various combinations of the guidance data, determining the queried subword or combination as the subword sequence of the guidance data. In response to not querying the subword sequence dictionary for the subwords and various combinations, dividing the subwords and various combinations of the guidance data into at least one character unit, merging the character units according to a character merging rule, and determining the merged character unit as the subword sequence of the guidance data. The subword sequence dictionary refers to a dictionary generated by end-to-end training on a large-scale, diversified text corpus using a subword segmentation algorithm (such as a BPE (Byte Pair Encoding) algorithm, which can effectively decompose a word into smaller units (such as a character unit)).

[0034] Please refer to Figure 2 , Figure 2 is a flowchart of a specific embodiment of the guidance database construction method provided by the present application. Step S21: define the node structure and overall data structure of the initial prefix tree. Step S22: determine whether the subword sequence dictionary can query the subwords and various combinations corresponding to the guidance data. In the case where the determination result of step S22 is yes, step S23 is performed: determining the subword or combination queried in the subword sequence dictionary as the subword sequence of the guidance data. In the case where the determination result of step S22 is no, step S24 is performed: dividing the subwords and various combinations of the guidance data into at least one character unit, merging the character units according to a character merging rule, and determining the merged character unit as the subword sequence of the guidance data. After step S22 or step S24 is performed, step S25 is performed: inserting the subword sequence into the pre-built initial prefix tree, and adding an end marker at the end of each subword sequence. Step S26: determining the prefix tree with the inserted subword sequence and end marker as the guidance database.

[0035] The constructed guide database can be used for efficient subword sequence retrieval. That is, the constructed prefix tree can be used for efficient subword sequence retrieval.

[0036] Specifically, in response to not querying the subword and various combinations thereof in the subword sequence dictionary, the subword and various combinations thereof corresponding to the guide data are divided into at least one character unit, the character units are merged according to the character merging rule, and the merged character string unit is determined as the subword sequence of the guide data. The step can include: decomposing the subword and various combinations thereof corresponding to the guide data according to the word composition order to obtain at least one character unit. Adjacent character units are sequentially merged to obtain at least one corresponding character string. Based on a word decomposition dictionary, the dictionary serial numbers corresponding to each character string are sequentially determined, wherein the word decomposition dictionary is obtained by training a subword segmentation algorithm. The character units contained in the character strings that meet the character merging rule are merged, and the merged character string unit is determined as the subword sequence of the guide data. The word decomposition dictionary can represent the mapping relationship between a word group and a subword sequence, for example, word1 corresponds to subword sequence sub1 sub2 sub3, wherein word1 represents a word group, and sub1 sub2 sub3 is the subword sequence corresponding to the word group.

[0037] For example, the subword corresponding to the guidance data and various combinations thereof is yiming. yiming is decomposed according to the order of the words to obtain six character units of "y", "i", "m", "i", "n", and "g". Adjacent character units are merged in turn to obtain five strings of "yi", "im", "mi", "in", and "ng". Based on the word decomposition dictionary, the dictionary serial numbers corresponding to "yi", "im", "mi", "in", and "ng" are determined in turn, and the character units contained in the strings that meet the character merging rule are merged. For example, in the case that "yi" meets the character merging rule, yi is merged, and the character units are updated to "yi", "m", "i", "n", and "g". The merging operation of the character units is cycled until it cannot be merged, and the final merged string is determined as the subword sequence of the guidance data. For example, the second merging is "yi", "mi", "n", and "g", the third merging is "yi", "min", and "g", the fourth merging is "yi", "ming", and the fifth merging is not possible. "yi" and "ming" are taken as the subword sequence of yiming. Among them, whether the character units can be merged or not can be determined by searching whether the string can be queried in the word decomposition dictionary. That is, in the case that the string cannot be queried in the word decomposition dictionary, it can be confirmed that the string cannot be merged any more, and the current obtained string is determined as the subword sequence of the guidance data. For example, in the fifth merging, "yiming" should be obtained, but the string "yiming" cannot be queried in the word decomposition dictionary, so it is considered that "yi" and "ming" cannot be merged, and "yi" and "ming" are taken as the subword sequence of yiming. Of course, merging adjacent character units in turn can also refer to obtaining three strings of "yi", "mi", and "ng", and merging "yi" according to the character merging rule. Here, no limitation is made. It should be noted that non-adjacent strings cannot be directly merged by skipping the intermediate string.

[0038] Specifically, the step of merging the character units contained in the string that meets the character merging rule and determining the merged string unit as the subword sequence of the guidance data can include: sorting the dictionary serial numbers of each string according to the serial number size. The character units in the string at the front / last of the sorting are merged, wherein the string at the front / last of the sorting is the string with the smallest dictionary serial number. The dictionary serial numbers of each string can be sorted according to the serial number size from small to large, that is, the character units in the string at the front of the sorting are merged. Or it can be sorted according to the serial number size from large to small, that is, the character units in the string at the last of the sorting are merged. Here, no limitation is made.

[0039] For example, the subword corresponding to the guidance data and various combinations thereof is yiming. yiming is decomposed according to the order of word composition to obtain six character units of “y”, “i”, “m”, “i”, “n”, and “g”. Adjacent character units are merged in turn to obtain three strings of “yi”, “mi”, and “ng”. Based on the word decomposition dictionary, the dictionary serial numbers corresponding to “yi”, “mi”, and “ng” are determined in turn. For example, the dictionary serial numbers corresponding to “yi”, “mi”, and “ng” are 1, 2, and 3 respectively, and the dictionary serial numbers of the strings are sorted according to the serial number size. The dictionary serial number of “yi” is the smallest, so “y” and “i” in “yi” are merged, and the string is updated to “yi”, “m”, “i”, “n”, and “g”. The merging operation of the character units is cycled until it cannot be merged, and the finally merged string is determined as the subword sequence of the guidance data. For specific merging operation of the character units, reference can be made to the related description in step S13, which will not be repeated here.

[0040] Specifically, before the step of merging adjacent character units in turn to obtain at least one corresponding string, it can also include judging whether the total number of character units exceeds a preset threshold. If the total number of character units does not exceed the preset threshold, the whole character unit is taken as a subword, and the subword is taken as the subword sequence corresponding to the subword and various combinations thereof. The preset threshold can be set according to actual conditions, which is not limited here. For example, the preset threshold can be set to one, that is, whether the subword corresponding to the guidance data and various combinations thereof is a single letter is judged.

[0041] For example, the subword corresponding to the guidance data and various combinations thereof is decomposed according to the order of word composition to obtain at least one character unit. It is judged whether the decomposed character unit exceeds one. In the case where the decomposed character unit does not exceed one, the whole character unit is taken as a subword, and the subword is taken as the subword sequence corresponding to the subword and various combinations thereof (i.e., the whole single letter is taken as the corresponding subword sequence). In the case where the decomposed character unit exceeds one, adjacent character units are merged in turn to obtain at least one corresponding string. Based on the word decomposition dictionary, the dictionary serial numbers corresponding to the strings are determined in turn. The character units contained in the string that meets the character merging rule are merged, and the merged string unit is determined as the subword sequence of the guidance data.

[0042] Specifically, before determining the dictionary index of each string based on the word decomposition dictionary, the process may include: determining whether each string is found in the word decomposition dictionary. If the string is not found in the word decomposition dictionary, each string is treated as a corresponding subword, and each subword is treated as a subword sequence. That is, by determining whether each string is found in the word decomposition dictionary, it is determined whether to merge the strings again. For details, please refer to the relevant description in step S13, which will not be elaborated here.

[0043] Specifically, a subword sequence is composed of at least one subword in the order in which words are formed.

[0044] Before generating the subwords and their various combinations corresponding to the guidance data, subword markers can be added to the beginning and / or end of the final merged string. That is, the final merged string is treated as each subword, all subwords together are treated as a subword sequence, and subword markers are added to the beginning and / or end of each subword in the subword sequence. The output is a subword sequence containing the subword markers. Subword markers can be @@, or #; there are no restrictions here.

[0045] Please see Figure 3 , Figure 3 This is a flowchart illustrating a specific embodiment of the subword sequence acquisition method provided in this application. The subword sequence acquisition method is a method for acquiring subwords and their various combinations that are not found in the subword sequence dictionary. Step S31: Decompose the subwords and their various combinations corresponding to the guidance data according to the word composition order to obtain at least one character unit. Step S32: Determine whether the total number of character units obtained from the decomposition exceeds a preset threshold. If the determination result of step S32 is yes, proceed to step S33: Merge adjacent character units sequentially to obtain at least one corresponding string. Step S34: Determine whether each string is found in the word decomposition dictionary. If the determination result of step S34 is yes, proceed to step S35: Create a data type name, where the data type name is used to record the string index and dictionary sequence number. Step S36: Determine whether the data stored in the data type name is empty. If the determination result of step S36 is no, proceed to step S37: Sort the dictionary sequence numbers of each string according to the sequence number. Step S38: Merge the character units in the first / last sorted strings, where the first / last sorted string is the string with the smallest dictionary index. If the result of step S38, step S32, step S35, or step S36 is yes, proceed to step S39: Add a sub-word marker to the beginning and / or end of each string. Step S310: Output the sub-words corresponding to the guidance data and the sub-word sequences corresponding to their various combinations.

[0046] In another specific implementation, before constructing the guidance database using the sub-word sequence, the method further includes: removing duplicate content from the sub-word sequence. That is, constructing the guidance database using non-repeating sub-word sequences.

[0047] Step S14: Based on the guidance database, use the decoder to decode the audio encoded data to obtain the target speech text corresponding to the speech data to be recognized.

[0048] The encoder is obtained by guiding the database configuration.

[0049] In one embodiment, decoding audio encoded data using a decoder based on a guidance database may include: In the case of initial decoding, the decoder uses subwords and their various combinations present in the guidance database to decode the audio encoded data, obtaining a first subword probability distribution. All subword probabilities in the first subword probability distribution are then sorted in descending order of probability, and the decoding paths obtained through this descending order are traversed. Each subword probability corresponds one-to-one with a decoding path. In the case of subsequent decoding, based on the subwords of the decoding path corresponding to the previous decoding, the decoder again uses subwords and their various combinations present in the guidance database to decode the audio encoded data, obtaining a second subword probability distribution. All subword probabilities in the second subword probability distribution are added to the corresponding subword probabilities in the previous decoding path (a first preset number) to obtain a third subword probability distribution. All subword probabilities in the third subword probability distribution are then sorted in descending order of probability, and the decoding paths obtained through this descending order are traversed. Each subword probability corresponds one-to-one with a decoding path. The first preset number can be set according to actual conditions and is not limited here.

[0050] The first and second sub-word probability distributions are both probability distributions of sub-words and their various combinations existing in the database. For example, if the database contains word1, word2, and word3, then the first and second sub-word probability distributions refer to the probabilities of word1, word2, and word3. That is, the first sub-word probability distribution corresponds to three decoding paths (the node paths corresponding to word1, word2, and word3). The third sub-word probability distribution is obtained by adding all sub-word probabilities in the second sub-word probability distribution to the corresponding sub-word probabilities in the previous decoding path with a first preset number of paths. For example, if the first preset number is 2, and the remaining decoding paths are word1 and word2 (the number of traversed decoding paths is 2), then the second sub-word probability distribution refers to the probabilities of word1, word2, and word3. Adding the probabilities corresponding to the remaining decoding paths word1 and word2 to the second sub-word probability distribution yields the third sub-word probability distribution. The third sub-word probability distribution corresponds to six decoding paths (the node paths corresponding to word1 and word1, word1 and word2, word1 and word3, word2 and word1, word2 and word2, and word2 and word3). Here, word1 and word1 correspond to nodes at different levels. For example, the first word1 is the path for the first decoding, located at the first level node. The second word1 is the path for the second decoding, located at the second level node. The node paths containing the first and second word1 are defined as the decoding paths corresponding to word1 and word1. The other decoding paths corresponding to the third sub-word probability distribution also correspond to nodes at different levels than the paths mentioned above, and will not be elaborated upon here.

[0051] In one specific implementation, traversing the decoded paths obtained by descending sorting may include: in response to the number of traversed decoded paths not exceeding a first preset number, determining whether the number of ending decoded paths exceeds a second preset number. If the number of ending decoded paths exceeds the second preset number, selecting a target path from the first preset number, and obtaining the sub-word sequence corresponding to the target path, wherein the target path includes the ending decoded path. If the number of ending decoded paths does not exceed the second preset number, backtracking to the last sub-word sequence in the currently traversed decoded paths, and obtaining the sub-word sequence corresponding to the audio encoding number based on the last sub-word sequence. Here, the ending decoded path refers to a path that ends with an end marker. The second preset number can be set according to actual needs and is not limited here.

[0052] For example, if the number of currently traversed decoding paths does not exceed N, determine if the number of ending decoding paths exceeds K. If the number of ending decoding paths exceeds K, select N target paths according to the word selection rules and obtain the corresponding word sequences for each target path. Specifically, you can first select K ending decoding paths as target paths, and then select NK decoding paths as target paths based on word probability. This will yield the word sequences corresponding to the N selected target paths. If the number of ending decoding paths does not exceed K, backtrack to the last word sequence in each current decoding path. Based on the last word sequence, obtain the word sequence corresponding to the audio encoding number. Here, you can select the NK decoding paths with higher probabilities as target paths.

[0053] The process of obtaining the sub-sequence corresponding to the audio encoding number based on the last sub-sequence can include: traversing the last sub-sequence in the current decoding path; deleting the decoding path containing the last sub-sequence if it is not found in the guidance database; and determining whether the last sub-sequence is a complete word if it is found in the guidance database and has an end marker. If it is a complete word, retaining the decoding path containing the last sub-sequence as the end decoding path and incrementing the path count of the end decoding path. If it is not a complete word, deleting the current decoding path and traversing the next decoding path obtained by descending sorting. "Not finding the last sub-sequence in the guidance database" can refer to the last sub-word within the last sub-sequence not found in the guidance database. The last sub-word in the sub-sequence can be identified by the sub-word marker in the sub-sequence. The current decoding path refers to the currently traversed decoding path, and the next decoding path refers to the next decoding path after the currently traversed decoding path. Deleted decoding paths are not counted in the path count of the traversed decoding paths.

[0054] In one specific implementation, in response to the failure to find the last sub-word sequence in the guidance database or the absence of an end marker for the sub-word sequence, the decoding path containing the last sub-word sequence is deleted, and the next decoding path obtained by descending sorting is traversed.

[0055] The decoder can use path search methods (such as BeamSearch) to decode the audio encoded data and obtain the target speech text corresponding to the speech data to be recognized. Before decoding, the nodes, probabilities, indices, previous node numbers, and current node numbers of the path search can be defined. The indices are used to determine the corresponding nodes during backtracking, and the probabilities are used to record the calculated path probabilities, such as the path probabilities corresponding to a path Node1->Node2->...->NodeN (the probabilities of the sub-words corresponding to each decoded path are summed).

[0056] Please see Figure 4 , Figure 4This is a flowchart illustrating a specific embodiment of the target speech-text acquisition method provided in this application. Step S41: Define path search nodes, including probability, index, previous node number, and current node number. Step S42: The decoder uses subwords and their various combinations existing in the guidance database to decode the audio encoded data for the first time to obtain the corresponding subword probability distribution. Step S43: Create N path search nodes. Step S44: Determine whether the decoder is performing the first decoding. If the determination result of step S44 is yes, proceed to step S45: Initialize the path probability, index, previous node number, and current node number in the N path search nodes to 0. If the determination result of step S44 is no, proceed to step S46: Calculate the probability of the current decoding path using the N previously retained decoding paths and the current decoding path. After executing step S45 or step S46, proceed to step S47: Sort the probabilities corresponding to the decoding paths in descending order. Step S48: Traverse the decoded paths after descending order. Step S49: Determine if the number of traversed decoding paths has reached the first preset number. If the result of step S49 is yes, proceed to step S42. If the result of step S49 is no, proceed to step S410: Determine if the number of ending decoding paths exceeds the second preset number. If the result of step S49 is no, proceed to step S411: Backtrack the last sub-word sequence in the current decoding path and determine if the last sub-word is found in the guidance database. If the result of step S411 is yes, proceed to step S412: Determine if the node in the current decoding path is the ending node. If the result of step S412 is no, proceed to step S413: Record the current node sequence number and other information, and increment the number of traversed decoding paths by one. If step S413 is executed or if the result of step S411 is no, re-execute step S48. If the judgment result in step S412 is yes, proceed to step S414: determine whether the last sub-word sequence in the current decoding path is a complete word. If the judgment result in step S414 is yes, proceed to step S415: record the decoding count and the current node number, and increment the path count of the end decoding path. If the judgment result in step S414 is no, or after proceeding to step S415, re-execute step S48. If the judgment result in step S410 is yes, proceed to step S416: output N target paths.

[0057] This can involve recording information such as the probability, index, previous node number, and current node number of the decoded path that has been traversed into the path search node.

[0058] In another embodiment, the method for obtaining the target speech text corresponding to the speech data to be identified may include: obtaining at least one target path and a corresponding target path probability obtained by decoding audio encoded data, wherein the target path probability is obtained based on the subwords contained in the subword sequence of the target path. A weighted probability of the audio encoded data is obtained based on the target path probability. The weighted probabilities are sorted according to their scores, and the subword sequence of the target path corresponding to the weighted probability that meets the path selection criteria is taken as the target speech text. For example, a first preset number of target paths are obtained by decoding the audio encoded data, and the sum of the probabilities of the subwords contained in the subword sequence of the first preset number of target paths is taken as the target path probability.

[0059] In one specific implementation, obtaining the activation score of a sub-word sequence based on the sub-word sequence probability includes: obtaining the activation probability of the audio encoded data using a unigram language model; and weighting the target path probability using the activation probability to obtain the corresponding weighted probability, wherein the unigram language model is trained based on guided data.

[0060] To better understand the excitation probability, the principle of path calculation excitation can be expressed by the following formula (1):

[0061]

[0062] Where AM represents the acoustic model, PM represents the language model, and P(w i |x) represents the subword w i The conditional probability of the spoken text being x. P(w) i ) indicates the subword w i The prior probability is N, where N is the number of subwords in the speech text.

[0063] Assume logP(w) i |x) are all the same, that is, equation (2):

[0064]

[0065] Now, we stimulate a certain word, that is, increase the probability of the unigram language model. Specifically, the prior probability of the word is increased from P0 to P1, then the above equation becomes equation (3):

[0066]

[0067] After simplification and calculation, we obtain equation (4):

[0068]

[0069] For ease of understanding, in equation (4) This can be understood as the probability of the target path. The value added to the original target path probability is the activation probability. From the above comparative analysis, it can be seen that using a unigram language model to activate a single word can directly increase the activation probability based on the original target path probability.

[0070] For example, iterate through each sub-word and its various combinations (such as words) in the target path, determining whether the sub-word and its various combinations are in the test paper resources. If not, iterate through the next sub-word and its various combinations. If they are in the test paper, obtain the incentive probability corresponding to the current sub-word and its various combinations, weight it, and add it to the target path probability of the target path to obtain the weighted probability corresponding to each target path. For example, the weighted probability corresponding to each target path is the sum of the target path probability and the weighted probability.

[0071] Please see Figure 5 , Figure 5 This is a flowchart illustrating a specific embodiment of the target speech text acquisition method provided in this application. Step S51: Obtain a target path from a first preset number of target paths. Step S52: Determine whether a target path has been successfully obtained. If the determination result of step S52 is yes, proceed to step S53: Traverse the subwords in the target path. Step S54: Determine whether the currently traversed subword exists in the test paper resource. If the determination result of step S54 is no, repeat step S53. If the determination result of step S54 is yes, proceed to step S55: Use a unigram language probability model to obtain the activation probability of the currently traversed subword. Step S56: Use the activation probability of the currently traversed subword to weight the target path probability of the currently selected target path to obtain a weighted probability. If the determination result of step S52 is no, proceed to step S57: Sort the weighted probabilities of the first preset number of target paths. Step S58: Select the subword sequence corresponding to the target path with the highest weighted probability as the target speech text.

[0072] The above scheme can improve the accuracy of oral assessment by constructing a guidance database using at least one of a general dictionary and test paper resources. Specifically, during the decoding process, the guidance database can reduce the occurrence of new words in the recognition results, thereby improving the accuracy of oral assessment. Furthermore, test paper resources can assist the encoder in obtaining text that better fits the domain context, thus reducing semantic bias and resulting in more accurate target speech text. For example, if the corpus in the test paper resources is: "four littleducks wen t out to play", and assuming there is a target path "four littleducks went out to play" and another similar path "for littleducks went out to play", the guidance database constructed using the test paper resources will select the target path that best matches the answer as the corresponding target speech text.

[0073] See Figure 6 , Figure 6 This is a schematic diagram of the framework of an embodiment of the oral language assessment device provided in this application. The oral language assessment device 600 includes a speech data acquisition module 610, an audio encoded data acquisition module 620, a guidance database construction module 630, and a target speech text acquisition module 640. The speech data acquisition module 610 is used to acquire speech data to be recognized. The audio encoded data acquisition module 620 is used to encode the speech data to be recognized using an encoder to obtain audio encoded data. The guidance database construction module 630 is used to construct a guidance database based on guidance data, wherein the guidance data includes at least one of a general dictionary and test paper resources. The target speech text acquisition module 640 is used to decode the audio encoded data using a decoder based on the guidance database to obtain the target speech text corresponding to the speech data to be recognized. The encoder is configured through the guidance database.

[0074] In some embodiments, when the guidance database construction module 630 specifically executes the construction of a guidance database based on guidance data, the process includes: traversing the guidance data to obtain subwords and their various combinations; performing text segmentation on the subwords and their various combinations to generate subword sequences; and constructing the guidance database using the subword sequences.

[0075] In some embodiments, when the guidance database construction module 630 specifically performs the step of constructing a guidance database using sub-word sequences, it includes: inserting the sub-word sequences into a pre-built initial prefix tree; adding an end marker to the end of each sub-word sequence; and determining the prefix tree after the addition is completed as the guidance database.

[0076] In some embodiments, when the guidance database construction module 630 specifically performs the step of text segmentation of the subwords and their various combinations in the guidance data to generate a subword sequence, the method includes: querying the subwords and their various combinations corresponding to the guidance data in a subword sequence dictionary. In response to finding at least one subword and its various combinations corresponding to the guidance data in the subword sequence dictionary, determining the queried subword or combination as the subword sequence of the guidance data. In response to not finding any subword or its various combinations in the subword sequence dictionary, dividing the subwords and their various combinations corresponding to the guidance data into at least one character unit, merging the character units according to character merging rules, and determining the merged string unit as the subword sequence of the guidance data.

[0077] In some embodiments, when the guidance database construction module 630 specifically performs the step of responding to the absence of a subword and its various combinations in the subword sequence dictionary, dividing the subwords and their various combinations corresponding to the guidance data into at least one character unit, merging the character units according to character merging rules, and determining the merged string unit as the subword sequence of the guidance data, the method includes: decomposing the subwords and their various combinations corresponding to the guidance data according to the word composition order to obtain at least one character unit; merging adjacent character units sequentially to obtain at least one corresponding string; determining the dictionary index corresponding to each string sequentially based on the word decomposition dictionary, wherein the word decomposition dictionary is trained by a subword segmentation algorithm; merging the character units contained in the strings that conform to the character merging rules, and determining the merged string unit as the subword sequence of the guidance data.

[0078] In some embodiments, when the database construction module 630 specifically executes the step of merging character units contained in strings that conform to character merging rules and determining the merged string units as the subword sequence of the guidance data, the method includes: sorting the dictionary indices of each string according to their indices; merging the character units in the first / last sorted string, wherein the first / last sorted string is the string with the smallest dictionary indices.

[0079] In some embodiments, before guiding the database construction module 630 to specifically execute the step of sequentially merging adjacent character units to obtain at least one corresponding string, the method further includes: determining whether the total number of character units exceeds a preset threshold. If the total number of character units does not exceed the preset threshold, the entire character unit is treated as a subword, and the subword is treated as a subword sequence corresponding to its various combinations.

[0080] In some embodiments, before the database construction module 630 specifically executes the process of determining the dictionary number corresponding to each string based on the word decomposition dictionary, the method further includes: determining whether each string is found in the word decomposition dictionary. If each string is not found in the word decomposition dictionary, each string is treated as a corresponding subword, and each subword is treated as a subword sequence.

[0081] In some embodiments, a subword sequence is composed of at least one subword in word order.

[0082] In some embodiments, before the guidance database construction module 630 specifically executes the construction of the guidance database using the sub-word sequence, it further includes: deleting duplicate content in the sub-word sequence.

[0083] In some embodiments, the target speech-text acquisition module 640 specifically performs decoding of audio encoded data using a decoder based on a guidance database. This includes: in the case of initial decoding, the decoder decodes the audio encoded data using sub-words and their various combinations present in the guidance database to obtain a first sub-word probability distribution. All sub-word probabilities in the first sub-word probability distribution are then sorted in descending order of probability, and the decoding paths obtained through this descending order are traversed. Each sub-word probability corresponds one-to-one with a decoding path. In cases of subsequent decoding, based on the sub-words from the previous decoding path, the decoder again decodes the audio encoded data using sub-words and their various combinations present in the guidance database to obtain a second sub-word probability distribution. All sub-word probabilities in the second sub-word probability distribution are added to the corresponding sub-word probabilities in the previous decoding path (a first preset number) to obtain a third sub-word probability distribution. All sub-word probabilities in the third sub-word probability distribution are then sorted in descending order of probability, and the decoding paths obtained through this descending order are traversed. Each sub-word probability corresponds one-to-one with a decoding path.

[0084] In some embodiments, the target speech-text acquisition module 640, when specifically traversing the decoding paths obtained by descending order, includes: in response to the fact that the number of traversed decoding paths does not exceed a first preset number, determining whether the number of ending decoding paths exceeds a second preset number. If the number of ending decoding paths exceeds the second preset number, selecting a target path from the first preset number, and obtaining the sub-word sequence corresponding to the target path, wherein the target path includes the ending decoding path. If the number of ending decoding paths does not exceed the second preset number, backtracking the last sub-word sequence in the currently traversed decoding path, and obtaining the sub-word sequence corresponding to the audio encoding number based on the last sub-word sequence.

[0085] In some embodiments, the target speech-text acquisition module 640, when specifically performing the process of obtaining the sub-word sequence corresponding to the audio encoding number based on the last sub-word sequence, includes: traversing the last sub-word sequence in the current decoding path. In response to not finding the last sub-word sequence in the guidance database, deleting the decoding path containing the last sub-word sequence. In response to finding the last sub-word sequence in the guidance database and the sub-word sequence having an end marker, determining whether the last sub-word sequence is a complete word. If it is a complete word, retaining the decoding path containing the last sub-word sequence as the end decoding path, and incrementing the path count of the end decoding path by one. If it is not a complete word, deleting the current decoding path, and traversing the next decoding path obtained by descending order.

[0086] In some embodiments, the target speech text acquisition module 640 specifically performs the acquisition of target speech text corresponding to the speech data to be recognized, including: acquiring at least one target path and corresponding target path probabilities obtained by decoding audio encoded data, wherein the target path probabilities are obtained based on the subwords contained in the subword sequence of the target path; obtaining the weighted probabilities of the audio encoded data based on the target path probabilities; sorting the weighted probabilities according to their scores; and taking the subword sequence of the target path corresponding to the weighted probabilities that meet the path selection conditions as the target speech text.

[0087] In some embodiments, the target speech-text acquisition module 640 specifically performs the process of obtaining a weighted probability of audio encoded data based on the target path probability, including: obtaining the activation probability of the audio encoded data using a unigram language model; and weighting the target path probability using the activation probability to obtain the corresponding weighted probability, wherein the unigram language model is trained based on guided data.

[0088] In some embodiments, before the audio encoded data acquisition module 620 specifically executes the encoding of the speech data to be recognized using an encoder to obtain audio encoded data, the method further includes: preprocessing the speech data to be recognized, and preprocessing the preprocessed speech data to be recognized to obtain corresponding filter group features. Encoding the speech data to be recognized using an encoder to obtain audio encoded data includes: encoding the filter group features using a configured encoder to obtain audio encoded data.

[0089] In some embodiments, the speech data to be identified is a speech file in a standard waveform audio file format.

[0090] Please see Figure 7 , Figure 7This is a schematic diagram of a framework of an embodiment of the electronic device of this application. The electronic device 70 includes a memory 71 and a processor 72 coupled to each other. The processor 72 is used to execute program instructions stored in the memory 71 to implement the steps in any of the above-described oral assessment method embodiments. In a specific implementation scenario, the electronic device 70 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 70 may also include mobile devices such as laptops and tablets, which are not limited here.

[0091] Specifically, processor 72 controls itself and memory 71 to implement the steps in any of the above-described oral assessment method embodiments. Processor 72 can also be referred to as a CPU (Central Processing Unit). Processor 72 may be an integrated circuit chip with signal processing capabilities. Processor 72 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 72 can be implemented using integrated circuit chips.

[0092] Please see Figure 8 , Figure 8 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 80 stores program instructions 81 that can be executed by a processor. The program instructions 81 are used to implement the steps in any of the above-described oral assessment method embodiments.

[0093] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0094] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0096] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0097] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method of spoken language assessment, characterized by, The oral test method comprises: obtaining to-be-recognized voice data; encoding the to-be-recognized voice data using an encoder to obtain audio encoded data; constructing a guide database based on guide data, wherein the guide data comprises at least one of a general dictionary and a test paper resource; decoding the audio encoded data using a decoder based on the guide database to obtain target voice text corresponding to the to-be-recognized voice data, wherein the encoder is obtained by configuring the guide database.

2. The method of claim 1, wherein, The step of constructing the guide database based on the guide data comprises: traversing the guide data to obtain subwords and various combinations thereof; text tokenizing the subwords and various combinations thereof of the guide data to generate a subword sequence; constructing the guide database using the subword sequence.

3. The method of claim 2, wherein, The step of constructing the guide database using the subword sequence comprises: inserting the subword sequence into a pre-built initial prefix tree; adding an end marker at the end of each subword sequence, and determining the prefix tree after the addition as the guide database.

4. The method of claim 2, wherein, The step of text tokenizing the subwords and various combinations thereof of the guide data to generate a subword sequence comprises: querying the subwords and various combinations thereof of the guide data in a subword sequence dictionary; in response to querying at least one of the subwords and various combinations thereof of the guide data in the subword sequence dictionary, determining the queried subword or combination as the subword sequence of the guide data; in response to not querying the subwords and various combinations thereof in the subword sequence dictionary, dividing the subwords and various combinations thereof of the guide data into at least one character unit, merging the character units according to a character merging rule, and determining the merged string unit as the subword sequence of the guide data.

5. The method of claim 4, wherein, The step of dividing the subwords and various combinations thereof of the guide data into at least one character unit, merging the character units according to a character merging rule, and determining the merged string unit as the subword sequence of the guide data comprises: decomposing the subwords and various combinations thereof of the guide data according to a word composition order to obtain the at least one character unit; merging adjacent character units in sequence to obtain at least one corresponding string; determining the dictionary serial numbers corresponding to each string in sequence based on a word decomposition dictionary, wherein the word decomposition dictionary is obtained by training a subword segmentation algorithm; merging the character units contained in the string that meet the character merging rule, and determining the merged string unit as the subword sequence of the guide data.

6. The method of claim 5, wherein, The step of merging the character units contained in the string that meet the character merging rule, and determining the merged string unit as the subword sequence of the guide data comprises: sorting the dictionary serial numbers of each string according to serial number size; merge the character units in the string ranked first / last, wherein the string ranked first / last is the string with the smallest dictionary number.

7. The method of claim 5, wherein, Before the step of sequentially merging adjacent character units to obtain at least one corresponding string, the method further comprises: determining whether the total number of character units exceeds a preset threshold; if the total number of character units does not exceed the preset threshold, taking the whole character units as a subword, and taking the subword as the subword sequence corresponding to the subword and various combinations thereof; and / or, before the step of sequentially determining the dictionary number corresponding to each string based on the word segmentation dictionary, the method further comprises: determining whether each string is queried in the word segmentation dictionary; if each string is not queried in the word segmentation dictionary, taking each string as a corresponding subword, and taking each subword as the subword sequence; and / or, the subword sequence is composed of at least one subword in the order of word composition.

8. The method of claim 2, wherein, Before the step of constructing the guidance database using the subword sequence, the method further comprises: deleting duplicate content in the subword sequence.

9. The method of claim 1, wherein, The step of decoding the audio encoded data using the decoder based on the guidance database comprises: in the case of first decoding, the decoder decodes the audio encoded data using the subwords and various combinations thereof existing in the guidance database to obtain a first subword probability distribution, sorts all subword probabilities in the first subword probability distribution in descending order according to probability size, and traverses decoding paths obtained after descending sorting, wherein the subword probability corresponds to the decoding path one by one; in the case of non-first decoding, based on the subwords corresponding to the decoding path of the last decoding, the decoder decodes the audio encoded data using the subwords and various combinations thereof existing in the guidance database to obtain a second subword probability distribution; adds all subword probabilities in the second subword probability distribution to the corresponding subword probabilities in the first preset number of last decoding paths to obtain a third subword probability distribution, sorts all subword probabilities in the third subword probability distribution in descending order according to probability size, and traverses decoding paths obtained after descending sorting, wherein the subword probability corresponds to the decoding path one by one.

10. The method of claim 9, wherein, The step of traversing decoding paths obtained after descending sorting comprises: in response to the number of traversed decoding paths not exceeding the first preset number, determining whether the number of end decoding paths exceeds the second preset number; in the case that the number of end decoding paths exceeds the second preset number, selecting a target path of the first preset number, and obtaining a subword sequence corresponding to the target path, wherein the target path includes the end decoding path; in the case that the number of end decoding paths does not exceed the second preset number, backtracking the last subword sequence in the currently traversed decoding path, and obtaining the subword sequence corresponding to the audio encoded number according to the last subword sequence.

11. The method of claim 10, wherein, The obtaining the sub-word sequence corresponding to the audio coding number according to the last sub-word sequence comprises: traversing the last sub-word sequence in the current decoding path; in response to not querying the last sub-word sequence in the guidance database, deleting the decoding path in which the last sub-word sequence is located; in response to querying the last sub-word sequence in the guidance database and the sub-word sequence existing an end mark, judging whether the last sub-word sequence is a complete word; in the case of being a complete word, retaining the decoding path in which the last sub-word sequence is located as an end decoding path, and adding one to the path number of the end decoding path; in the case of not being a complete word, deleting the current decoding path and traversing the next decoding path obtained by descending sorting.

12. The method of claim 1, wherein, The method for obtaining the target speech text corresponding to the to-be-recognized speech data comprises: obtaining at least one target path and a corresponding target path probability obtained by decoding the audio coding data, wherein the target path probability is obtained based on sub-words contained in a sub-word sequence in the target path; obtaining a weighted probability of the audio coding data based on the target path probability; sorting the weighted probabilities according to scores, and taking the sub-word sequence of the target path corresponding to the weighted probability meeting the path selection condition as the target speech text.

13. The method of claim 12, wherein, The method for obtaining a weighted probability of the audio coding data based on the target path probability comprises: obtaining an excitation probability of the audio coding data by using a unigram language model; weighting the target path probability by using the excitation probability to obtain the corresponding weighted probability, wherein the unigram language model is obtained by training based on guidance data.

14. The method of claim 1, wherein, Before the step of encoding the to-be-recognized speech data by using an encoder to obtain audio coding data, the method further comprises: preprocessing the to-be-recognized speech data and obtaining corresponding filter group features by preprocessing the to-be-recognized speech data; The method for obtaining audio coding data by using an encoder to encode the to-be-recognized speech data comprises: encoding the filter group features by using a configured encoder to obtain the audio coding data; and / or, the to-be-recognized speech data is a speech file in a standard waveform audio file format.

15. A device for assessing spoken language, characterized in that The device comprises: a speech data obtaining module configured to obtain to-be-recognized speech data; an audio coding data obtaining module configured to encode the to-be-recognized speech data by using an encoder to obtain audio coding data; a guidance database constructing module configured to construct a guidance database based on guidance data; wherein the guidance data comprises at least one of a general dictionary and a test paper resource; a target speech text obtaining module configured to decode the audio coding data by using a decoder based on the guidance database to obtain a target speech text corresponding to the to-be-recognized speech data; wherein the encoder is configured by using the guidance database.

16. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores program instructions, and the processor is configured to execute the program instructions to implement the spoken language assessment method according to any one of claims 1-14.

17. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store program instructions, and the program instructions can be executed to implement the spoken language assessment method according to any one of claims 1-14.

Images