Speech evaluation method, computer device, storage medium and computer program product

Abstract

The present application relates to a speech evaluation method, apparatus, computer equipment, storage medium and computer program product. By obtaining the matching result information of the speech to be evaluated and the corresponding sample text, the matching situation of the words contained in the speech to be evaluated and the words contained in the sample text is obtained, and based on the matching result information, the target words corresponding to the sample text in the speech to be evaluated are determined, and the phonemes of each target word are obtained, so as to determine the evaluation result of the speech to be evaluated under the preset evaluation index based on the matching result information, phonemes and other information. Compared with the traditional manual speech evaluation method, this solution uses the matching results of the speech to be evaluated and the sample text to evaluate multiple indicators based on the phonemes of each word contained in the speech to be evaluated, thereby achieving the effect of improving the objectivity of singing evaluation.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a speech evaluation method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] Singing, as one of the human arts, has a long history. Advances in computer technology have made it possible to record singing on smart devices through software and other means. After recording, the quality of the singing needs to be evaluated. Currently, this is typically done manually, assessing and evaluating the singing from multiple perspectives. However, manual voice evaluation is highly subjective and does not reflect the true quality of the singing.

[0003] Therefore, the current singing evaluation method has the defect of being non-objective. Summary of the Invention

[0004] Based on this, it is necessary to provide a voice evaluation method, device, computer equipment, computer-readable storage medium and computer program product that can improve the objectivity of singing evaluation in response to the above technical problems.

[0005] In a first aspect, the present application provides a speech evaluation method, the method comprising:

[0006] Obtaining a speech to be evaluated and a sample text corresponding to the speech to be evaluated, wherein the sample text includes a plurality of sample words;

[0007] Inputting the speech to be evaluated into a speech recognition model to obtain recognized words;

[0008] Obtaining matching result information between the recognized word and the sample text; the matching result information represents a matching condition between the recognized word and the sample word contained in the sample text;

[0009] Based on the matching result information, determining target words in the recognized words corresponding to the sample words, and obtaining phonemes corresponding to each target word;

[0010] An evaluation result of the speech to be evaluated under a preset evaluation index is determined based on the matching result information and the phonemes corresponding to the target word.

[0011] In one embodiment, obtaining matching result information between the recognized word and the sample text includes:

[0012] Obtaining a target speech matching model; the target speech matching model is trained based on the sample text;

[0013] The speech to be evaluated is recognized according to the target speech matching model, and at least one of the following matching result information is obtained: a first speech segment in the speech to be evaluated that matches the sample text, a second speech segment in the speech to be evaluated that does not match the sample text, and remaining sample words contained in the sample text but not matching words in the speech to be evaluated.

[0014] In one embodiment, if the matching result information includes a first speech segment, determining a target word corresponding to the sample word in the recognized words based on the matching result information, and obtaining a phoneme corresponding to each target word, includes:

[0015] Obtaining a word contained in the first speech segment as a target word;

[0016] For each frame of speech corresponding to the target word, obtaining the triphones corresponding to the frame of speech, and using the middle phoneme in the triphones as the frame phoneme corresponding to the frame of speech;

[0017] The phonemes of the target word are determined according to the frame phonemes of the plurality of frames of speech corresponding to the target word.

[0018] In one embodiment, the evaluation index includes pronunciation accuracy, which includes at least one of phoneme accuracy, word accuracy, and sentence accuracy; and determining the evaluation result of the speech to be evaluated under the preset evaluation index based on the matching result information and the phonemes corresponding to the target word includes:

[0019] For each phoneme of the target word, a feature vector corresponding to each frame in which the phoneme exists is obtained, and based on the feature vector, a posterior probability corresponding to the phoneme in each frame is obtained. Furthermore, based on the number of frames in which the phoneme exists and the posterior probability corresponding to the phoneme in each frame, the phoneme accuracy corresponding to the phoneme is determined; the posterior probability represents the probability that the phoneme accuracy of the phoneme is accurate given the feature vector;

[0020] For each target word in the speech to be evaluated, obtaining the number of phonemes contained in the target word, determining an average value of the phoneme accuracy of the target word based on the phoneme accuracy of each phoneme corresponding to the target word and the number of phonemes contained in the target word, and using the average value of the phoneme accuracy as the word accuracy corresponding to the target word;

[0021] For each sentence in the speech to be evaluated, obtaining the number of target words contained in the sentence, determining an average value of the word accuracy of the sentence based on the word accuracy of each target word corresponding to the sentence and the number of target words contained in the sentence, and using the average value of the word accuracy as the sentence accuracy corresponding to the sentence;

[0022] An evaluation result of the speech to be evaluated under pronunciation accuracy is determined according to at least one of the phoneme accuracy, the word accuracy, and the sentence accuracy.

[0023] In one embodiment, the evaluation indicator further includes sentence completeness. If the matching result information includes the second speech segment and the remaining sample words, determining the evaluation result of the speech to be evaluated under the preset evaluation indicator based on the matching result information and the phonemes corresponding to the target word includes:

[0024] Determining, based on the second speech segment, unmatched words contained in the speech to be evaluated but not matching sample words in the sample text;

[0025] An evaluation result of the speech to be evaluated under sentence completeness is determined according to the number of the unmatched words, the number of the remaining sample words, and the total number of sample words in the sample text.

[0026] In one embodiment, the method further includes: determining the start and end time of each target word in the speech to be evaluated based on the matching result information;

[0027] The evaluation index also includes fluency. Determining the evaluation result of the speech to be evaluated under the preset evaluation index based on the matching result information and the phonemes corresponding to the target word includes:

[0028] For each target word, obtaining the sample start and end time of the sample word corresponding to the target word in the sample text, and obtaining the matching degree between the start and end time of the target word and the sample start and end time;

[0029] Obtain the sum of the matching degrees of the start and end times of multiple target words contained in the speech to be evaluated and the start and end times of the sample, and obtain the ratio of the sum of the matching degrees to the total duration of the speech to be evaluated to determine the evaluation result of the speech to be evaluated in terms of fluency.

[0030] In one embodiment, the evaluation index further includes stress position accuracy, and determining the evaluation result of the speech to be evaluated under the preset evaluation index based on the matching result information and the phonemes corresponding to the target word includes:

[0031] Obtaining the speech energy, time length, and pitch corresponding to each target word in the speech to be evaluated;

[0032] Inputting the speech energy, time length and pitch into a preset classifier, and obtaining the stress position corresponding to each target word output by the preset classifier;

[0033] An evaluation result of the speech to be evaluated under the accuracy of the stress position is determined according to the stress position corresponding to each target word and the sample stress position of the corresponding sample word in the sample text.

[0034] In a second aspect, the present application provides a speech evaluation device, the device comprising:

[0035] A first acquisition module is used to acquire the speech to be evaluated and the sample text corresponding to the speech to be evaluated;

[0036] A second acquisition module is configured to acquire matching result information between the speech to be evaluated and the sample text; the matching result information represents matching conditions between words contained in the speech to be evaluated and sample words contained in the sample text;

[0037] A determination module is configured to determine target words in the speech to be evaluated based on the matching result information and obtain phonemes corresponding to each target word; the target words are matched with sample words corresponding to the sample text;

[0038] The evaluation module is used to determine the evaluation result of the speech to be evaluated under a preset evaluation index based on the matching result information and the phonemes corresponding to the target word.

[0039] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0041] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps of the above method when executed by a processor.

[0042] The aforementioned speech evaluation method, apparatus, computer device, storage medium, and computer program product obtain matching information between the speech to be evaluated and the corresponding sample text to determine the matching status of words contained in the speech to be evaluated and words contained in the sample text. Based on the matching information, the method then determines the target words in the speech to be evaluated that correspond to the sample text and obtains the phonemes of each target word. The method then determines the evaluation results of the speech to be evaluated under preset evaluation indicators based on the matching information, phonemes, and other information. Compared to traditional manual speech evaluation methods, this method utilizes the matching results between the speech to be evaluated and the sample text to evaluate multiple indicators based on the phonemes of each word contained in the speech to be evaluated, thereby achieving the effect of improving the objectivity of singing evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A diagram illustrating an application environment of a speech evaluation method according to an embodiment;

[0044] Figure 2 1 is a flow chart of a speech evaluation method according to an embodiment;

[0045] Figure 3 Schematic diagram of a matching process in one embodiment;

[0046] Figure 4 is a flowchart of a matching step in another embodiment;

[0047] Figure 5 A schematic diagram of an interface showing matching results in one embodiment;

[0048] Figure 6 A schematic diagram of an interface for a fluency evaluation step in one embodiment;

[0049] Figure 7 1 is a flow chart of a speech evaluation method according to another embodiment;

[0050] Figure 8 1 is a flow chart of a voice evaluation method according to another embodiment;

[0051] Figure 9 is a structural block diagram of a speech evaluation device in one embodiment;

[0052] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0054] The speech evaluation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 can obtain the user's input speech to be evaluated and obtain the corresponding sample text based on the speech to be evaluated. The terminal 102 can then match the speech to be evaluated with the sample text and obtain the phonemes in the speech to be evaluated based on the matching results. The terminal 102 can then evaluate the speech to be evaluated based on multiple evaluation indicators based on the phonemes. In addition, some embodiments also include a server 104. The terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104 or placed on the cloud or other network servers. The above-mentioned sample text can be stored on the server 104, and the terminal 102 can obtain the sample text corresponding to the speech to be evaluated from the server 104. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0055] Traditional automatic singing scoring and evaluation also involves establishing an evaluation mechanism through feature matching. This approach generally consists of three steps: 1. Extracting three characteristic parameters, such as intensity, pitch, and breath, from the test voice sample and the corresponding original voice; 2. Comparing these parameters using feature matching; and 3. Using a scoring mechanism to assign an objective score based on the degree of similarity. This approach primarily compares the characteristic parameters of the singer and the original singer, providing a reference-based evaluation method. However, this approach is limited by age, gender, and timbre differences between singers, as well as the accuracy of feature parameter estimation. The greater the difference from the original singer and the more inaccurate the estimated feature parameters, the less accurate the singing evaluation score. In other words, these methods all reference the original singer and require a reference template, making them supervised evaluation methods. The speech evaluation method provided in this embodiment, however, implements an unsupervised evaluation method by evaluating multiple metrics based on the phonemes of each word in the speech to be evaluated, based on the matching results between the speech to be evaluated and the sample text.

[0056] In one embodiment, Figure 2 As shown, a speech evaluation method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0057] Step S202 , obtaining a speech to be evaluated and a sample text corresponding to the speech to be evaluated, where the sample text includes a plurality of sample words; Step S204 , inputting the speech to be evaluated into a speech recognition model to obtain recognized words.

[0058] The speech to be evaluated may be a speech input by a user that needs to be evaluated and assessed. For example, the speech to be evaluated may be a song, and the user may be the singer. The terminal 102 may then evaluate and assess the song sung by the singer. The song sung by the user may have a corresponding original song. The terminal 102 may obtain the lyrics of the original song corresponding to the speech to be evaluated as the sample text corresponding to the speech to be evaluated. In addition, the terminal 102 may also input the speech to be evaluated into a speech recognition model to identify the recognition words contained in the speech to be evaluated. The sample text and the recognition words may both include the lyrics of the song, and may also include the timestamps corresponding to the lyrics, including the time when the lyrics appeared and when the lyrics ended.

[0059] Step S206 , obtaining matching result information between the recognized word and the sample text; the matching result information represents the matching status between the recognized word and the sample word included in the sample text.

[0060] Among them, speech-to-text alignment is the most basic processing process of the entire evaluation. It completes the alignment operation between the input text and the actual speech. Through this step of processing, we can obtain the time, acoustic score, content and other information of the corresponding original speech at the phoneme / word level. With this information, we can further combine other basic statistical models to calculate the required evaluation indicators. After the terminal 102 obtains the above-mentioned speech to be evaluated and the corresponding sample text, it can match and align the speech to be evaluated and the sample text. Specifically, it can be by identifying the recognized words in the speech to be evaluated and matching the recognized words with the corresponding words in the sample text. Among them, a word refers to the smallest unit of text with practical meaning in the above-mentioned text, such as a character or an English word. After matching and aligning the speech to be evaluated and the sample text, the terminal 102 can obtain corresponding matching result information. The matching result information includes the matching status of the words contained in the speech to be evaluated and the words contained in the sample text.

[0061] In actual applications, users often sing more, less, or sing incorrectly, so the conventional alignment strategy cannot be used. Usually, the content sung by the user and the given lyrics are not strictly matched, so the conventional alignment method cannot adapt to the situation where the user sings more, less, or misses a song. Therefore, the matching of the speech to be evaluated and the sample text can be based on a customized language model. Figure 3 As shown, Figure 3The figure is a flow chart of the matching step in one embodiment. Terminal 102 can use the sample text of the lyrics to train a language model defined by the lyrics. Thus, terminal 102 can perform a WFST (Weighted Finite-State Transducer) network search based on the acoustic model and the above-mentioned language recognition model to realize the recognition and extraction of lyrics in the sample text. For the input speech to be evaluated, terminal 102 can extract the feature vectors in the speech to be evaluated and perform alignment matching based on Viterbi algorithm decoding with the lyrics information in the sample text output by the WFST search network, thereby realizing the recognition of words in the speech to be evaluated and the alignment matching of words in the speech to be evaluated and the sample text. Among them, WFST is a weighted finite state transition machine, a search graph commonly used in speech recognition processing. It represents all possible language spaces. The scale of this graph can be large or small in different usage scenarios. The speech recognition process is actually a process of finding the optimal path on the WFST graph. Viterbi decoding is currently the mainstream ASR decoding algorithm. Each speech recognition process is a Viterbi decoding process. This algorithm searches for the optimal path on a specified WFST graph that best matches the output speech feature sequence and then outputs the label information for that path. Acoustic models can be based on triphone modeling. Traditional acoustic models are generally GMM-HMM speech recognition. Currently, mainstream acoustic models are generally neural networks, such as DNN-HMM and LSTM-HMM, for speech recognition. Neural network modeling methods are significantly more effective than GMM-HMM speech recognition methods.

[0062] Step S208: Based on the matching result information, determine the target words corresponding to the sample words in the recognized words, and obtain the phonemes corresponding to each target word; the target words are matched with the sample words corresponding to the sample text.

[0063] Based on the matching result information, terminal 102 can obtain the matching status of the words in the speech to be evaluated and the words in the sample text. Based on the matching status, terminal 102 can determine the target word in the speech to be evaluated that corresponds to the sample word. The target word can be a word that matches the sample word in the speech to be evaluated that corresponds to the sample text. After obtaining the target word in the speech to be evaluated, terminal 102 can obtain the phonemes corresponding to each target word. Phonemes, also known as phones, are the smallest modeling units of the acoustic model. The pronunciation of a word is composed of several phonemes. Phonemes are different from the pronunciation units in the International Phonetic Alphabet, but they have a certain corresponding relationship. In reality, people's pronunciation of a word will vary depending on the words before and after it. We call this contextual relevance. Therefore, triphones are often used as units in acoustic modeling. This triphone is a combination of three phonemes. For example, a-b+c represents the pronunciation of b, and its preceding and following phones are a and c respectively.

[0064] Step S210 : determining an evaluation result of the speech to be evaluated under a preset evaluation index based on the matching result information and the phonemes corresponding to the target word.

[0065] The matching result information may include the matching status of the words in the speech to be evaluated and the words in the corresponding sample text; the target word may include multiple phonemes. The terminal 102 may then evaluate the speech to be evaluated based on the above matching status and the phonemes of the above target word. The terminal 102 may evaluate the speech to be evaluated based on a variety of preset evaluation indicators to obtain corresponding evaluation results. The preset evaluation indicators may include at least one of the indicators such as pronunciation accuracy, sentence completeness, fluency and stress position accuracy. After the terminal 102 completes the above evaluation of the speech to be evaluated, it may output the evaluation results under the above evaluation indicators. For example, the terminal 102 may output at least one evaluation result of the pronunciation accuracy, sentence completeness, fluency and stress position accuracy of the speech to be evaluated to display to the user.

[0066] Goodness of Pronunciation (GOP) can include at least one of phoneme accuracy, word accuracy, and sentence accuracy. Phoneme accuracy is measured using phonemes as the smallest measurement unit, providing the most nuanced assessment of a word's pronunciation. In actual products, the International Phonetic Alphabet (IPA) is typically used, allowing users to determine the pronunciation quality of each phonetic unit within the phonetic alphabet. Word accuracy is calculated based on phoneme accuracy, typically taking the mean or weighted average of the phoneme scores for a word. Sentence accuracy is calculated based on word accuracy, typically taking the mean or weighted average of the word accuracy scores. Sentence accuracy generally does not consider words that are read more or less frequently; rather, only words whose pronunciation is detected are considered based on the text. Completion measures the ratio of the number of words actually pronounced by the user that match the text to the total number of words in the text. For words that are read less frequently by the user or have very low pronunciation accuracy scores, such as mispronunciations, terminal 102 can determine that they are missing from the text. Fluency is an important indicator for evaluating the quality of singing. It is used to measure whether the singer pauses for each word, prolongs the vowels appropriately, whether the singer can keep up with the rhythm of the accompaniment, and whether the singer is jumping the beat. Providing a result evaluation of singing fluency can effectively improve the user's singing speed and grasp the accompaniment rhythm. Stress position (STRESS) plays an important role in the pronunciation of English words. Correct stress pronunciation will make the pronunciation sound more pure and easier to understand. Even different stress positions can express different meanings. For example, in the word "Present", when the stress position is on the first syllable ['pre ], which means the noun gift. When the stress is on the second syllable, [pri'zent] means the verb present. Therefore, detecting whether the stress position is correct is also a very valuable evaluation point.

[0067] In the above-mentioned speech evaluation method, by obtaining matching information between the speech to be evaluated and the corresponding sample text, the matching status of the words contained in the speech to be evaluated and the words contained in the sample text is determined. Based on the matching information, the target words in the speech to be evaluated that correspond to the sample text are determined, and the phonemes of each target word are obtained. The evaluation results of the speech to be evaluated under the preset evaluation indicators are then determined based on the matching information, phonemes, and other information. Compared with traditional manual speech evaluation methods, this solution utilizes the matching results of the speech to be evaluated and the sample text to evaluate multiple indicators based on the phonemes of each word contained in the speech to be evaluated, thereby achieving the effect of improving the objectivity of singing evaluation.

[0068] In one embodiment, obtaining matching result information between recognized words and sample text includes: obtaining a target speech matching model; training the target speech matching model based on the sample text; recognizing the speech to be evaluated according to the target speech matching model, and obtaining at least one of the following matching result information: a first speech segment in the speech to be evaluated that matches the sample text, a second speech segment in the speech to be evaluated that does not match the sample text, and remaining sample words contained in the sample text but that do not match words in the speech to be evaluated.

[0069] In this embodiment, the target speech matching model can be a neural network model used to perform speech recognition on the speech to be evaluated. The target speech matching model can be trained based on sample text. Different sample texts can also result in different target speech matching models trained by terminal 102. That is, the target speech matching model is a recognition model limited by the sample text. After obtaining the target speech matching model, terminal 102 can recognize the speech to be evaluated based on the model and obtain corresponding matching result information, including at least one of a first speech segment in the speech to be evaluated that matches the sample text, a second speech segment in the speech to be evaluated that does not match the sample text, and remaining sample words contained in the sample text but that do not match words in the speech to be evaluated. The first speech segment can include the target word, the second speech segment can include words that the user read more, and the remaining sample words can be words that the user read less or misread. Thus, terminal 102 can obtain matching result information between the speech to be evaluated and the sample text based on at least one of the first speech segment, the second speech segment, and the remaining sample words.

[0070] The terminal 102 may perform a series of natural language training on the sample text and then perform WFST decoding on the feature vector extracted from the speech to be evaluated to obtain matching result information. In addition, in some embodiments, the terminal 102 may determine the corresponding start and end time of each target word in the speech to be evaluated based on the matching result information. For example, Figure 4 As shown, Figure 4The figure is a flowchart of the matching step in another embodiment. Terminal 102 can perform N-gram training on a sample text containing lyrics to generate a corresponding G.fst, i.e., a language model corresponding to the sample text. Terminal 102 can also further generate an HCLG file based on G.fst. Terminal 102 can also perform feature extraction from the audio file of the speech to be evaluated to obtain a corresponding feature vector. Terminal 102 can then perform WFST decoding based on the HCLG file and feature vector corresponding to the sample text, ultimately obtaining the words in the speech to be evaluated and the corresponding start and end times of each word. N-Gram is an algorithm based on a statistical language model. Its basic idea is to perform a sliding window operation of size N on the content of the text according to bytes, forming a sequence of byte segments of length N; the HCLG file can be a file composed of H.fst, C.fst, L.fst, and G.fst through a series of algorithms (combination, determinization, and minimization, etc.), H.fst can be an HMM model, C.fst can be the context of the sample text, and L.fst can be a dictionary; the above-mentioned speech to be evaluated can be divided into multiple frames. Since the speech data cannot be processed directly, the terminal 102 needs to extract processable feature information from the speech, such as MFCC, FBank, and energy, etc. The terminal 102 can divide the speech into a series of segments called frames according to a fixed time length through a speech feature extraction algorithm, and then extract feature data on each frame, and finally output a sequence of feature vectors as the basic data for subsequent speech processing. After the terminal 102 obtains the above-mentioned matching result information, it can also be displayed, such as Figure 5 As shown, Figure 5 FIG2 is a schematic diagram of an interface showing matching results in one embodiment. The matching result information obtained by terminal 102 may include multiple target words contained in the speech to be evaluated and corresponding to the sample text. These words have start and end timestamps, forming corresponding start and end times, which may be consistent with the start and end times of the corresponding words in the sample text.

[0071] Through this embodiment, the terminal 102 can identify the words contained in the speech to be evaluated through the speech recognition model corresponding to the sample text, and obtain corresponding matching result information by matching with the sample text, so that the terminal 102 can evaluate the speech to be evaluated based on the matching result information, thereby improving the objectivity of the singing evaluation.

[0072] In one embodiment, if the matching result information includes a first speech segment, based on the matching result information, the target word corresponding to the sample word in the recognized word is determined, and the phonemes corresponding to each target word are obtained, including: obtaining the word contained in the first speech segment as the target word; for each frame of speech corresponding to the target word, obtaining the three phonemes corresponding to the frame of speech, and using the middle phoneme in the three phonemes as the frame phoneme corresponding to the frame of speech; and determining the phonemes of the target word based on the frame phonemes of multiple frames of speech corresponding to the target word.

[0073] In this embodiment, the terminal 102 can match and align the speech to be evaluated with the sample text to obtain corresponding matching result information. The terminal 102 can determine the target word in the speech to be evaluated based on the matching result information. The first speech segment in the matching result information can be a segment that matches the speech to be evaluated and its corresponding sample text, and the terminal 102 can obtain the word contained in the first speech segment as the target word. The terminal 102 can also obtain the phonemes of each target word. The start and end time of the target word in the speech to be evaluated can span multiple frames of speech. For each frame of speech corresponding to the target word, the terminal 102 can obtain the three phonemes corresponding to the frame of speech, and obtain the frame phonemes corresponding to the frame of speech by removing the intermediate phonemes. The frame speech can be the speech of each frame in the multiple frames spanned by the target word, and the frame phonemes can be the phonemes corresponding to the speech of each frame. As can be seen from the above, when the user speaks the target word, the target word may last for multiple frames in the speech to be evaluated. Then, the terminal 102 can determine the phonemes of the target word based on the frame phonemes corresponding to the multiple frames of speech in the first speech segment to be evaluated.

[0074] Through this embodiment, the terminal 102 can determine the phonemes of the target word based on the target word that matches the sample text in the speech to be evaluated through a three-phoneme method, so that the terminal 102 can evaluate the speech to be evaluated based on different evaluation indicators based on phonemes, thereby improving the objectivity of the singing evaluation.

[0075] In one embodiment, based on the matching result information and the phonemes corresponding to the target words, the evaluation result of the speech to be evaluated under the preset evaluation index is determined, including: for each phoneme of the target word, obtaining the feature vector corresponding to the phoneme of each frame in which the phoneme exists, obtaining the posterior probability corresponding to the phoneme of each frame according to the feature vector, and determining the phoneme accuracy corresponding to the phoneme according to the number of frames in which the phoneme exists and the posterior probability corresponding to the phoneme in each frame; for each target word in the speech to be evaluated, obtaining the number of phonemes contained in the target word, and determining the phoneme accuracy corresponding to the phoneme according to the phoneme accuracy of each phoneme corresponding to the target word and the posterior probability of the phoneme corresponding to the target word. The method comprises the following steps: first, obtaining the number of target words contained in the sentence, and determining the average value of the phoneme accuracy of the target word according to the number of phonemes contained, and taking the average value of the phoneme accuracy as the word accuracy corresponding to the target word; second, obtaining the number of target words contained in the sentence, and determining the average value of the word accuracy of the sentence according to the word accuracy of each target word corresponding to the sentence and the number of target words contained in the sentence, and taking the average value of the word accuracy as the sentence accuracy corresponding to the sentence; and third, determining the evaluation result of the speech to be evaluated under pronunciation accuracy according to at least one of the phoneme accuracy, word accuracy and sentence accuracy.

[0076] In this embodiment, the evaluation index includes pronunciation accuracy, and pronunciation accuracy includes at least one of phoneme accuracy, word accuracy, and sentence accuracy. The terminal 102 can evaluate the speech to be evaluated based on pronunciation accuracy. The terminal 102 can determine the phoneme accuracy of the speech to be evaluated. The above-mentioned target words may include multiple ones. For each phoneme of the target word, the terminal 102 can obtain the feature vector corresponding to the phoneme in each frame of the speech occupied by the target word, and obtain the posterior probability corresponding to the phoneme in each frame of the phoneme based on the feature vector, so that the terminal 102 can determine the phoneme accuracy corresponding to the phoneme based on the number of frames where the phoneme exists and the posterior probability corresponding to the phoneme in each frame. Among them, the posterior probability represents the probability that the phoneme accuracy of the phoneme is accurate when the feature vector is known; the terminal 102 can calculate the posterior probability using a neural network as an acoustic model. For example, during the above alignment and matching, the terminal 102 can obtain the triphone corresponding to each frame of speech. We take the middle phone of the triphone as the actual phone. At the same time, we can also obtain the posterior probability output by the NN network: P(p|0), where p is the phone and 0 is the feature vector of the corresponding frame. A phoneme spans several frames. The terminal 102 can take the mean value of the posterior probability of the phoneme GOP(p) as the evaluation score of the phoneme. The calculation formula is as follows:

[0077]

[0078] Among them, t s and te Indicates the start and end frames of phoneme p in the target word, 0 t Represents the features of frame t.

[0079] After obtaining the above-mentioned phoneme accuracy, terminal 102 can calculate the accuracy of the target word based on the phoneme accuracy. There can be multiple target words. For each target word in the speech to be evaluated, terminal 102 can obtain the number of phonemes corresponding to the target word and determine the average phoneme accuracy of the target word based on the phoneme accuracy and number of phonemes of each phoneme corresponding to the target word, thereby using the average phoneme accuracy as the word accuracy corresponding to the target word. Specifically, terminal 102 can obtain the word accuracy based on the phoneme GOP average value, and its calculation formula is as follows:

[0080]

[0081] Among them, GOP(w) represents word accuracy, N represents the number of phonemes contained in the target word w, and p s Indicates the first phoneme in the target word, p e Indicates the last phoneme in the target word.

[0082] The terminal 102 can also calculate sentence accuracy based on word accuracy. The above-mentioned multiple target words can form a sentence. For each sentence in the speech to be evaluated, the terminal 102 can obtain the number of target words contained in the sentence, and determine the average word accuracy of the sentence based on the word accuracy of each target word corresponding to the sentence and the number of target words, so as to use the average word accuracy as the sentence accuracy corresponding to the sentence. Specifically, for sentence accuracy, the terminal 102 can only consider the pronunciation of the detected words that match the input text, and will not consider words that are read more or less. The terminal 102 can take the GOP average of the above-mentioned matching target words as the pronunciation accuracy of the sentence. The calculation formula is as follows:

[0083]

[0084] Among them, GOP(s) is the sentence accuracy, w m Indicates the target word in the detection result that matches the sample text S, and N is the number of matching target words. For example, if the sample text is ABCD and the words detected in the speech to be evaluated are AKBC, the terminal 102 can determine that the words detected in the above results that match the input text are A, B, and C. Word K is a word that the user reads more often, and word D is a word that the user reads less often. In this case, the terminal 102 can only consider A, B, and C for sentence accuracy.

[0085] Thus, the terminal 102 can determine the evaluation result of the speech to be evaluated under the pronunciation accuracy based on at least one of the phoneme accuracy, word accuracy, and sentence accuracy. For example, the terminal 102 can output the evaluation result of at least one of the phoneme accuracy, word accuracy, and sentence accuracy of the speech to be evaluated under the pronunciation accuracy on the speech evaluation interface, and display the evaluation result of the at least one of the accuracy items to the user on the speech evaluation interface.

[0086] Through this embodiment, the terminal 102 can obtain the pronunciation accuracy of the speech to be evaluated based on the posterior probability mean value, thereby improving the objectivity of the singing evaluation.

[0087] In one embodiment, if the matching result information includes a second speech segment and the remaining sample words, the evaluation result of the speech to be evaluated under the preset evaluation indicators is determined based on the matching result information and the phonemes corresponding to the target words, including: determining the unmatched words contained in the speech to be evaluated but not matching the sample words in the sample text based on the second speech segment; determining the evaluation result of the speech to be evaluated under sentence completeness based on the number of unmatched words, the number of remaining sample words and the total number of sample words in the sample text.

[0088] In this embodiment, the above-mentioned evaluation index also includes sentence completeness. It can be seen from the matching result information of the speech to be evaluated and the sample text that the matching result may include at least one of the first speech segment, the second speech segment and the remaining words. The second speech segment contains unmatched words that are contained in the speech to be evaluated but do not match the sample words in the sample text. If the matching result information includes the second speech segment and the remaining sample words, the terminal 102 can determine the above-mentioned unmatched words from the second speech segment, and obtain the sentence completeness corresponding to the speech to be evaluated based on the number of unmatched words, the number of remaining sample words and the total number of sample words in the sample text. For example, the terminal 102 can compare the number of words in the speech to be evaluated after removing the above-mentioned unmatched words and the remaining sample words with the total number of sample words to obtain sentence completeness, thereby determining the evaluation result of the speech to be evaluated under sentence completeness.

[0089] In some embodiments, the calculation condition for sentence completeness may further include a pronunciation accuracy GOP. When the pronunciation accuracy GOP is below a specified threshold, terminal 102 may determine that the word is an incompletely pronounced word. Specifically, terminal 102 may measure the ratio of the number of words actually pronounced by the user to the number of words in the text to determine sentence completeness. For words that are underpronounced, overpronounced, mispronounced, or have a pronunciation GOP value below a specified threshold, terminal 102 considers them incompletely pronounced and excludes them from the calculation of sentence completeness.

[0090] Through this embodiment, the terminal 102 can identify the sentence completeness in the speech to be evaluated based on the matching result information and the pronunciation accuracy information, thereby improving the objectivity of the singing evaluation.

[0091] In one embodiment, based on the matching result information and the phonemes corresponding to the target words, the evaluation result of the speech to be evaluated under the preset evaluation indicators is determined, including: for each target word, obtaining the sample start and end time of the sample word corresponding to the target word in the sample text, and obtaining the matching degree between the start and end time of the target word and the sample start and end time; obtaining the sum of the matching degrees of the start and end time of multiple target words contained in the speech to be evaluated and the sample start and end time, and obtaining the ratio of the sum of the matching degrees to the total duration of the speech to be evaluated, to determine the evaluation result of the speech to be evaluated under fluency.

[0092] In this embodiment, the above-mentioned evaluation index also includes fluency, and the terminal 102 can also determine the corresponding start and end time of each target word in the speech to be evaluated based on the matching result information. The start and end time can be expressed in the form of a timestamp. The terminal 102 can calculate the user's singing fluency for the word. The above-mentioned target words may include multiple, and for each target word, the terminal 102 can obtain the sample start and end time of the sample word corresponding to the target word in the sample text, including the sample start time and the sample end time. Thus, the terminal 102 can obtain the matching degree of the start and end time of the above-mentioned target word and the above-mentioned sample start and end time, and the terminal 102 can determine the evaluation result of the speech to be evaluated under fluency based on the ratio of the sum of the matching degrees of multiple target words to the total duration of the speech to be evaluated.

[0093] Specifically, the interface diagram of the terminal 102 calculating the fluency is as follows: Figure 6 As shown, Figure 6 The figure is a schematic diagram of the interface of the fluency evaluation step in one embodiment. The calculation of fluency by terminal 102 is based on the timestamp of each word calculated in the above matching. The original song, that is, the sample text has the timestamp of each word, including the start time of the sample word, which is called ground truth. Terminal 102 can obtain the start timestamp of each word from the singer’s speech to be evaluated through recognition alignment technology, which is called Predict output or System output. In this way, terminal 102 can count the segment area that correctly matches each word and calculate the fluency score. Combined with Figure 6 , the calculation formula is as follows: Accuracy = (Lenth of "correct" regions) / (Total lenth of songs). Among them, Accuracy is the fluency, and Lenth of "correct" regions is Figure 6The length of the correct area in the song, and Total lenth of songs is the total length of the speech to be evaluated.

[0094] Through this embodiment, the terminal 102 can determine the fluency index based on the matching degree between the start and end time of the target word and the start and end time of the sample word, thereby improving the objectivity of the evaluation of the speech to be evaluated.

[0095] In one embodiment, based on the matching result information and the phonemes corresponding to the target words, the evaluation results of the speech to be evaluated under preset evaluation indicators are determined, including: obtaining the speech energy, time length and fundamental pitch corresponding to each target word in the speech to be evaluated; inputting the speech energy, time length and fundamental pitch into a preset classifier to obtain the stress position corresponding to each target word output by the preset classifier; and determining the evaluation results of the speech to be evaluated under the stress position accuracy based on the stress position corresponding to each target word and the sample stress position of the corresponding sample word in the sample text.

[0096] In this embodiment, the evaluation index also includes stress position accuracy. Terminal 102 can obtain the speech energy, duration, and pitch corresponding to each target word in the speech to be evaluated, input the speech energy, duration, and pitch into a preset classifier, and obtain the stress position corresponding to each target word output by the preset classifier. Terminal 102 can then determine the evaluation result of the speech to be evaluated in terms of stress position accuracy based on the stress position corresponding to each target word and the sample stress position of the corresponding sample word in the sample text.

[0097] The preset classifier can be an SVN (subversion) classifier, and terminal 102 can determine the stress position based on the SVN classifier. For example, the position of word stress is clearly marked in the International Phonetic Alphabet. Terminal 102 can determine the speech segment to be detected based on the stress position of the word phonemes in the basic statistical model and the phoneme information output by word decoding, and apply relevant algorithms to detect whether the pronunciation of the segment is truly a stress. In a speech signal, the position, speech energy, duration, and pitch of the stress are significantly different from those of other non-stressed positions. Therefore, terminal 102 can use these three variables as a vector and train an SVM (support vector machine) classifier using a certain amount of manually annotated data to detect the stress position STRESS(w) in the target word. The calculation formula is as follows: STRESS(w) = SVM(E, T, P). Where E represents the average speech energy in the speech segment to be detected, T represents the duration of the speech segment to be detected, and P represents the pitch. Thus, the terminal 102 may determine the stress position accuracy based on the comparison result of the stress position of the target word with the stress position of the sample word.

[0098] Through this embodiment, the terminal 102 can evaluate the accuracy of the stress position of the words in the speech to be evaluated based on the speech energy, time length and fundamental pitch, thereby improving the objectivity of the singing evaluation.

[0099] In one embodiment, Figure 7 As shown, Figure 7 The figure is a flow chart of a speech evaluation method according to another embodiment. The method includes the following steps: Terminal 102 receives a user-input speech to be evaluated and a corresponding sample text of the speech to be evaluated, performs alignment matching between the input speech and the sample text based on the target speech matching model, and obtains basic information such as phonemes and target words in the speech to be evaluated. Terminal 102 can then calculate multiple evaluation indicators for the speech to be evaluated using a basic statistical model formed by different evaluation indicators.

[0100] The evaluation process of the speech to be evaluated is divided into three parts: input, composition and decoding, and index calculation. Figure 8 As shown, Figure 8The figure is a flow chart of a speech evaluation method according to another embodiment. Terminal 102 can extract features from the speech to be evaluated, construct a sample text based on a pronunciation dictionary, and input the constructed sample text into the WSFT network. Using Viterbi decoding and a preset acoustic model, the feature vectors extracted from the speech to be evaluated and the sample text are aligned and matched, and metrics are calculated based on the matching results. The pronunciation dictionary contains all the vocabulary supported by the engine and the phone sequence for each word.

[0101] Through the above embodiment, the terminal 102 can use the matching results of the speech to be evaluated and the sample text to evaluate multiple indicators based on the phonemes of each word contained in the speech to be evaluated, thereby achieving the effect of improving the objectivity of the singing evaluation. In addition, the solution provided by this application obtains information such as the time, acoustic score, and content of the corresponding original speech at the phoneme / word level through recognition and alignment. With this information, the terminal 102 combines other basic statistical models to calculate the required evaluation indicators, automatically evaluate and score the singer's pronunciation, detect errors in singing, and allow users to continuously adjust their singing methods to improve their singing level. At the same time, this technology is also deployed in K song recording products to timely score each user's work, select high-quality works, promptly discover potential singers, and promote and disseminate their works to achieve a win-win situation.

[0102] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0103] Based on the same inventive concept, embodiments of the present application also provide a speech evaluation device for implementing the aforementioned speech evaluation method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more speech evaluation device embodiments provided below can be found in the above-mentioned limitations of the speech evaluation method and will not be repeated here.

[0104] In one embodiment, Figure 9As shown, a speech evaluation device is provided, comprising: a first acquisition module 500, a second acquisition module 502, a determination module 504 and an evaluation module 506, wherein:

[0105] The first acquisition module 500 is configured to acquire a speech to be evaluated and a sample text corresponding to the speech to be evaluated, wherein the sample text includes a plurality of sample words; and input the speech to be evaluated into a speech recognition model to obtain recognized words.

[0106] The second acquisition module 502 is configured to acquire matching result information between the recognized word and the sample text; the matching result information represents a matching condition between the recognized word and the sample word.

[0107] The determination module 504 is configured to determine target words corresponding to the sample words in the recognized words based on the matching result information, and obtain phonemes corresponding to each target word.

[0108] The evaluation module 506 is configured to determine an evaluation result of the speech to be evaluated under a preset evaluation index based on the matching result information and the phonemes corresponding to the target word.

[0109] In one embodiment, the above-mentioned second acquisition module 502 is specifically used to obtain a target speech matching model; the target speech matching model is obtained based on sample text training; the speech to be evaluated is identified according to the target speech matching model, and at least one of the following matching result information is obtained: a first speech segment in the speech to be evaluated that matches the sample text, a second speech segment in the speech to be evaluated that does not match the sample text, and the remaining sample words contained in the sample text but do not match the words in the speech to be evaluated; based on the first speech segment, the second speech segment and the remaining sample words, the matching result information between the speech to be evaluated and the sample text is obtained.

[0110] In one embodiment, if the matching result information includes a first voice segment, the above-mentioned determination module 504 is specifically used to obtain the word contained in the first voice segment as the target word; for each frame of speech corresponding to the target word, obtain the three phonemes corresponding to the frame of speech, and use the middle phoneme in the three phonemes as the frame phoneme corresponding to the frame of speech; determine the phoneme of the target word based on the frame phonemes of multiple frames of speech corresponding to the target word.

[0111] In one embodiment, the evaluation module 506 is specifically used to obtain, for each phoneme of the target word, a feature vector corresponding to each frame of the phoneme in which the phoneme exists, obtain the posterior probability corresponding to the phoneme in each frame of the phoneme based on the feature vector, and determine the phoneme accuracy corresponding to the phoneme based on the number of frames in which the phoneme exists and the posterior probability corresponding to the phoneme in each frame; the posterior probability represents the probability that the phoneme accuracy of the phoneme is accurate when the feature vector is known; for each target word in the speech to be evaluated, obtain the number of phonemes corresponding to the target word, and determine the phoneme accuracy corresponding to the phoneme based on the phoneme accuracy of each phoneme contained in the target word and the posterior probability of the phoneme in the target word. The number of phonemes contained in the word is used to determine the average value of the phoneme accuracy of the target word, and the average value of the phoneme accuracy is used as the word accuracy corresponding to the target word; for each sentence in the speech to be evaluated, the number of target words contained in the sentence is obtained, and the average value of the word accuracy of the sentence is determined according to the word accuracy of each target word corresponding to the sentence and the number of target words contained in the sentence, and the average value of the word accuracy is used as the sentence accuracy corresponding to the sentence; based on at least one of the phoneme accuracy, word accuracy and sentence accuracy, the evaluation result of the speech to be evaluated under the pronunciation accuracy is determined.

[0112] In one embodiment, if the matching result information includes a second voice segment and remaining sample words, the above-mentioned evaluation module 506 is specifically used to determine, based on the second voice segment, unmatched words contained in the voice to be evaluated but that do not match the sample words in the sample text; and determine the evaluation result of the voice to be evaluated under sentence completeness based on the number of unmatched words, the number of remaining sample words, and the total number of sample words in the sample text.

[0113] In one embodiment, the apparatus further includes: a start and end time determination module configured to determine the start and end time corresponding to each target word in the speech to be evaluated based on the matching result information.

[0114] In one embodiment, the above-mentioned evaluation module 506 is specifically used to obtain, for each target word, the sample start and end time of the sample word corresponding to the target word in the sample text, and obtain the matching degree between the start and end time of the target word and the sample start and end time; obtain the sum of the matching degrees of the start and end time of multiple target words contained in the speech to be evaluated and the sample start and end time, and obtain the ratio of the sum of the matching degrees to the total duration of the speech to be evaluated, and determine the evaluation result of the speech to be evaluated in terms of fluency.

[0115] In one embodiment, the above-mentioned evaluation module 506 is specifically used to obtain the speech energy, time length and fundamental pitch corresponding to each target word in the speech to be evaluated; input the speech energy, time length and fundamental pitch into a preset classifier to obtain the stress position corresponding to each target word output by the preset classifier; and determine the evaluation result of the speech to be evaluated under the stress position accuracy based on the stress position corresponding to each target word and the sample stress position of the corresponding sample word in the sample text.

[0116] Each module in the aforementioned speech evaluation device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0117] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a voice evaluation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0118] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0119] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned speech evaluation method when executing the computer program.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned speech evaluation method is implemented.

[0121] In one embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned speech evaluation method when executed by a processor.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0123] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0124] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A speech evaluation method, characterized in that: The method comprises: Obtaining a speech to be evaluated and a sample text corresponding to the speech to be evaluated, wherein the sample text includes a plurality of sample words; Inputting the speech to be evaluated into a speech recognition model to obtain recognized words; Obtaining matching result information between the recognized word and the sample text; the matching result information represents a matching condition between the recognized word and the sample word; Based on the matching result information, determining the target words in the recognized words corresponding to the sample words, determining the start and end times of each target word in the speech to be evaluated, and obtaining the phonemes corresponding to each target word; Based on the matching result information and the phonemes corresponding to the target words, the evaluation result of the speech to be evaluated under the preset evaluation indicators is determined, including: obtaining the timestamps of each target word in the speech to be evaluated and the timestamps of each corresponding sample word, obtaining the area where each target word overlaps with the corresponding sample word as the correct area, obtaining the ratio of the length of the correct area to the total duration of the speech to be evaluated, and obtaining the evaluation result of the speech to be evaluated under fluency.

2. The method according to claim 1, characterized in that The obtaining of matching result information between the recognized word and the sample text includes: Obtaining a target speech matching model; the target speech matching model is trained based on the sample text; The speech to be evaluated is recognized according to the target speech matching model, and at least one of the following matching result information is obtained: a first speech segment in the speech to be evaluated that matches the sample text, a second speech segment in the speech to be evaluated that does not match the sample text, and remaining sample words contained in the sample text but not matching words in the speech to be evaluated.

3. The method according to claim 2, characterized in that If the matching result information includes the first speech segment, determining target words corresponding to the sample words in the recognized words based on the matching result information, and obtaining phonemes corresponding to each target word, includes: Obtaining a target word contained in the first speech segment; For each frame of speech corresponding to the target word, obtaining the triphones corresponding to the frame of speech, and using the middle phoneme in the triphones as the frame phoneme corresponding to the frame of speech; The phonemes of the target word are determined according to the frame phonemes of the plurality of frames of speech corresponding to the target word.

4. The method according to claim 2, characterized in that The evaluation index also includes sentence completeness. If the matching result information includes the second speech segment and the remaining sample words, determining the evaluation result of the speech to be evaluated under the preset evaluation index based on the matching result information and the phonemes corresponding to the target word includes: Determining, based on the second speech segment, unmatched words contained in the speech to be evaluated but not matching sample words in the sample text; An evaluation result of the speech to be evaluated under sentence completeness is determined according to the number of unmatched words, the number of remaining sample words, and the total number of sample words in the sample text.

5. The method according to claim 1, wherein The evaluation index includes pronunciation accuracy, which includes at least one of phoneme accuracy, word accuracy, and sentence accuracy; Determining an evaluation result of the speech to be evaluated under a preset evaluation index based on the matching result information and the phonemes corresponding to the target word includes: For each phoneme of the target word, a feature vector corresponding to each frame in which the phoneme exists is obtained, and based on the feature vector, a posterior probability corresponding to the phoneme in each frame is obtained. Furthermore, based on the number of frames in which the phoneme exists and the posterior probability corresponding to the phoneme in each frame, the phoneme accuracy corresponding to the phoneme is determined; the posterior probability represents the probability that the phoneme accuracy of the phoneme is accurate given the feature vector; For each target word in the speech to be evaluated, obtaining the number of phonemes contained in the target word, determining an average value of the phoneme accuracy of the target word based on the phoneme accuracy of each phoneme corresponding to the target word and the number of phonemes contained in the target word, and using the average value of the phoneme accuracy as the word accuracy corresponding to the target word; For each sentence in the speech to be evaluated, obtaining the number of target words contained in the sentence, determining an average value of the word accuracy of the sentence based on the word accuracy of each target word corresponding to the sentence and the number of target words contained in the sentence, and using the average value of the word accuracy as the sentence accuracy corresponding to the sentence; An evaluation result of the speech to be evaluated under pronunciation accuracy is determined according to at least one of the phoneme accuracy, the word accuracy, and the sentence accuracy.

6. The method according to claim 1, characterized in that The evaluation index also includes stress position accuracy. The determining of the evaluation result of the speech to be evaluated under the preset evaluation index based on the matching result information and the phonemes corresponding to the target word includes: Obtaining the speech energy, time length, and pitch corresponding to each target word in the speech to be evaluated; Inputting the speech energy, time length and pitch into a preset classifier, and obtaining the stress position corresponding to each target word output by the preset classifier; An evaluation result of the speech to be evaluated under the accuracy of the stress position is determined according to the stress position corresponding to each target word and the sample stress position of the corresponding sample word in the sample text.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images