Audio evaluation method, computer device and storage medium

Abstract

The application relates to an audio evaluation method, a computer device and a storage medium. By obtaining a plurality of emission probabilities corresponding to the audio to be evaluated, obtaining first transition probabilities between each template character in a template audio text corresponding to the audio to be evaluated, and second transition probabilities between each preset character in a preset audio text database, determining a first decoding path and a second decoding path based on the decoding of the emission probabilities, the first transition probabilities and the second transition probabilities, comparing a template decoding path corresponding to the template audio with the first decoding path and the second decoding path, and determining an evaluation result of the audio to be evaluated based on the first evaluation value and the second evaluation value determined thereby. Compared with the traditional multi-dimensional evaluation of audio, the scheme decodes two evaluation values based on the emission probabilities, the first transition probabilities and the second transition probabilities, and then determines the evaluation result, solves the misjudgment problem in multi-dimensional evaluation, and improves the accuracy of audio evaluation.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio evaluation method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, people can now listen to and sing songs on various computer terminals. For example, they can sing songs using singing software on the terminal. After singing a song in the software, the user needs to rate the audio to assess their singing level. Currently, the common method for rating song audio is through multiple dimensions of singing techniques. However, because each user's singing style is different, evaluating song audio using a fixed set of dimensions can lead to misjudgments and reduce the accuracy of the evaluation.

[0003] Therefore, current methods for evaluating audio suffer from low accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide an audio evaluation method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the accuracy of evaluation in response to the above-mentioned technical problems.

[0005] Firstly, this application provides an audio evaluation method, the method comprising:

[0006] Obtain the audio to be evaluated, and obtain the template audio and template audio text corresponding to the audio to be evaluated;

[0007] Obtain multiple emission probabilities corresponding to audio frames in the audio to be evaluated; the emission probability represents the probability that the audio frame belongs to each pronunciation state in the template audio; the pronunciation state represents the pronunciation stage of the text to which the audio frame in the audio to be evaluated belongs;

[0008] Obtain the first transition probability between the pronunciation states corresponding to each template character in the template audio text, and obtain the second transition probability between the pronunciation states corresponding to each preset character in the preset audio text database; the preset audio text database contains the template audio text and the amount of text data in the preset audio text database is greater than the amount of text data in the template audio text.

[0009] A first decoding path is determined based on the emission probability and the first transition probability, and a second decoding path is determined based on the emission probability and the second transition probability. A first evaluation value and a second evaluation value are determined based on the comparison results between the template decoding path corresponding to the template audio and the first decoding path and the second decoding path, respectively.

[0010] The evaluation result of the audio to be evaluated is determined based on the first evaluation value and the second evaluation value.

[0011] In one embodiment, obtaining multiple emission probabilities corresponding to audio frames in the audio to be evaluated includes:

[0012] Obtain the audio features corresponding to the audio to be evaluated;

[0013] The audio features are input into a Gaussian mixture model to obtain multiple emission probabilities corresponding to the audio features of each audio frame output by the Gaussian mixture model; the Gaussian mixture model is trained based on the pronunciation state to which each frame of template audio belongs.

[0014] In one embodiment, obtaining the first transition probability between the pronunciation states corresponding to each template character in the template audio text, and obtaining the second transition probability between the pronunciation states corresponding to each preset character in the preset audio text database, includes:

[0015] Obtain the first phoneme sequence corresponding to each template character in the template audio text; the first phoneme sequence includes multiple first phonemes corresponding to each template character.

[0016] Based on the pronunciation state of each first phoneme in the first phoneme sequence and the pronunciation state of each first phoneme's adjacent first phonemes, the first transition probability between each pronunciation state in the template audio text is determined.

[0017] Obtain the second phoneme sequence corresponding to each preset character in the preset audio text database; the second phoneme sequence includes multiple second phonemes corresponding to each preset character.

[0018] Based on the pronunciation state of each second phoneme in the second phoneme sequence and the pronunciation state of each second phoneme's adjacent second phonemes, the second transition probability between each pronunciation state in the preset audio text database is determined.

[0019] In one embodiment, determining the first decoding path based on the transmission probability and the first transition probability, and determining the second decoding path based on the transmission probability and the second transition probability, includes:

[0020] For each audio frame, within the range of pronunciation states corresponding to the template text, the first transition probability of the audio frame to the next audio frame in each pronunciation state is obtained, and the first frame probability value corresponding to the audio frame is determined based on the weighted sum of the emission probability corresponding to the audio frame and the first transition probability.

[0021] A first decoding path is determined based on the first frame probability value corresponding to each audio frame in the audio to be evaluated; the sum of the first frame probability values ​​corresponding to the audio frames in the first decoding path is the largest.

[0022] For each audio frame, within the preset audio text database, the second transition probability of the audio frame to the next audio frame in each pronunciation state is obtained, and the second frame probability value corresponding to the audio frame is determined based on the weighted sum of the emission probability corresponding to the audio frame and the second transition probability.

[0023] A second decoding path is determined based on the second frame probability value corresponding to each audio frame in the audio to be evaluated; the sum of the second frame probability values ​​corresponding to the audio frames in the second decoding path is maximized.

[0024] In one embodiment, determining the first evaluation value and the second evaluation value based on the comparison results between the template decoding path corresponding to the template audio and the first decoding path and the second decoding path, respectively, includes:

[0025] A first pronunciation state sequence corresponding to the audio to be evaluated is determined according to the first decoding path, and a second pronunciation state sequence corresponding to the audio to be evaluated is determined according to the second decoding path.

[0026] Based on the template decoding path corresponding to the template audio, determine the template pronunciation state sequence corresponding to the template audio;

[0027] A first evaluation value is determined based on the similarity between the template pronunciation state sequence and the first pronunciation state sequence; and a second evaluation value is determined based on the similarity between the template pronunciation state sequence and the second pronunciation state sequence.

[0028] In one embodiment, determining the evaluation result of the audio to be evaluated based on the first evaluation value and the second evaluation value includes:

[0029] Obtain the ratio of the first evaluation value to the second evaluation value, and determine the evaluation result of the audio to be evaluated based on the logarithm of the ratio.

[0030] In one embodiment, obtaining the template audio corresponding to the audio to be evaluated includes:

[0031] Obtain the original audio and / or at least one cover audio corresponding to the audio to be evaluated, and use it as template audio.

[0032] In one embodiment, obtaining multiple emission probabilities corresponding to audio frames in the audio to be evaluated includes:

[0033] If the template audio includes multiple template audios, a Gaussian mixture model is trained based on the multiple template audios;

[0034] The audio features corresponding to the audio to be evaluated are obtained and input into the Gaussian mixture model. The Gaussian mixture model determines the target template audio with the highest similarity based on the audio features of the audio to be evaluated, and outputs multiple emission probabilities corresponding to each audio frame based on the similarity between the audio features of each audio frame and the mid-audio features of the target template audio.

[0035] Secondly, this application provides an audio evaluation device, the device comprising:

[0036] The response module is used to obtain the audio to be evaluated, and to obtain the template audio and template audio text corresponding to the audio to be evaluated;

[0037] The first acquisition module is used to acquire multiple emission probabilities corresponding to audio frames in the audio to be evaluated; the emission probability represents the probability that the audio frame belongs to each pronunciation state in the template audio; the pronunciation state represents the pronunciation stage of the text to which the audio frame in the audio to be evaluated belongs;

[0038] The second acquisition module is used to acquire the first transition probability between the pronunciation states corresponding to each template character in the template audio text, and to acquire the second transition probability between the pronunciation states corresponding to each preset character in the preset audio text database; the preset audio text database contains the template audio text and the amount of text data in the preset audio text database is greater than the amount of text data in the template audio text.

[0039] The decoding module is used to determine a first decoding path based on the emission probability and the first transition probability, and to determine a second decoding path based on the emission probability and the second transition probability. Based on the comparison results between the template decoding path corresponding to the template audio and the first decoding path and the second decoding path, a first evaluation value and a second evaluation value are determined.

[0040] The evaluation module is used to determine the evaluation result of the audio to be evaluated based on the first evaluation value and the second evaluation value.

[0041] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0042] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0043] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0044] The aforementioned audio evaluation method, apparatus, computer equipment, storage medium, and computer program product, after recording, acquires multiple emission probabilities corresponding to audio frames in the audio to be evaluated, acquires the first transition probability between each template text in the template audio text corresponding to the audio to be evaluated, acquires the second transition probability between each preset text in a preset audio text database, determines a first decoding path based on the emission probabilities and the first transition probabilities, determines a second decoding path based on the emission probabilities and the second transition probabilities, and determines a first evaluation value and a second evaluation value based on the comparison results between the template decoding path corresponding to the template audio and the first and second decoding paths, respectively, and determines the evaluation result of the audio to be evaluated based on the first evaluation value and the second evaluation value. Compared with the traditional method of evaluating song audio through fixed multiple dimensions, this solution decodes and determines the first evaluation value and the second evaluation value based on the emission probability of the audio, the first transition probability corresponding to the audio in the template text, and the second transition probability corresponding to the audio in the preset audio text database, and determines the evaluation result based on the first evaluation value and the second evaluation value. This eliminates the need to consider the misjudgment problem caused by multiple-dimensional evaluation, thus improving the accuracy of audio evaluation. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating an audio evaluation method in one embodiment;

[0046] Figure 2 This is a flowchart illustrating the decoding steps in one embodiment;

[0047] Figure 3 This is a flowchart illustrating the audio evaluation method in another embodiment;

[0048] Figure 4 This is a structural block diagram of an audio evaluation device in one embodiment;

[0049] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] In one embodiment, such as Figure 1As shown, an audio evaluation method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server, including the following steps:

[0052] Step S202: Obtain the audio to be evaluated, and obtain the template audio and template audio text corresponding to the audio to be evaluated.

[0053] When using the terminal, users can record audio using software installed on the terminal. For example, if the audio is a song, the software could be a singing app, which the user can use to record the song audio. When the terminal detects that the audio recording is complete, or when it detects an audio evaluation command from the user, it can acquire the user-recorded audio as the audio to be evaluated. After acquiring the audio to be evaluated, the terminal can obtain the corresponding template audio and template audio text. For example, if the audio to be evaluated is a song, the terminal can obtain the corresponding template song as the template audio and the corresponding template lyrics as the template audio text. The template audio and template audio text can then be used to compare with the audio to be evaluated to determine the degree of conformity between the audio to be evaluated and the template audio.

[0054] There can be one or more template audio files. For example, in one embodiment, obtaining the template audio corresponding to the audio to be evaluated includes: obtaining the original audio and / or at least one cover audio corresponding to the audio to be evaluated as template audio. In this embodiment, the terminal can obtain multiple types of audio as template audio corresponding to the audio to be evaluated. For example, the terminal can obtain the original audio corresponding to the audio to be evaluated as template audio, the terminal can obtain at least one cover audio corresponding to the audio to be evaluated as template audio, or the terminal can obtain both the original audio and at least one cover audio as template audio, that is, there can be multiple template audio files corresponding to the audio to be evaluated. The cover audio can be a cover audio with an audio evaluation value higher than a certain evaluation value threshold, that is, each of the cover audio files can have a corresponding audio evaluation value. Specifically, taking a song as an example, to adapt to the diversity of singing and ensure that singers have sufficient room for performance, the terminal can obtain the original song and / or high-quality cover songs corresponding to the song to be evaluated as template songs, that is, each song has one or more templates, and each song's template corresponds to the same lyrics. The high-quality cover songs can be cover songs with an audio evaluation value greater than a preset evaluation value threshold.

[0055] Step S204: Obtain multiple emission probabilities corresponding to audio frames in the audio to be evaluated; emission probability represents the probability that an audio frame belongs to each pronunciation state in the template audio; pronunciation state represents the pronunciation stage of the text to which the audio frame in the audio to be evaluated belongs.

[0056] Among them, the audio to be evaluated can be composed of multiple audio frames, and each audio frame is combined in the chronological order of the user's speech to obtain the audio to be evaluated. Each audio frame in the audio to be evaluated has a corresponding emission probability. The terminal can obtain multiple emission probabilities corresponding to the audio frames in the audio to be evaluated. Among them, the emission probability represents the probability that each audio frame belongs to each pronunciation state in the template audio. A pronunciation state can be the pronunciation feature of a phoneme in one of its corresponding pronunciation stages, and the pronunciation state can be the pronunciation stage of the text to which the audio frame belongs. For example, the pronunciation state of the text to which the audio frame in the audio to be evaluated belongs. Among them, each of the above audio frames can be a part of a text pronunciation, that is, a stage of the text pronunciation. Multiple audio frames can form a phoneme corresponding to a text, and a text can be composed of multiple phonemes. For example, for the text "我 (wǒ)", its corresponding pronunciation is "wo", then "w" and "o" are respectively two phonemes of this text, and each phoneme has a corresponding pronunciation stage, including a pronunciation start stage, a pronunciation middle stage, and a pronunciation end stage. The above three pronunciation stages can be three states corresponding to the HMM (Hidden Markov Model). The pronunciation states of the pronunciation stages of the phonemes corresponding to each text can be not completely the same. For the above "w", it corresponds to three pronunciation stages, and each pronunciation stage can be reflected by one frame of the above audio. Then each audio frame in the above audio to be evaluated can have a pronunciation state.

[0057] Since the pronunciation states of the phonemes corresponding to each text can be different, there can be multiple pronunciation states in the candidate state corresponding to each audio frame in the above audio to be evaluated. The terminal can determine one of them as the pronunciation state corresponding to an audio frame from multiple pronunciation states in the candidate state. The terminal can obtain the similarity between each audio frame in the above audio to be evaluated and each pronunciation state in the template audio, and obtain the pronunciation state whose similarity is greater than the preset similarity threshold as the pronunciation state corresponding to each audio frame in the audio to be evaluated. And the terminal can also convert the above similarity into an emission probability. The emission probability can be positively correlated with the similarity of one frame of audio belonging to each pronunciation state. For example, it can be a 1:1 relationship.

[0058] The aforementioned emission probabilities can be obtained through a model. For example, in one embodiment, obtaining multiple emission probabilities corresponding to audio frames in the audio to be evaluated includes: obtaining audio features corresponding to the audio to be evaluated; inputting the audio features into a Gaussian Mixed Model (GMM) to obtain multiple emission probabilities corresponding to each audio frame output by the GMM; and training the GMM based on the articulation state of each template audio frame. In this embodiment, the terminal can obtain the audio features of the audio to be evaluated and input the audio features into a GMM. The GMM identifies the probability that each audio frame in the audio to be evaluated belongs to each HMM articulation state, thereby obtaining multiple emission probabilities corresponding to each audio frame. The aforementioned GMM can be trained based on the articulation state of each template audio frame. The terminal uses the template audio to train the GMM, so that the terminal can use the GMM to identify the similarity between the articulation state of each audio frame in the audio to be evaluated and each audio frame in the template audio. Specifically, the aforementioned audio features can be MFCC (Mel Frequency Cepstrum Coefficient), and the aforementioned audio to be evaluated can be a song. The terminal can extract the Mel Frequency Cepstrum Coefficient features from the song to be evaluated, input the aforementioned features into a Gaussian Mixture Model (HMM), and calculate the probability that each frame feature belongs to each vocalization state of the aforementioned HMM through the Gaussian Mixture Model, thereby obtaining multiple emission probabilities corresponding to each audio frame in the song to be evaluated.

[0059] Since there can be multiple template audios, the terminal can use a Gaussian mixture model trained from these template audios to identify the emission probabilities. For example, in one embodiment, obtaining multiple emission probabilities corresponding to audio frames in the audio to be evaluated includes: if there are multiple template audios, training a Gaussian mixture model based on these template audios; obtaining the audio features corresponding to the audio to be evaluated, inputting these features into the Gaussian mixture model, and having the Gaussian mixture model determine the target template audio with the highest similarity based on the audio features of the audio to be evaluated, and outputting multiple emission probabilities corresponding to each audio frame based on the similarity between the audio features of each audio frame and the audio features of the target template audio. In this embodiment, since the style of audio singing can be diverse, a user-recorded audio with a different style from the original audio does not necessarily mean the user is singing poorly. Therefore, the template audios can include the original audio and / or at least one cover audio. When there are multiple template audios, the terminal can train a Gaussian mixture model based on these template audios, and this Gaussian mixture model trained from the multiple template audios can determine which template audio best matches each audio frame in the audio to be evaluated.

[0060] When determining the emission probability, the terminal can obtain the audio features of the audio to be evaluated, such as the above-mentioned MFCC features, and input the audio features into the Gaussian mixture model trained by multiple template audios. The Gaussian mixture model determines the target template audio with the highest similarity based on the audio features of the audio to be evaluated, and outputs multiple emission probabilities corresponding to each audio frame based on the audio features of the target template audio. For example, the terminal calculates the similarity between the audio features of each audio frame of the audio to be evaluated and the audio features of each audio frame in each template audio through the Gaussian mixture model, and takes the template audio with the largest sum of similarities of multiple frames of audio features as the target template audio. Thus, the terminal can obtain the similarity between the audio features of each audio frame of the audio to be evaluated and the audio features of each frame in the target template audio, and obtain multiple emission probabilities corresponding to each audio frame in the audio to be evaluated. Specifically, the above-mentioned audio to be evaluated can be a song, and the audio features of the audio to be evaluated can be the audio features of the song to be evaluated. Then, the obtained emission probabilities can be multiple emission probabilities corresponding to each audio frame in the song to be evaluated. In addition, it should be noted that the above-mentioned Gaussian mixture model can also be replaced by other models, such as a DNN (Deep Neural Networks) model, etc.; the above-mentioned MFCC features can also be replaced by other acoustic features.

[0061] Step S206: Obtain the first transition probability between the pronunciation states corresponding to each template word in the template audio text, and obtain the second transition probability between the pronunciation states corresponding to each preset word in the preset audio text database; the preset audio text database includes the template audio text and the text data volume of the preset audio text database is greater than the text data volume of the template audio text.

[0062] Among them, the transition probability can be the probability that words can be transferred to each other. Since words are composed of phonemes, the transition probability between words includes the transition probability between phonemes. And phonemes can be composed of multiple pronunciation states, so the transition probability between phonemes can include the transition probability between pronunciation states. For example, in a text, there are two texts, "I and you" and "I love you". Since the texts adjacent to the word "I" are "love" and "and", the transition probability between "I" and "and" is 1 / 2, and the transition probability between "I" and "love" is 1 / 2. There is also a transition probability between the phonemes that make up the word. For example, for the word "I", its corresponding phonemes are "w" and "o", so there is a transition probability between the two phonemes "w" and "o". There is also a transition probability between the pronunciation states that make up the phoneme. For example, for the phoneme "w", there are three pronunciation states, and there is a transition probability between these three pronunciation states.

[0063] For the aforementioned audio to be evaluated, the terminal can determine the transition probability between the pronunciation states corresponding to each audio frame on a frame-by-frame basis. Alternatively, it can combine multiple audio frames to obtain the audio to be evaluated on a phoneme-by-phoneme basis and determine the transition probability between phonemes. It can also combine multiple phonemes to obtain the audio to be evaluated on a character-by-character basis and determine the transition probability between the audio corresponding to each character. The terminal can identify the pronunciation state of each audio frame in the audio to be evaluated and determine the transition probability between two frames based on the pronunciation states between each pair of frames. To this end, the terminal first needs to determine the transition probability between each type of character. The terminal can acquire two sets of transition probabilities: one based on template characters and one based on a preset audio text database. The preset audio text database contains template audio texts, and the amount of text data in the preset audio text database is greater than the amount of text data in the template audio texts. The preset audio text database can include multiple audio texts. The terminal can acquire the first transition probability between each template character in the aforementioned template audio text, and it can also acquire the second transition probability between each preset character in the aforementioned preset audio text database. Specifically, the terminal can extract each adjacent character from the aforementioned template audio text or preset audio text database, determine the proportion of identical adjacent characters among all characters, and thus determine the transition probability between each character. Based on the transition probabilities between these characters, the terminal can then obtain the corresponding phoneme transition probabilities and pronunciation state transition probabilities.

[0064] Step S208: Determine the first decoding path based on the emission probability and the first transition probability, and determine the second decoding path based on the emission probability and the second transition probability. Determine the first evaluation value and the second evaluation value based on the comparison results between the template decoding path corresponding to the template audio and the first decoding path and the second decoding path, respectively.

[0065] The terminal determines multiple emission probabilities for each audio frame in the audio to be evaluated, the first transition probabilities between template characters in the template audio text, and the second transition probabilities between preset characters in the preset audio text database. It can then decode the audio in two ways: restricted decoding based on the template characters and free decoding based on the preset audio text database. For restricted decoding, the terminal determines a first decoding path based on the emission probabilities and the first transition probabilities. For free decoding, the terminal determines a second decoding path based on the emission probabilities and the second transition probabilities. Each decoding path can include the pronunciation state at various moments in the audio. The terminal can then determine the transition probabilities between the pronunciation states corresponding to the characters based on these transition probabilities and perform decoding based on these transition probabilities.

[0066] Specifically, the terminal can determine multiple emission probabilities corresponding to each audio frame in the audio to be evaluated. Each emission probability represents a pronunciation state. The terminal can obtain the pronunciation states corresponding to each emission probability in two adjacent audio frames and determine the transition probabilities between each pair of pronunciation states. Thus, based on the emission probabilities and transition probabilities, the terminal determines the path of pronunciation states between these two frames. The terminal can decode the audio using the above decoding method based on the first transition probability corresponding to the template text and the second decoding path corresponding to the preset audio text database, respectively, to obtain two decoding paths. Each decoding path contains multiple pronunciation states corresponding to the audio to be evaluated arranged in chronological order.

[0067] The terminal can also decode the template audio using the same method to obtain the template decoding path corresponding to the template audio. The pronunciation state of each audio frame in the template audio can be known; therefore, the template decoding path can be a path composed of known pronunciation states arranged in the temporal order of the template audio. The terminal can compare the first decoding path with the template decoding path and obtain a first evaluation value based on the comparison result. It can also compare the second decoding path with the template decoding path and obtain a second evaluation value based on the comparison result. This evaluation value, also known as the likelihood value, represents the degree of conformity between the decoding path and the template decoding path; a higher degree of conformity results in a higher evaluation value.

[0068] Step S210: Determine the evaluation result of the audio to be evaluated based on the first evaluation value and the second evaluation value.

[0069] Here, the first evaluation value represents the degree of conformity between the first decoding path obtained by the terminal in decoding the audio to be evaluated and the template decoding path within the template audio text range; the second evaluation value represents the degree of conformity between the second decoding path obtained by the terminal in decoding the audio to be evaluated and the template decoding path within the preset audio text database range. After obtaining the first and second evaluation values, the terminal can determine the evaluation result of the audio to be evaluated based on the first and second evaluation values. Specifically, the terminal can determine the evaluation result of the audio to be evaluated based on the ratio of the first and second evaluation values.

[0070] The evaluation results mentioned above can also be referred to as the GOP (Goodness of Pronunciation) score. For restricted decoding, the decoding difficulty within a limited space is small, and the evaluation value obtained is larger than that of free decoding. The evaluation value of free decoding depends on the matching degree between the feature frames in the audio and each sub-Gaussian in the GMM model. Each sub-Gaussian represents a sub-Gaussian model trained for each audio text in the audio text database. The GMM model is trained using template songs. Therefore, if the singing audio is closer to a certain template in multiple dimensions such as pronunciation, technique, and emotion, the evaluation value output by free decoding will be larger. When the likelihood values ​​of free decoding and restricted decoding are closer, the ratio between the two is also larger. Therefore, the higher the GOP score, the higher the singing level of the user's audio to be evaluated.

[0071] In the aforementioned audio evaluation method, after recording, multiple emission probabilities corresponding to audio frames in the audio to be evaluated are obtained, along with the first transition probabilities between template characters in the template audio text corresponding to the audio to be evaluated, and the second transition probabilities between preset characters in a preset audio text database. A first decoding path is determined based on the emission probabilities and the first transition probabilities, and a second decoding path is determined based on the emission probabilities and the second transition probabilities. A first evaluation value and a second evaluation value are determined based on the comparison results between the template decoding path corresponding to the template audio and the first and second decoding paths, respectively. The evaluation result of the audio to be evaluated is then determined based on the first and second evaluation values. Compared to traditional methods that evaluate song audio using fixed multiple dimensions, this solution decodes and determines the first and second evaluation values ​​based on the emission probabilities of the audio, the first transition probabilities of the audio in the template text, and the second transition probabilities of the audio in the preset audio text database, respectively. The evaluation result is then determined based on the first and second evaluation values, eliminating the need to consider the misjudgment problem caused by multiple-dimensional evaluations and improving the accuracy of audio evaluation.

[0072] In one embodiment, obtaining a first transition probability between pronunciation states corresponding to each template character in a template audio text, and obtaining a second transition probability between pronunciation states corresponding to each preset character in a preset audio text database includes: obtaining a first phoneme sequence corresponding to each template character in the template audio text; the first phoneme sequence includes multiple first phonemes corresponding to each template character; determining the first transition probability between each pronunciation state in the template audio text according to the pronunciation states of each first phoneme in the first phoneme sequence and the pronunciation states of adjacent first phonemes of each first phoneme; obtaining a second phoneme sequence corresponding to each preset character in the preset audio text database; the second phoneme sequence includes multiple second phonemes corresponding to each preset character; determining the second transition probability between each pronunciation state in the preset audio text database according to the pronunciation states of each second phoneme in the second phoneme sequence and the pronunciation states of adjacent second phonemes of each second phoneme.

[0073] In this embodiment, a character can be composed of phonemes, and a phoneme can be composed of several pronunciation states. For example, for the character "我", its corresponding pronunciation is "wo", then "w" and "o" are respectively two phonemes of this character, and each phoneme has a corresponding pronunciation stage, including a pronunciation start stage, a pronunciation middle stage, and a pronunciation end stage. For the above "w", it corresponds to three pronunciation stages, and the pronunciation state of each pronunciation stage can be different according to the different phonemes. For the above transition probability, since the terminal can perform decoding based on each audio frame of the audio to be evaluated during decoding, and an audio frame can correspond to a pronunciation state, the terminal can determine the transition probability based on the pronunciation state. The terminal can obtain the transition probability between the corresponding pronunciation states within two text ranges.

[0074] Within the range of the template audio text, the terminal can obtain a first phoneme sequence corresponding to each character in the template audio text, where the first phoneme sequence includes multiple first phonemes corresponding to each template character. The terminal can determine the first transition probability between each pronunciation state in the template audio text according to the pronunciation states of each first phoneme in the first phoneme sequence and the pronunciation states of adjacent first phonemes of each first phoneme. Among them, the pronunciation states of the above template audio text can be arranged in chronological order of the audio text to form a pronunciation state sequence. The transition probability between pronunciation states includes the transition probability between pronunciation states within a character and the transition probability between pronunciation states of adjacent characters.

[0075] Within a preset audio text database, the terminal can obtain the second phoneme sequence corresponding to each preset character in the preset audio text database, wherein the second phoneme sequence includes multiple second phonemes corresponding to each preset character. The terminal can determine the second transition probability between each pronunciation state in the preset audio text database based on the pronunciation state of each second phoneme in the second phoneme sequence and the pronunciation states of adjacent second phonemes. The pronunciation states in the preset audio text database can be arranged in the temporal order of the audio text to form a pronunciation state sequence, and the transition probability between pronunciation states includes the transition probability between pronunciation states within a single character and the transition probability between pronunciation states of adjacent characters.

[0076] Specifically, since the aforementioned pronunciation states are arranged in chronological order, when determining the transition probability, the terminal can detect the next pronunciation state for each pronunciation state. The terminal can obtain all adjacent pronunciation states corresponding to the same pronunciation state. The proportion of a certain type of adjacent pronunciation state to all adjacent pronunciation states can be used as the transition probability of the same pronunciation state transitioning to that adjacent pronunciation state. Since the amount of data in the template audio text is less than the amount of data in the preset audio text database, the transition probability between pronunciation states within the template audio text will be closer to the pronunciation state changes in the template audio text. The transition probability between pronunciation states within the preset audio text database can be more diverse, allowing the terminal to evaluate the audio to be evaluated based on both of these transition probabilities.

[0077] Through this embodiment, the terminal can determine the transition probability based on two different text ranges with different data volumes, thereby enabling the terminal to evaluate the audio to be evaluated based on the two transition probabilities, which improves the accuracy of audio evaluation.

[0078] In one embodiment, determining a first decoding path based on the emission probability and a first transition probability, and determining a second decoding path based on the emission probability and a second transition probability, includes: for each audio frame, within the range of pronunciation states corresponding to the template text, obtaining the first transition probability of the audio frame transitioning to each pronunciation state of the next audio frame under each pronunciation state; determining the first frame probability value corresponding to the audio frame based on the weighted sum of the emission probability and the first transition probability corresponding to the audio frame; determining the first decoding path based on the first frame probability values ​​corresponding to each audio frame in the audio to be evaluated; the sum of the first frame probability values ​​corresponding to the audio frames in the first decoding path is maximized; for each audio frame, within the range of a preset audio text database, obtaining the second transition probability of the audio frame transitioning to each pronunciation state of the next audio frame under each pronunciation state; determining the second frame probability value corresponding to the audio frame based on the weighted sum of the emission probability and the second transition probability corresponding to the audio frame; determining the second decoding path based on the second frame probability values ​​corresponding to each audio frame in the audio to be evaluated; the sum of the second frame probability values ​​corresponding to the audio frames in the second decoding path is maximized.

[0079] In this embodiment, the terminal can decode each audio frame in the audio to be evaluated to determine the pronunciation state corresponding to each audio frame, and thus determine the degree of conformity between the audio to be evaluated and the template audio. The decoding can include two methods: decoding within the template audio text range and decoding within a preset audio text database range. Within the template audio text range, for each audio frame in the audio to be evaluated, the terminal can obtain multiple pronunciation states corresponding to that audio frame and obtain the first transition probability of that audio frame transitioning to the next audio frame under each pronunciation state. The audio frame has multiple pronunciation states, and the next audio frame can also have multiple pronunciation states. The terminal can arbitrarily combine the pronunciation states of two audio frames. Each pronunciation state corresponding to each audio frame in the audio to be evaluated has a corresponding emission probability, and the change in pronunciation states between adjacent frames has a corresponding transition probability. The terminal can determine the first frame probability value corresponding to the audio frame based on the weighted sum of the emission probability and the first transition probability. Specifically, both the emission probability and the transition probability have corresponding weights, which can be set according to actual conditions. The terminal can multiply the emission probability and the first transition probability by their respective weights and then sum them as the weighted sum. The terminal can obtain the first frame probability value for each audio frame in the audio to be evaluated, thus obtaining the first frame probability value corresponding to each audio frame. The terminal can determine a first decoding path based on the first frame probability values ​​corresponding to each audio frame in the audio to be evaluated, where the sum of the first frame probability values ​​of the audio frames in the first decoding path is the largest, meaning the pronunciation combination formed by the pronunciation states in the first decoding path is closest to the template audio text.

[0080] Similarly, the terminal can also perform the above decoding within a preset audio text database. For each audio frame in the audio to be evaluated, within the preset audio text database, the terminal can obtain the second transition probability of the audio frame transitioning to each pronunciation state of the next audio frame under each pronunciation state. There are multiple pronunciation states for the audio frame, and there can also be multiple pronunciation states for the next audio frame. The terminal can arbitrarily combine the pronunciation states of two audio frames. Each pronunciation state of each audio frame corresponds to a emission probability, and the change of pronunciation state between two adjacent audio frames has a corresponding transition probability. Thus, the terminal can obtain the weighted sum of the emission probability and the second transition probability corresponding to the audio frame to determine the second frame probability value corresponding to the audio frame. Specifically, the emission probability and the transition probability each correspond to a corresponding weight, which can be set according to the actual situation. The terminal can multiply the emission probability and the second transition probability by the weight respectively and then sum them as the weighted sum. The terminal can determine the second frame probability value for each audio frame in the audio to be evaluated, thereby obtaining the second frame probability value corresponding to each audio frame in the audio to be evaluated within the preset audio text database. Based on the obtained multiple second frame probability values, the terminal determines the second decoding path, so that the sum of the second frame probability values ​​corresponding to the audio frames in the second decoding path is maximized.

[0081] Specifically, a schematic diagram of the terminal decoding the audio to be evaluated can be shown as follows: Figure 2 As shown, Figure 2 This is a flowchart illustrating the decoding steps in one embodiment. Figure 2 Each origin point in the diagram represents a pronunciation state. Figure 2 The pronunciation states in the diagram include S0 to S4, where S0 and S4 are the starting and ending states, respectively, representing the beginning and end of a song. The lines connecting the pronunciation states on the vertical axis represent the transition probabilities between them. Figure 2 The transition probabilities in the middle include a 01 ~a 34 , Figure 2 The horizontal axis represents each frame of the audio to be evaluated, arranged chronologically. Here, seven frames are used as an example. Each audio frame can have multiple corresponding articulation states, each with a corresponding emission probability. Furthermore, changes in articulation states between adjacent frames have corresponding transition probabilities. The terminal determines the frame probability value for each audio frame by calculating a weighted sum of the emission and transition probabilities. The terminal can then perform the evaluation as follows: Figure 2 The method shown yields multiple decoding paths, each with a corresponding probability value. This probability value can be obtained by summing the frame probability values ​​of each audio frame. When the terminal decodes the audio to be evaluated, it finds the path with the highest probability value among the multiple decoding paths.

[0082] Taking a song as an example, the template audio text is the template lyrics, and the preset audio text database is the preset lyrics database. The terminal can perform two types of decoding on the audio to be evaluated. The first is restricted decoding within the template lyrics space. The terminal converts the lyrics into a phoneme sequence, and then converts the phoneme sequence into the state sequence of the HMM mentioned above, thereby obtaining the pronunciation state sequence corresponding to the template lyrics and determining the transition probability between each pronunciation state. Since the lyrics corresponding to the song are known, the terminal has less difficulty decoding within the limited template lyrics space, and the result is more reliable. After decoding, the terminal can obtain the first decoding path and the frame probability value of the pronunciation state to which each audio frame belongs in the first decoding path. The terminal can obtain the corresponding evaluation value based on the frame probability value. The second is free decoding within the preset lyrics database. Compared with the limited decoding space, when the terminal decodes based on the preset lyrics database, it does not tell the decoding model the lyrics corresponding to the song. It allows the audio frames to search the entire decoding space, that is, within the aforementioned preset lyrics database, to obtain the second decoding path with the highest conformity. After decoding, the terminal can obtain the frame probability value of the pronunciation state to which each audio frame belongs in the second decoding path. The terminal can obtain the corresponding evaluation value based on the frame probability value.

[0083] Through this embodiment, the terminal can determine two decoding paths based on two data volume decoding methods. Thus, the terminal can determine the evaluation value of each frame corresponding to the pronunciation state in the two decoding paths, and determine the evaluation value of the audio to be evaluated based on the evaluation values ​​of the two decoding paths. There is no need to consider the misjudgment problem caused by multiple dimensions of evaluation, which improves the accuracy of audio evaluation.

[0084] In one embodiment, determining a first evaluation value and a second evaluation value based on comparisons between the template decoding path corresponding to the template audio and a first decoding path and a second decoding path, respectively, includes: determining a first pronunciation state sequence corresponding to the audio to be evaluated based on the first decoding path; determining a second pronunciation state sequence corresponding to the audio to be evaluated based on the second decoding path; determining a template pronunciation state sequence corresponding to the template audio based on the template decoding path corresponding to the template audio; determining a first evaluation value based on the similarity between the template pronunciation state sequence and the first pronunciation state sequence; and determining a second evaluation value based on the similarity between the template pronunciation state sequence and the second pronunciation state sequence.

[0085] In this embodiment, after determining the first decoding path and the second decoding path through the above two decoding methods, the terminal can obtain the pronunciation state of each audio frame in the audio to be evaluated from each decoding path. That is, both the first decoding path and the second decoding path can be a sequence of pronunciation states. The terminal can determine the first pronunciation state sequence of the audio to be evaluated based on the first decoding path, and determine the second pronunciation state sequence corresponding to the audio to be evaluated based on the second decoding path. The terminal can also obtain the template decoding path corresponding to the template audio. The template decoding path can be a path obtained based on the pronunciation state of each frame in the template audio. That is, the terminal can determine the template pronunciation state sequence corresponding to the template audio based on the template decoding path. The terminal can compare the similarity of the template pronunciation state sequence with the first pronunciation state sequence and the second pronunciation state sequence, for example, by comparing the pronunciation state of each frame in the sequence. Thus, the terminal can determine the first evaluation value based on the similarity between the template pronunciation state sequence and the first pronunciation state sequence, and determine the second evaluation value based on the similarity between the template pronunciation state sequence and the second pronunciation state sequence. Among them, the greater the similarity, the greater the evaluation value. For example, the similarity and the evaluation value can be in a 1:1 relationship.

[0086] Through this embodiment, the terminal can compare the pronunciation status of different decoding paths with that of the template decoding path to determine the evaluation value of each decoding path. Thus, the terminal can determine the evaluation result of the audio to be evaluated based on multiple evaluation values, thereby improving the accuracy of audio evaluation.

[0087] In one embodiment, determining the evaluation result of the audio to be evaluated based on a first evaluation value and a second evaluation value includes: obtaining the ratio of the first evaluation value to the second evaluation value, and determining the evaluation result of the audio to be evaluated based on the logarithm of the ratio.

[0088] In this embodiment, the terminal can determine a first evaluation value based on a comparison between the first decoding path and the template decoding path, and determine a second evaluation value based on a comparison between the second decoding path and the template decoding path. After determining the first and second evaluation values, the terminal can determine the evaluation result of the audio to be evaluated based on the first and second evaluation values. For example, the terminal can obtain the ratio of the first and second evaluation values, and determine the evaluation result of the audio to be evaluated based on the logarithm of the ratio. Specifically, the terminal can compare the first evaluation value obtained by restricted decoding with the second evaluation value obtained by free decoding, obtain the ratio, take the logarithm of the ratio, and then take the negative value of the logarithm to obtain the evaluation result of the audio to be evaluated, i.e., the GOP score. Among them, the restricted decoding can be a Viterbi decoding method. For restricted decoding, the decoding difficulty within a limited space is small, and the evaluation value obtained is larger than that of free decoding. The evaluation value of free decoding depends on the matching degree of each frame in the audio with each sub-Gaussian in the GMM model. The GMM model is trained using template songs, including the template song corresponding to the audio to be evaluated and the template song corresponding to the template lyrics in the preset lyrics database. Therefore, if the singing audio is closer to a certain template in multiple dimensions such as pronunciation, technique, and emotion, the higher the evaluation value output by free decoding, and the closer the evaluation values ​​of free decoding and restricted decoding are, the larger the ratio between the two, and thus the higher the GOP score.

[0089] In this embodiment, the terminal can determine the score of the audio to be evaluated based on the ratio of the evaluation values ​​obtained by two different decoding methods, without having to consider the misjudgment problem caused by multi-dimensional evaluation, thus improving the accuracy of audio evaluation.

[0090] In one embodiment, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating an audio evaluation method in another embodiment. In this embodiment, the audio is a song, as shown below. Figure 3 As shown, when evaluating audio, the terminal can first collect templates. To accommodate the diversity of singing styles, the terminal can select original songs and high-quality cover songs as templates. The terminal can obtain template songs corresponding to multiple songs, with each song having multiple template songs, and each song's templates corresponding to the same lyrics. The terminal can extract audio features from these songs, specifically extracting Mel-frequency cepstral coefficient features from the song audio, inputting the spectral features into a Gaussian Mixture Model (GMM), converting the lyrics into a phoneme sequence, and then converting the phoneme sequence into an HMM (Hypergraph) sequence of articulation states to determine the transition probabilities of each articulation state in the lyrics. The terminal can calculate the emission probability of each frame's features using the Gaussian Mixture Model, i.e., the probability that each frame belongs to each articulation state. The terminal can then use the above... Figure 2The method shown decodes the audio to be evaluated, including determining the pronunciation state of each frame of the audio. Decoding includes restricted decoding within the template lyrics range and free decoding within a preset lyrics database, which can be a lyrics dictionary. The decoded graph obtained on the terminal based on the lyrics dictionary and template lyrics can be as follows: Figure 2 As shown.

[0091] Specifically, the aforementioned restricted decoding can be a Viterbi decoding method. In restricted decoding, the terminal can convert lyrics into a phoneme sequence, and then convert the phoneme sequence into a Hidden Markov Model (HMM) state sequence. Each phoneme corresponds to three HMM state units, representing the states corresponding to the start, middle, and end stages of pronunciation. Knowing the template lyrics corresponding to the song to be evaluated, decoding within a limited space is less difficult and the results are more reliable. After decoding, the evaluation value of the pronunciation state of each frame is saved. Similarly, the terminal can perform free decoding on the aforementioned HMM state sequence. Compared to the limited decoding space, the lyrics corresponding to the song audio are not given to the recognition model. The audio frames are searched throughout the entire decoding space, i.e., the aforementioned preset lyrics database. After decoding, the evaluation value of the pronunciation state of each frame is saved. These evaluation values ​​are also called likelihood values. After obtaining the likelihood values ​​corresponding to the two decoding methods, the terminal can obtain the ratio of the two likelihood values, obtain the logarithm of the ratio, and then take the negative number to obtain the aforementioned GOP score, which serves as the evaluation result for the audio to be evaluated.

[0092] Through the above embodiments, the terminal decodes and determines the first evaluation value and the second evaluation value based on the transmission probability of the audio, the first transition probability of the audio in the template text, and the second transition probability of the audio in the preset audio text database, respectively. The evaluation result is determined based on the first evaluation value and the second evaluation value. There is no need to consider the misjudgment problem caused by multiple evaluation dimensions, which improves the accuracy of audio evaluation.

[0093] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0094] Based on the same inventive concept, this application also provides an audio evaluation apparatus for implementing the audio evaluation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more audio evaluation apparatus embodiments provided below can be found in the limitations of the audio evaluation method described above, and will not be repeated here.

[0095] In one embodiment, such as Figure 4 As shown, an audio evaluation device is provided, including: a response module 500, a first acquisition module 502, a second acquisition module 504, a decoding module 506, and an evaluation module 508, wherein:

[0096] The response module 500 is used to obtain the audio to be evaluated and the corresponding template audio and template audio text.

[0097] The first acquisition module 502 is used to acquire multiple emission probabilities corresponding to audio frames in the audio to be evaluated; the emission probability represents the probability that the audio frame belongs to each pronunciation state in the template audio; the pronunciation state represents the pronunciation stage of the text to which the audio frame in the audio to be evaluated belongs.

[0098] The second acquisition module 504 is used to acquire the first transition probability between the pronunciation states of each template text in the template audio text, and to acquire the second transition probability between the pronunciation states of each preset text in the preset audio text database; the preset audio text database contains template audio text and the amount of text data in the preset audio text database is greater than the amount of text data in the template audio text.

[0099] The decoding module 506 is used to determine a first decoding path based on the emission probability and the first transition probability, and to determine a second decoding path based on the emission probability and the second transition probability. Based on the comparison results between the template decoding path corresponding to the template audio and the first decoding path and the second decoding path, a first evaluation value and a second evaluation value are determined. The decoding path includes the pronunciation state at each moment in the audio.

[0100] Evaluation module 508 is used to determine the evaluation result of the audio to be evaluated based on the first evaluation value and the second evaluation value.

[0101] In one embodiment, the first acquisition module 502 is specifically used to acquire the audio features corresponding to the audio to be evaluated; input the audio features into the Gaussian mixture model to acquire multiple emission probabilities corresponding to each audio frame output by the Gaussian mixture model; the Gaussian mixture model is trained based on the pronunciation state to which each frame of template audio belongs.

[0102] In one embodiment, the second acquisition module 504 is specifically used to acquire a first phoneme sequence corresponding to each template character in the template audio text; the first phoneme sequence includes multiple first phonemes corresponding to each template character; determine a first transition probability between each pronunciation state in the template audio text based on the pronunciation state of each first phoneme in the first phoneme sequence and the pronunciation state of each adjacent first phoneme; acquire a second phoneme sequence corresponding to each preset character in a preset audio text database; the second phoneme sequence includes multiple second phonemes corresponding to each preset character; determine a second transition probability between each pronunciation state in the preset audio text database based on the pronunciation state of each second phoneme in the second phoneme sequence and the pronunciation state of each adjacent second phoneme.

[0103] In one embodiment, the decoding module 506 is specifically configured to, for each audio frame, within the range of pronunciation states corresponding to the template text, obtain a first transition probability of the audio frame transitioning to each pronunciation state of the next audio frame under each pronunciation state; determine a first frame probability value corresponding to the audio frame based on the weighted sum of the emission probability corresponding to the audio frame and the first transition probability; determine a first decoding path based on the first frame probability values ​​corresponding to each audio frame in the audio to be evaluated; the sum of the first frame probability values ​​corresponding to the audio frames in the first decoding path is maximized; for each audio frame, within a preset audio text database range, obtain a second transition probability of the audio frame transitioning to each pronunciation state of the next audio frame under each pronunciation state; determine a second frame probability value corresponding to the audio frame based on the weighted sum of the emission probability corresponding to the audio frame and the second transition probability; determine a second decoding path based on the second frame probability values ​​corresponding to each audio frame in the audio to be evaluated; the sum of the second frame probability values ​​corresponding to the audio frames in the second decoding path is maximized.

[0104] In one embodiment, the decoding module 506 is specifically configured to: determine a first pronunciation state sequence corresponding to the audio to be evaluated based on a first decoding path; determine a second pronunciation state sequence corresponding to the audio to be evaluated based on a second decoding path; determine a template pronunciation state sequence corresponding to the template audio based on a template decoding path; determine a first evaluation value based on the similarity between the template pronunciation state sequence and the first pronunciation state sequence; and determine a second evaluation value based on the similarity between the template pronunciation state sequence and the second pronunciation state sequence.

[0105] In one embodiment, the evaluation module 508 is specifically used to obtain the ratio of the first evaluation value to the second evaluation value, and to determine the evaluation result of the audio to be evaluated based on the logarithm of the ratio.

[0106] In one embodiment, the response module 500 is specifically used to obtain the original audio and / or at least one cover audio corresponding to the audio to be evaluated, as template audio.

[0107] In one embodiment, the first acquisition module 502 is specifically used to: if there are multiple template audios, train a Gaussian mixture model based on the multiple template audios; acquire the audio features corresponding to the audio to be evaluated, and input the audio features into the Gaussian mixture model; the Gaussian mixture model determines the target template audio with the highest similarity based on the audio features of the audio to be evaluated, and outputs multiple emission probabilities corresponding to each audio frame based on the similarity between the audio features of each audio frame and the mid-audio features of the target template audio.

[0108] Each module in the aforementioned audio evaluation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0109] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an audio evaluation method. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0110] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0111] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the audio evaluation method described above.

[0112] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the audio evaluation method described above.

[0113] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the audio evaluation method described above.

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An audio evaluation method, characterized in that, The method includes: Obtain the audio to be evaluated, and obtain the template audio and template audio text corresponding to the audio to be evaluated; including: obtaining the original audio and / or at least one cover audio corresponding to the audio to be evaluated, as the template audio; the audio evaluation value of the cover audio is greater than a preset evaluation value threshold; Obtain multiple emission probabilities corresponding to audio frames in the audio to be evaluated; the emission probability represents the probability that the audio frame belongs to each pronunciation state in the template audio; the pronunciation state represents the pronunciation stage of the text to which the audio frame in the audio to be evaluated belongs; Obtain the first transition probability between the pronunciation states corresponding to each template character in the template audio text, and obtain the second transition probability between the pronunciation states corresponding to each preset character in the preset audio text database; the preset audio text database contains the template audio text and the amount of text data in the preset audio text database is greater than the amount of text data in the template audio text. A first decoding path is determined based on the emission probability and the first transition probability, and a second decoding path is determined based on the emission probability and the second transition probability. A first evaluation value and a second evaluation value are determined based on the comparison results between the template decoding path corresponding to the template audio and the first and second decoding paths, respectively. The first evaluation value represents a first degree of conformity between the template decoding path and the first decoding path, and the first evaluation value is positively correlated with the first degree of conformity. The second evaluation value represents a second degree of conformity between the template decoding path and the second decoding path, and the second evaluation value is positively correlated with the second degree of conformity. The evaluation result of the audio to be evaluated is determined based on the first evaluation value and the second evaluation value, including: obtaining the ratio of the first evaluation value to the second evaluation value, and determining the evaluation result of the audio to be evaluated based on the logarithm of the ratio; the larger the ratio, the more the audio to be evaluated matches the template audio in multiple singing dimensions.

2. The method according to claim 1, characterized in that, The step of obtaining multiple emission probabilities corresponding to audio frames in the audio to be evaluated includes: Obtain the audio features corresponding to the audio to be evaluated; The audio features are input into a Gaussian mixture model to obtain multiple emission probabilities corresponding to each audio frame output by the Gaussian mixture model; the Gaussian mixture model is trained based on the pronunciation state to which each frame of template audio belongs.

3. The method according to claim 1, characterized in that, The step of obtaining the first transition probability between the pronunciation states corresponding to each template character in the template audio text and obtaining the second transition probability between the pronunciation states corresponding to each preset character in the preset audio text database includes: Obtain the first phoneme sequence corresponding to each template character in the template audio text; the first phoneme sequence includes multiple first phonemes corresponding to each template character. Based on the pronunciation state of each first phoneme in the first phoneme sequence and the pronunciation state of each first phoneme's adjacent first phonemes, the first transition probability between each pronunciation state in the template audio text is determined. Obtain the second phoneme sequence corresponding to each preset character in the preset audio text database; the second phoneme sequence includes multiple second phonemes corresponding to each preset character. Based on the pronunciation state of each second phoneme in the second phoneme sequence and the pronunciation state of each second phoneme's adjacent second phonemes, the second transition probability between each pronunciation state in the preset audio text database is determined.

4. The method according to claim 1, characterized in that, The step of determining a first decoding path based on the transmission probability and the first transition probability, and determining a second decoding path based on the transmission probability and the second transition probability, includes: For each audio frame, within the range of pronunciation states corresponding to the template text, the first transition probability of the audio frame to the next audio frame in each pronunciation state is obtained, and the first frame probability value corresponding to the audio frame is determined based on the weighted sum of the emission probability corresponding to the audio frame and the first transition probability. A first decoding path is determined based on the first frame probability value corresponding to each audio frame in the audio to be evaluated; the sum of the first frame probability values ​​corresponding to the audio frames in the first decoding path is the largest. For each audio frame, within the preset audio text database, the second transition probability of the audio frame to the next audio frame in each pronunciation state is obtained, and the second frame probability value corresponding to the audio frame is determined based on the weighted sum of the emission probability corresponding to the audio frame and the second transition probability. A second decoding path is determined based on the second frame probability value corresponding to each audio frame in the audio to be evaluated; the sum of the second frame probability values ​​corresponding to the audio frames in the second decoding path is maximized.

5. The method according to claim 1, characterized in that, The step of determining the first evaluation value and the second evaluation value based on the comparison results between the template decoding path corresponding to the template audio and the first decoding path and the second decoding path, respectively, includes: A first pronunciation state sequence corresponding to the audio to be evaluated is determined according to the first decoding path, and a second pronunciation state sequence corresponding to the audio to be evaluated is determined according to the second decoding path. Based on the template decoding path corresponding to the template audio, determine the template pronunciation state sequence corresponding to the template audio; A first evaluation value is determined based on the similarity between the template pronunciation state sequence and the first pronunciation state sequence; and a second evaluation value is determined based on the similarity between the template pronunciation state sequence and the second pronunciation state sequence.

6. The method according to claim 1, characterized in that, The step of obtaining multiple emission probabilities corresponding to audio frames in the audio to be evaluated includes: If the template audio includes multiple template audios, a Gaussian mixture model is trained based on the multiple template audios; The audio features corresponding to the audio to be evaluated are obtained and input into the Gaussian mixture model. The Gaussian mixture model determines the target template audio with the highest similarity based on the audio features of the audio to be evaluated, and outputs multiple emission probabilities corresponding to each audio frame based on the similarity between the audio features of each audio frame and the mid-audio features of the target template audio.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images