Music evaluation method and device, electronic equipment and storage medium

Abstract

The present application provides a kind of music evaluation method, device, electronic equipment and storage medium, wherein the method comprises: determining the singing audio to be evaluated;The pitch of the singing audio is extracted to obtain pitch auxiliary information, and the pitch auxiliary information is used to reflect the distribution of the pitch of the singing audio on the sound level;The acoustic characteristics of the singing audio are encoded to obtain music coding features, and the music coding features are used to reflect various music properties of the singing audio;Based on the music coding features and the pitch auxiliary information, the music evaluation is carried out to obtain the music score of the singing audio, which can effectively improve the credibility and accuracy of the music evaluation process, reduce the dependence on the content of the song, improve the objectivity of the score, overcome the defects of low credibility, accuracy and objectivity in the traditional scheme, poor evaluation effect and poor evaluation stability, so that the music score is more robust, the evaluation effect is better and the stability is stronger.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a music evaluation method, apparatus, electronic device, and storage medium. Background Technology

[0002] With economic development and technological progress, people are paying more and more attention to art. Consequently, the evaluation of art is becoming more and more rigorous. As the main branch of vocal art, music requires a very high level of control over sound and timing, and its evaluation process is also quite complicated.

[0003] Most current music evaluation schemes rely on manually designed scoring features. Models can evaluate music based on these features to obtain music scores. However, when conducting music evaluations, there are often differences between manually designed scoring features and the key scoring points perceived by humans. This inconsistency between scoring features and key scoring points leads to significant differences between machine scores and expert scores, resulting in low correlation between the two and low credibility and accuracy of music evaluations, leading to poor evaluation results. Summary of the Invention

[0004] This invention provides a music evaluation method, apparatus, electronic device, and storage medium to address the shortcomings of existing technologies, such as low credibility, accuracy, and objectivity of scores, poor evaluation results, and poor evaluation stability, thereby improving evaluation results and stability.

[0005] This invention provides a music evaluation method, comprising:

[0006] Identify the singing audio to be evaluated;

[0007] Pitch is extracted from the singing audio to obtain pitch auxiliary information, which is used to reflect the distribution of pitch in the singing audio across the pitch levels.

[0008] The acoustic features of the singing audio are encoded to obtain music encoding features, which are used to reflect various musical attributes of the singing audio;

[0009] Based on the music encoding features and the pitch auxiliary information, a music evaluation is performed to obtain a music score for the singing audio.

[0010] According to a music evaluation method provided by the present invention, the music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio includes:

[0011] Based on the music encoding features, the song category of the singing audio is predicted to obtain the song category probability distribution of the singing audio;

[0012] Based on the probability distribution of the song category, the music encoding features, and the pitch auxiliary information, a music evaluation is performed to obtain a music score for the performance audio.

[0013] According to a music evaluation method provided by the present invention, the music evaluation based on the probability distribution of the song category, the music encoding features, and the pitch auxiliary information to obtain a music score for the singing audio includes:

[0014] Based on the probability distribution of the song category, the song category features of the singing audio are determined, and based on the pitch auxiliary information, the pitch distribution features of the singing audio in terms of pitch levels are determined.

[0015] The pitch distribution features, the song category features, and the music encoding features are fused to obtain the fused features of the singing audio.

[0016] Based on the fusion features, a music evaluation is performed to obtain a music score for the singing audio.

[0017] The music score includes at least one of the following: music performance score, music fluency score, music pitch score, music rhythm score, music lyrics score, and overall music score.

[0018] According to a music evaluation method provided by the present invention, the step of extracting pitch from the singing audio to obtain pitch auxiliary information includes:

[0019] Pitch information of the singing audio is obtained by extracting the pitch of the singing audio.

[0020] The pitch information is extracted to determine the pitch information of the singing audio.

[0021] The pitch auxiliary information is obtained by counting the number of pitch cents corresponding to each note in the sung audio.

[0022] According to a music evaluation method provided by the present invention, the step of encoding the acoustic features of the singing audio to obtain music encoding features includes:

[0023] Musical attribute extraction is performed on the acoustic features to obtain various musical attribute features;

[0024] The various music attribute features are encoded to obtain initial music encoding features;

[0025] The initial music coding features are aligned to obtain the music coding features.

[0026] According to a music evaluation method provided by the present invention, the music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio includes:

[0027] The music encoding features and the pitch auxiliary information are input into the music evaluation model to obtain the music score of the singing audio output by the music evaluation model.

[0028] The music evaluation model is trained based on the scoring error of each sample audio. The scoring error is the difference between the sample music score and the predicted music score of the corresponding sample audio. The predicted music score is determined by the music evaluation model based on the corresponding sample audio.

[0029] According to a music evaluation method provided by the present invention, the music evaluation model is trained based on the following steps:

[0030] Based on the initial music evaluation model, the predicted music score for each sample audio is determined;

[0031] The mean square loss is determined based on the scoring error of each audio sample and the number of audio samples.

[0032] Based on the scoring errors of different sample audio, determine the score difference consistency loss;

[0033] Based on the mean square loss and the difference consistency loss, the initial music evaluation model is iterated to obtain the music evaluation model.

[0034] The present invention also provides a music evaluation device, comprising:

[0035] The audio determination unit is used to determine the singing audio to be evaluated;

[0036] The pitch extraction unit is used to extract the pitch of the singing audio to obtain pitch auxiliary information, which is used to reflect the distribution of the pitch of the singing audio in the pitch level.

[0037] The feature encoding unit is used to encode the acoustic features of the singing audio to obtain music encoding features, which are used to reflect various musical attributes of the singing audio.

[0038] The music evaluation unit is used to perform music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio.

[0039] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the music evaluation method as described above.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the music evaluation method as described above.

[0041] The music evaluation method, apparatus, electronic device, and storage medium provided by this invention combine pitch-aided information reflecting the distribution of pitch in the basic pitch levels of the sung audio with music coding features characterizing various musical attributes of the sung audio to perform music evaluation and obtain a music score for the sung audio. This not only effectively improves the credibility and accuracy of the music evaluation process but also reduces reliance on song content and enhances the objectivity of the score. It overcomes the shortcomings of traditional schemes, such as low credibility, accuracy, and objectivity, poor evaluation results, and poor evaluation stability, making music scoring more robust, with better evaluation results and stronger stability. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating the music evaluation method provided by the present invention;

[0044] Figure 2 This is a flowchart illustrating step 140 in the music evaluation method provided by the present invention;

[0045] Figure 3 This is a flowchart illustrating step 142 in the music evaluation method provided by the present invention;

[0046] Figure 4 This is a flowchart illustrating step 120 in the music evaluation method provided by the present invention;

[0047] Figure 5 This is a flowchart illustrating step 130 in the music evaluation method provided by the present invention;

[0048] Figure 6 This is a framework example diagram of the process for determining music encoding features provided by the present invention;

[0049] Figure 7 This is a flowchart illustrating the training process of the music evaluation model provided by the present invention;

[0050] Figure 8 This is a general framework diagram of the music evaluation method provided by the present invention;

[0051] Figure 9 This is a schematic diagram of the structure of the music evaluation device provided by the present invention;

[0052] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] Currently, music evaluation schemes typically begin by extracting acoustic features from the performance audio using cepstral mean normalization (CMN). These features are then fed into an acoustic model for forward computation to obtain scores for each state corresponding to each frame. Next, the vowel boundaries and lyrics are calculated based on these scores. Subsequently, the corresponding song resources (score, lyrics, etc.) are acquired, and scoring features are extracted using the above results. Finally, scoring feature regression is employed to derive the final music score.

[0055] However, the music evaluation process based on the above scheme still has many shortcomings, namely:

[0056] First, the set rating features are inconsistent with the key rating points perceived by humans. The difference between the two will result in a low correlation between machine ratings and expert ratings, and a low credibility of the evaluation.

[0057] For example, when reading lyrics, the machine scores are high because the posterior probability of speech recognition is used as the scoring feature, while the actual expert scores are often low. In addition, the recognition performance of speech recognition models based on HMM-GMM is poor, which also leads to poor evaluation results.

[0058] Secondly, music evaluation requires obtaining the song resources corresponding to the singing audio. This not only leads to music scoring being overly dependent on the song content, making the evaluation process highly subjective and the scoring less objective, but also limits its application scope, narrows its working scenarios, and reduces its universality.

[0059] Pitch features are a key piece of information to focus on during music evaluation, as they can, to a certain extent, indicate the quality of a song's performance. However, most current music evaluations do not consider or apply this feature, resulting in low credibility and accuracy of music evaluation scores, low objectivity, and poor evaluation stability.

[0060] In view of this, in order to overcome the shortcomings of traditional methods, such as low credibility, accuracy, objectivity, poor evaluation effect, and poor evaluation stability, this invention provides a music evaluation method. It aims to use pitch auxiliary information, which can characterize the distribution of pitch of the sung audio on the basic pitch level, to conduct music evaluation. This method can not only effectively improve the credibility and accuracy of the music evaluation process, but also reduce the dependence on the song content, improve the objectivity of the score, and make the music scoring more robust, with better evaluation effect and stronger stability. Figure 1 This is a flowchart illustrating the music evaluation method provided by the present invention, as shown below. Figure 1 As shown, the method includes:

[0061] Step 110: Determine the singing audio to be evaluated;

[0062] Specifically, before conducting music evaluation, it is necessary to first determine the evaluation object, namely the singing audio to be evaluated. Here, the singing audio to be evaluated can be the audio collected in real time by the voice acquisition device during the singing process, or it can be a segment of audio extracted from the real-time audio, or it can be the entire segment of audio or a segment of audio selected from the recorded and pre-stored audio. This embodiment of the invention does not make specific limitations in this regard.

[0063] The audio recording contains the content of the song being sung. The song can be pre-set, meaning that music evaluation can be performed on the audio recording of a specific song, or it can be randomly selected, meaning that music evaluation can be performed on randomly recorded / selected / trimmed audio recordings.

[0064] In addition, the songs performed can be of any style and type, such as classic songs, rock songs, etc. In short, the style of the songs performed is arbitrary.

[0065] Here, the singing audio can be one segment or multiple segments. If the singing audio is in multiple segments, music evaluation needs to be performed on each segment to determine its audio score in multiple dimensions. These multiple dimensions can include performance dimension, fluency dimension, pitch dimension, rhythm dimension, lyrics dimension, and comprehensive dimension.

[0066] Step 120: Extract pitch from the singing audio to obtain pitch auxiliary information. The pitch auxiliary information is used to reflect the distribution of pitch in the singing audio across the pitch levels.

[0067] Considering that pitch features are a key feature to focus on during music evaluation, as they can reflect the quality of the song being sung, this embodiment of the invention can utilize pitch auxiliary information in the singing audio that reflects pitch features to conduct music evaluation, thereby effectively improving the evaluation effect of the music evaluation process.

[0068] Specifically, after determining the singing audio to be evaluated, step 120 can be executed to extract the pitch of the singing audio to obtain pitch auxiliary information. This process specifically includes:

[0069] First, pitch extraction can be performed on the singing audio to extract the pitch information contained therein. That is, pitch extraction is performed based on the singing audio to obtain the pitch information of the singing audio. The pitch information here represents the height of the sound in the singing audio, or the pitch information, and can also be understood as the information reflecting the audio amplitude of the singing audio.

[0070] Randomly, based on pitch information and guided by music expertise, pitch auxiliary information can be calculated. Specifically, the pitch information of the singing audio can be processed to obtain the cent information of the singing audio. A cent is one-hundredth of a semitone. In music theory, in order to improve the accuracy of pitch measurement, each semitone is divided into 100 cents to calculate the error rate.

[0071] After that, the pitch information can be statistically analyzed to obtain pitch auxiliary information; the pitch auxiliary information here can be understood as the distribution of the singing song on the basic pitch, which can reflect the distribution of the pitch of the singing audio on different basic pitches.

[0072] For example, when the audio corresponds to the song "Two Tigers", there are more do and re notes and fewer other notes. In this case, the pitch auxiliary information of the audio reflects the fact that the song has more do and re notes.

[0073] Step 130: Encode the acoustic features of the singing audio to obtain music coding features, which are used to reflect various musical attributes of the singing audio.

[0074] Specifically, in step 110, after determining the singing audio to be evaluated, step 130 can be executed to encode the acoustic features of the singing audio to obtain music coding features that can reflect the musical attributes of the singing audio. The specific process includes the following steps:

[0075] First, feature extraction can be performed on the singing audio to extract the acoustic characteristics of the speech contained therein, thereby obtaining the acoustic features of the singing audio. The acoustic features here can be Filter Bank features, MFCC (Mel-Frequency Cepstral Coefficeits) features, or other acoustic features. This embodiment of the invention does not specifically limit these features.

[0076] Subsequently, the acoustic features of the singing audio are encoded to obtain its music encoding features. These music encoding features can reflect various musical attributes of the singing audio. Specifically, this process can involve using a convolutional neural network to process the acoustic features of the singing audio to extract various musical attributes contained therein, such as rhythm, melody, pitch, and intonation, thereby obtaining various musical attribute features. Then, these various musical attribute features can be encoded into a fusion feature, thus obtaining music encoding features that can reflect various musical attributes of the singing audio.

[0077] It is worth noting that, in order to ensure the applicability of the music encoding features, the music encoding features output by the convolutional neural network can also be length aligned in this embodiment of the invention. That is, the music encoding features can be mapped to a fixed length through a recurrent neural network, so that the music encoding features can be applied to different scenarios and solve the dilemma of limited application scenarios in traditional solutions.

[0078] Here, recurrent neural networks can be used to ensure that singing audio of different durations has the same length of music coding features. That is, input features of different lengths can be mapped to the same length, so that the length of the music coding features of singing audio of different durations is the same, thus ensuring the universality of the music evaluation scheme.

[0079] Step 140: Based on music coding features and pitch auxiliary information, perform music evaluation to obtain a music score for the performance audio.

[0080] Specifically, after obtaining the pitch auxiliary information and music coding features of the singing audio through the above steps, step 140 can be executed. Based on this pitch auxiliary information and music coding features, music evaluation is performed to obtain a music score for the singing audio. The specific process includes the following steps:

[0081] Since pitch auxiliary information is represented as pitch values ​​broken by 12 semitones within an octave, and pitch is an important part of auditory perception, it is crucial to the final music score. For example, the pitch auxiliary information of a song corresponding to a low-scoring singing audio often shows a scattered distribution of pitch values, reflecting that the singer has difficulty accurately singing the main notes of the song. Conversely, the pitch auxiliary information of a song corresponding to a high-scoring singing audio usually shows sharp and narrow pitch peaks, reflecting that the main notes of the song are ready to be sung, that is, the singer's singing is very well in sync.

[0082] Therefore, in this embodiment of the invention, the various music attribute information represented by pitch auxiliary information and music coding features are fused to obtain comprehensive fused information, and music evaluation is carried out with the help of the fused information. The specific process may be as follows: First, the pitch auxiliary information reflecting the distribution of pitch on the pitch level is used to determine the pitch distribution features of the singing audio on the pitch level. Then, the pitch distribution features and music coding features are fused to obtain fused features. The fused features can reflect the fused information of the singing audio from a global perspective.

[0083] It should be noted that this section addresses the feature fusion of music encoding features and pitch distribution features. Music encoding features and pitch distribution features can complement each other; the former can compensate for missing music attribute information in the latter, while the latter can add detailed pitch distribution information from the former. The fusion of the two allows the fused features to more completely reflect the global information of the performance audio, thereby ensuring the comprehensiveness and accuracy of the music evaluation process and improving the effectiveness of the music evaluation. It is worth noting that the fusion method can be splicing, addition, weighted fusion, etc., and this embodiment of the invention does not specifically limit this.

[0084] Subsequently, music evaluation can be conducted based on the fusion characteristics to obtain a music score for the singing audio. That is, the quality of the singing of the corresponding song can be evaluated based on the fusion characteristics to obtain a music score for the singing audio. The music score here can correspond to dimensions such as performance, fluency, pitch accuracy, rhythm, lyrics, and comprehensive dimensions.

[0085] Here, the music evaluation process based on fusion features can be implemented with the help of a music evaluation model. Specifically, the fusion features are first input into the music evaluation model, and then the music evaluation model evaluates the performance of the song corresponding to the sung audio based on the input fusion features. Finally, the music score output by the music evaluation model can be obtained. The music evaluation here can be one or more of the following: music performance score, music fluency score, music pitch score, music rhythm score, music lyrics score, and music comprehensive score.

[0086] Before inputting the fusion features into the music evaluation model, sample audio, along with sample music evaluations and predicted music scores for the sample audio, can be used to pre-train the music evaluation model. The training process of the music evaluation model is as follows: First, a large number of sample audios are collected, which must be the performance audio of a song, and the sample music score for each sample audio is determined. Then, each sample audio is processed to determine the sample fusion features, and the predicted music score for each sample audio is determined through the initial music evaluation model. Subsequently, the initial music evaluation model can be trained based on the difference between the sample music evaluation and the predicted music score for each sample audio, thus obtaining the trained music evaluation model.

[0087] It should be noted that the initial music evaluation model here can be built on a fully connected neural network. That is, the sample fusion features of each audio sample can be input into a fully connected neural network with the input dimension being the feature dimension of the sample fusion features and the output dimension being the number dimension of the music score. Finally, the predicted music score of each audio sample is obtained by regression.

[0088] The music evaluation method provided by this invention combines pitch auxiliary information reflecting the distribution of pitch in the basic pitch levels of the singing audio with music coding features characterizing various musical attributes of the singing audio to conduct music evaluation and obtain a music score for the singing audio. This not only effectively improves the credibility and accuracy of the music evaluation process, but also reduces the dependence on the song content and improves the objectivity of the score. It overcomes the shortcomings of traditional schemes, such as low credibility, accuracy, and objectivity, poor evaluation effect, and poor evaluation stability, and makes the music scoring more robust, with better evaluation effect and stronger stability.

[0089] Based on the above embodiments, Figure 2 This is a flowchart illustrating step 140 of the music evaluation method provided by the present invention, as follows: Figure 2 As shown, step 140 includes:

[0090] Step 141: Based on music coding features, predict the song category of the singing audio to obtain the song category probability distribution of the singing audio;

[0091] Step 142: Based on the probability distribution of song categories, music coding features, and pitch auxiliary information, a music evaluation is performed to obtain a music score for the performance audio.

[0092] Considering that different categories of songs have different singing difficulties, and the final music score is also affected by the difficulty of the song, for example, the songs "Two Tigers" and "Qinghai-Tibet Plateau" are relatively difficult, so the former has a generally higher music score, while the latter has a higher difficulty and its music score is mostly lower.

[0093] Therefore, in order to ensure the comprehensiveness and objectivity of the music evaluation process, in this embodiment of the invention, the category of the song being sung must also be considered during the music evaluation process.

[0094] Specifically, step 140, which involves conducting music evaluation based on pitch auxiliary information and music coding features to obtain a music score for the performance audio, includes the following steps:

[0095] Step 141: First, based on the music encoding features of the singing audio, the song category of the singing audio can be predicted to estimate the song category corresponding to the singing audio, thereby obtaining the song category probability distribution. Specifically, based on the various music attributes of the singing audio reflected by the music encoding features, the song category corresponding to the singing audio can be predicted, thereby obtaining the song category probability distribution. The song category probability distribution includes the various song categories that the singing song may correspond to, as well as the probability of belonging to each song category.

[0096] Here, the process of predicting the song category based on music encoding features can be achieved with the help of a song category prediction model. Specifically, the music encoding features are input into the song category prediction model, which can then predict the song category based on the input music encoding features, and finally obtain the song category probability distribution of the output audio.

[0097] Before inputting the music encoding features into the track category prediction model, sample audio files and their track category labels can be used to pre-train the model. The training process for the track category prediction model is as follows: First, a large number of sample audio files are collected, specifically song performance audio, and each sample audio file is labeled with its track category to form a track category label. Then, each sample audio file is processed to obtain its music encoding features, and the predicted track category probability distribution for each sample audio file is determined using the initial track category prediction model. Finally, based on the track category labels and predicted track category probability distribution of each sample audio file, the initial track category prediction model is trained, resulting in the trained track category prediction model.

[0098] It should be noted that the track category prediction model here can be built on a fully connected neural network. That is, the sample music encoding features of each sample audio can be input into a fully connected neural network with the input dimension being the feature dimension of the sample music encoding features and the output dimension being the dimension of the number of track categories in the predicted track category probability distribution. A softmax operation is then performed to obtain the predicted track category probability distribution for each sample audio.

[0099] In this embodiment of the invention, predicting the song category of the singing audio can provide a bias term for the probability distribution of the song category for the subsequent music evaluation process, thereby making the scoring sensitivity of the music evaluation model different when facing songs of different song categories.

[0100] Step 142: Music evaluation can then be performed using the probability distribution of song categories, music coding features, and pitch auxiliary information to obtain a music score for the performance audio. Specifically, this involves fusing the information represented by the music coding features, the information reflected by the pitch auxiliary information, and the information indicated by the probability distribution of song categories to obtain comprehensive fused information. Specifically, pitch auxiliary information can be used to determine the pitch distribution features of the performance audio, and the song category features of the performance audio can be determined through the probability distribution of song categories. Then, these pitch distribution features, song category features, and music coding features can be fused to obtain a fused feature. This fused feature can more comprehensively reflect the information of the performance audio than the features obtained by fusing the two mentioned above. Subsequently, this fused feature can be used to perform music evaluation to obtain a music score for the performance audio. The process of performing music evaluation using a music evaluation model has been explained in detail above and will not be repeated here.

[0101] It should be noted that this refers to the feature fusion of the three elements. The song category feature can compensate for the missing information about the song category and the corresponding difficulty level in the features obtained from the fusion of the two elements mentioned above. The fusion of the three elements ensures that the resulting fused features can more completely reflect the global information of the performance audio, thus making the music evaluation process more comprehensive, the scoring more accurate, and the evaluation effect better. It is worth noting that the fusion method of these three elements can be splicing, addition, weighted fusion, etc., and this embodiment of the invention does not specifically limit it in this way.

[0102] In this embodiment of the invention, music evaluation is performed using the probability distribution of song categories, music coding features, and pitch auxiliary information. Essentially, the probability distribution of song categories and pitch auxiliary information are used as bias terms to assist the music evaluation model in scoring the songs corresponding to the singing audio. This ensures the comprehensiveness and accuracy of the music evaluation process to the greatest extent.

[0103] Based on the above embodiments, the formulas for the loss functions of the softmax and track category prediction models are as follows:

[0104]

[0105] Where i represents the i-th sample audio, C represents the number of sample audios, i.e., the total number of sample audios, and z iz represents the sample music coding feature of the i-th sample audio. c This represents the sample music encoding feature of the Cth sample audio.

[0106] The loss function for the track category prediction model is:

[0107] L CE =-[ylogy′+(1-y)log(1-y′)]

[0108] Among them, L CE Let y be the cross-entropy loss function, y be the track category label of the sample audio, and y′ be the track category indicated by the predicted track category probability distribution of the corresponding sample audio.

[0109] It is worth noting that calculating and backpropagating the cross-entropy loss function (CE Loss) on the obtained predicted track category probability distribution helps the track category prediction model converge during the training phase.

[0110] Based on the above embodiments, Figure 3 This is a flowchart illustrating step 142 of the music evaluation method provided by the present invention, as follows: Figure 3 As shown, step 142 includes:

[0111] Step 142-1: Based on the probability distribution of the song category, determine the song category features of the singing audio; based on the pitch auxiliary information, determine the pitch distribution features of the singing audio in terms of pitch levels.

[0112] Step 142-2: Fuse the pitch distribution features, song category features, and music coding features to obtain the fused features of the singing audio;

[0113] Step 142-3: Based on the fusion features, perform music evaluation to obtain a music score for the singing audio; the music score includes at least one of the following: music performance score, music fluency score, music pitch score, music rhythm score, music lyrics score, and music comprehensive score.

[0114] Specifically, step 142, which involves evaluating the music based on the probability distribution of the song category, music coding features, and pitch auxiliary information to obtain a music score for the performance audio, includes the following steps:

[0115] Step 142-1: First, the song category features and pitch distribution features of the singing audio can be determined by the song category probability distribution and pitch auxiliary information, respectively. That is, feature extraction can be performed on these two separately to extract the song category features contained in the song category probability distribution and the pitch distribution features of the singing song on different basic pitches covered in the pitch auxiliary information, thereby obtaining the song category features and pitch distribution features of the singing audio.

[0116] Step 142-2: Subsequently, the pitch distribution features, song category features, and music coding features can be fused to obtain the fused features of the singing audio. Specifically, the information of the singing song contained in the three features can be fused to obtain a feature fusion that can comprehensively reflect the global information of the singing song. The process of fusion of the three features is essentially a process of mutual information supplementation. Fusion can make up for the missing and enhanced information in each feature, and the fused features can reflect the information of the singing song more completely, comprehensively, and accurately than the individual features.

[0117] Step 142-3: After that, the fusion features obtained by fusion can be used to conduct music evaluation, thereby obtaining the music score of the singing audio. That is, the singing song corresponding to the singing audio can be scored based on the fusion features, thereby obtaining the music score of the singing audio. The music evaluation process based on the music evaluation model has been explained in detail above and will not be repeated here.

[0118] Here, the music score can be a music performance score under the performance dimension, a music fluency score under the fluency dimension, a music pitch score under the pitch dimension, a music rhythm score under the rhythm dimension, a music lyrics score under the lyrics dimension, a music comprehensive score under the comprehensive dimension, or any combination of the above music scores. This embodiment of the invention does not specifically limit this.

[0119] Based on the above embodiments, Figure 4 This is a flowchart illustrating step 120 of the music evaluation method provided by the present invention, as follows: Figure 4 As shown, step 120 includes:

[0120] Step 121: Extract the pitch of the singing audio to obtain the pitch information of the singing audio;

[0121] Step 122: Extract pitch fractions from the pitch information to determine the pitch fraction information of the singing audio;

[0122] Step 123: Count the number of cent information within the cent range corresponding to each note in the singing audio to obtain pitch auxiliary information.

[0123] Specifically, step 120 involves extracting pitch from the singing audio to obtain pitch auxiliary information reflecting the distribution of pitch across the scale levels. This process includes the following steps:

[0124] Step 121: First, the pitch of the singing audio can be extracted to extract the pitch information contained therein. That is, the pitch can be extracted based on the singing audio to obtain the pitch information of the singing audio. The pitch information here is the information that represents the height of the sound in the singing audio, or the pitch level, and can also be understood as the information that reflects the audio amplitude of the singing audio.

[0125] Step 122: Then, the pitch information can be extracted to obtain the pitch information of the singing audio. Specifically, non-positive pitch information can be discarded, then all pitch values ​​can be divided by 440 (A1 note, the universal pitch standard) and then the log can be taken. Then the median can be subtracted and the number in the range of 0-1200 (twelve equal temperament) can be taken to obtain the pitch information.

[0126] Step 123: After that, the pitch information can be statistically analyzed. The number of pitches in the range of each note in the singing audio can be counted, that is, the distribution of the number in the range of 0-1200 (twelve-tone equal temperament). This will give us pitch auxiliary information that reflects the distribution of pitch on different basic pitches.

[0127] Based on the above embodiments, Figure 5 This is a flowchart illustrating step 130 of the music evaluation method provided by the present invention, as follows: Figure 5 As shown, step 130 includes:

[0128] Step 131: Extract musical attributes from acoustic features to obtain various musical attribute features;

[0129] Step 132: Encode various music attribute features to obtain initial music coding features;

[0130] Step 133: Align the initial music coding features to obtain the music coding features.

[0131] Specifically, step 130, which encodes the acoustic features of the singing audio to obtain music coding features that reflect various musical attributes of the singing audio, includes the following steps:

[0132] Step 131: First, it is necessary to determine the acoustic features of the singing audio. These features can be obtained by feature extraction at the acoustic level of the singing audio. They can be Filter Bank features, MFCC features, or other acoustic features. This embodiment of the invention does not make specific limitations on these features.

[0133] As a preferred approach, the acoustic features here can be Filter Bank features. This can be achieved by framing the singing audio, pre-emphasizing the framed audio, and then performing a Fast Fourier Transform to convert the time-domain signal into a frequency-domain signal. Converting to a frequency-domain signal can separate complex sound waves into sound waves of various frequencies, so as to extract the spectral features of each audio frame. Considering that there is a lot of redundancy in the frequency-domain signal, a Mel-scale filter bank can be used to filter and simplify the amplitude in the frequency domain, so that each frequency band is represented by a single value. Finally, the Filter Bank features of the singing audio can be obtained.

[0134] Here, the dimensions of the obtained Filter Bank features are [40, length], where length is related to the length of the singing audio, which is 100 times the audio duration of the singing audio. It can be expressed as: length = audio duration * 100.

[0135] Then, musical attributes can be extracted from the acoustic features of the singing audio to obtain various musical attributes, such as rhythm, cadence, pitch, and intonation, thereby obtaining various musical attribute features. Figure 6 This is a framework example diagram of the process for determining music encoding features provided by the present invention, such as... Figure 6 As shown, the translation invariance and parameter sharing of convolutional neural networks can be used to model the acoustic features of singing audio to capture various musical attribute features carried in the acoustic features. For example, singing audio with a high music score has characteristics such as clear pronunciation, clear rhythm, and accurate pitch.

[0136] Step 132: Various music attribute features can then be encoded to obtain initial music encoding features. Specifically, a convolutional neural network can encode the captured various music attribute features to encode them into a fusion feature, thereby creating the music encoding features of the singing audio. These music encoding features can reflect the various music attributes of the singing audio.

[0137] See Figure 6 It can be seen that the acoustic features pass through the following structures in the convolutional neural network in sequence:

[0138] A convolutional layer with a kernel length of 3*7 and 32 channels;

[0139] Maximum pooling layer with a kernel length of 2*4;

[0140] A convolutional layer with a kernel length of 3*7 and 64 channels;

[0141] Maximum pooling layer with a kernel length of 3*5;

[0142] A convolutional layer with a kernel length of 3*7 and 128 channels;

[0143] Maximum pooling layer with a kernel length of 3*5;

[0144] This allows us to obtain the initial musical encoding characteristics of the singing audio.

[0145] Step 133: After this step, the initial music coding features obtained can be aligned to obtain music coding features. Specifically, the initial music coding features output by the convolutional neural network can be input into the GRU (Gated Recurrent Unit) neural network to obtain the hidden state of its last layer, so as to map the input initial music coding features to a fixed length, thereby obtaining the music coding features.

[0146] Here, when faced with singing audio of different durations, the GRU neural network can map input features of different lengths to the same length through GRU operations, so that singing audio of different durations has the same length of music coding features, thus ensuring the universality of the music evaluation scheme.

[0147] Based on the above embodiments, step 140 includes:

[0148] The music encoding features and pitch auxiliary information are input into the music evaluation model to obtain the music score of the singing audio output by the music evaluation model.

[0149] The music evaluation model is trained based on the scoring error of each sample audio. The scoring error is the difference between the sample music score and the predicted music score of the corresponding sample audio. The predicted music score is determined by the music evaluation model based on the corresponding sample audio.

[0150] Specifically, step 140, which involves conducting music evaluation based on pitch auxiliary information and music coding features to obtain a music score for the performance audio, includes the following steps:

[0151] First, pitch auxiliary information, which reflects the distribution of pitch across pitch levels, can be used to determine the pitch distribution characteristics of the singing audio. Then, the music coding features and pitch distribution features can be fused to obtain fused features that reflect the global information of the singing audio.

[0152] Subsequently, based on the fusion features, a music evaluation model can be applied to evaluate the song corresponding to the vocal audio, thereby obtaining a music score for the vocal audio. Specifically, the music encoding features and pitch auxiliary information can be input into the music evaluation model, which then evaluates the performance of the song corresponding to the vocal audio based on the fusion features, ultimately obtaining a music score for the output vocal audio. The music evaluation here can be one or more of the following: music performance score, music fluency score, music pitch score, music rhythm score, music lyrics score, and music comprehensive score.

[0153] Before inputting the fusion features into the music evaluation model, the model can be pre-trained using sample music evaluations and predicted music scores for each audio sample. The training process for the music evaluation model is as follows: First, a large number of sample audio files are collected, specifically the vocal recordings of songs, and a sample music score is determined for each audio sample. Then, each audio sample is processed to determine its fusion features, and a predicted music score is determined for each sample audio sample using the initial music evaluation model. Finally, the initial music evaluation model is trained based on the difference between the sample music evaluations and predicted music scores for each audio sample, resulting in the trained music evaluation model.

[0154] Based on the above embodiments, Figure 7 This is a flowchart illustrating the training process of the music evaluation model provided by this invention, as shown below. Figure 7 As shown, the music evaluation model is trained based on the following steps:

[0155] Step 710: Based on the initial music evaluation model, determine the predicted music score for each sample audio.

[0156] Step 720: Determine the mean square loss based on the scoring error of each sample audio and the number of sample audios; determine the score difference consistency loss based on the scoring error of different sample audios.

[0157] Step 730: Based on mean square loss and difference consistency loss, perform parameter iteration on the initial music evaluation model to obtain the music evaluation model.

[0158] Specifically, the training process of the music evaluation model includes the following steps:

[0159] Step 710: Based on the sample fusion features of each sample audio, the predicted music score of each sample audio can be determined through the initial music evaluation model. The process of determining the sample fusion features of each sample audio is basically the same as the process of determining the fusion features of the singing audio, which has been described in detail above and will not be repeated here.

[0160] Step 720: Determine the difference between the sample music score and the predicted music score for each sample audio. This difference is the scoring error of each sample audio. Combining the scoring error of each sample audio with the number of sample audios, the mean of the scoring error of each sample audio can be calculated, which is the mean squared loss. The mean squared loss can be used to measure the distance between the sample music score and the predicted music score.

[0161] Meanwhile, the score difference consistency loss can be calculated based on the score errors of different sample audios, which is the difference between the score errors of different sample audios. Using the score difference consistency loss as a constraint condition for model training can keep the score difference between the predicted music score of each sample audio and the sample music score consistent. This allows the music evaluation model to pay attention to the differences in the level of different singers when singing a song, thereby making the predictive music scores output by the model more discriminative. In other words, the model can better distinguish the differences between different singing audios, and thus better distinguish the quality of different singers' singing of songs.

[0162] Step 730: Using mean squared loss and difference consistency loss, the parameters of the initial music evaluation model are iterated to obtain the music evaluation model. Specifically, the mean squared loss and difference consistency loss can be used as objective functions for training. During the training process, the parameters of the initial music evaluation model are continuously adjusted so that the predicted music score output by the model continuously approaches the sample music score. Finally, the trained music evaluation model is obtained. This music evaluation model can accurately evaluate different singing audio and obtain the prepared music score.

[0163] Based on the above embodiments, the formula for calculating the loss function of the music evaluation model is as follows:

[0164] The mean squared error loss (MSE loss) is expressed as follows:

[0165]

[0166] Among them, L MSE Let y represent the mean squared loss of the music evaluation model, i represent the i-th sample audio, N represent the number of sample audios, and y represent the sample music score for the corresponding sample audio. ′ This is the predicted music score for the corresponding sample audio.

[0167] Here, the MSE loss function is a regression loss function, which measures the distance between the predicted and the true values ​​by calculating the mean of the differences between the predicted and the true values. It has the advantages of smooth curves, continuity, and differentiability everywhere, and is biased towards the gradient descent algorithm.

[0168] The difference-consistency loss (CC Loss) is expressed as:

[0169]

[0170] Among them, L Diff-Consist The difference consistency loss of the music evaluation model is represented by y, where i represents the i-th sample audio, N represents the number of sample audios, j represents the j-th sample audio, and y represents the difference consistency loss. i This represents the sample music score for the i-th sample audio. Let y represent the predicted music score for the i-th sample audio. j This represents the sample music score for the j-th sample audio. Let represent the predicted music score for the j-th sample audio.

[0171] Here, the difference consistency constraint maintains consistency between the predicted and true values. This allows the model to pay more attention to the differences in skill levels among different singers, resulting in stronger discrimination among the predicted values ​​output by the model.

[0172] Based on the above embodiments, Figure 8 This is a general framework diagram of the music evaluation method provided by the present invention, as shown below. Figure 8 As shown, the overall process of music evaluation methods includes the following steps:

[0173] First, identify the singing audio to be evaluated;

[0174] Subsequently, pitch is extracted from the singing audio to obtain pitch auxiliary information. The pitch auxiliary information is used to reflect the distribution of pitch in the singing audio across the pitch levels. Specifically, this can be done by: extracting pitch from the singing audio to obtain pitch information; extracting cents from the pitch information to determine the cent information of the singing audio; and counting the number of cents within the cent range corresponding to each note in the singing audio to obtain pitch auxiliary information.

[0175] Subsequently, the acoustic features of the singing audio are encoded to obtain music encoding features. The music encoding features are used to reflect various musical attributes of the singing audio. Specifically, this can be done by: extracting musical attributes from the acoustic features to obtain various musical attribute features; encoding the various musical attribute features to obtain initial music encoding features; and aligning the initial music encoding features to obtain the final music encoding features.

[0176] Subsequently, music evaluation is performed based on music coding features and pitch auxiliary information to obtain a music score for the singing audio. Specifically, based on music coding features, the song category of the singing audio is predicted to obtain the song category probability distribution of the singing audio; music evaluation is performed based on the song category probability distribution, music coding features, and pitch auxiliary information to obtain a music score for the singing audio.

[0177] The process of obtaining a music score for a performance audio based on the probability distribution of the song category, music coding features, and pitch auxiliary information includes: determining the song category features of the performance audio based on the probability distribution of the song category; determining the pitch distribution features of the performance audio in terms of pitch levels based on the pitch auxiliary information; fusing the pitch distribution features, song category features, and music coding features to obtain the fused features of the performance audio; and conducting music evaluation based on the fused features to obtain a music score for the performance audio. The music score here includes at least one of the following: music performance score, music fluency score, music pitch accuracy score, music rhythm score, music lyrics score, and music comprehensive score.

[0178] Furthermore, the process of obtaining a music score for a performance audio based on music coding features and pitch auxiliary information specifically includes: inputting music coding features and pitch auxiliary information into a music evaluation model to obtain the music score of the performance audio output by the music evaluation model; here, the music evaluation model is trained based on the scoring error of each sample audio, and the scoring error is the difference between the sample music score and the predicted music score of the corresponding sample audio, and the predicted music score is determined by the music evaluation model based on the corresponding sample audio.

[0179] The music evaluation model can be trained based on the following steps: Based on the initial music evaluation model, determine the predicted music score for each sample audio; based on the scoring error of each sample audio and the number of sample audios, determine the mean square loss; based on the scoring error of different sample audios, determine the score difference consistency loss; based on the mean square loss and the score difference consistency loss, iterate the parameters of the initial music evaluation model to obtain the music evaluation model.

[0180] In this embodiment of the invention, the difference consistency loss and mean square loss are used as objective functions for learning, which can optimize music scoring; at the same time, the cross-entropy loss function is used as the objective function for learning, which can optimize the classification results of track category prediction; finally, after five-fold cross-validation, the best-performing model is obtained, namely the track category prediction model and the music evaluation model.

[0181] The method provided in this invention combines pitch-aided information reflecting the distribution of pitch in the basic pitch levels of the singing audio with music coding features characterizing various musical attributes of the singing audio to conduct music evaluation and obtain a music score for the singing audio. This not only effectively improves the credibility and accuracy of the music evaluation process, but also reduces the dependence on the song content and improves the objectivity of the score. It overcomes the shortcomings of traditional schemes, such as low credibility, accuracy, and objectivity, poor evaluation effect, and poor evaluation stability, making the music score more robust, with better evaluation effect and stronger stability.

[0182] The music evaluation device provided by the present invention is described below. The music evaluation device described below can be referred to in correspondence with the music evaluation method described above.

[0183] Figure 9 This is a schematic diagram of the music evaluation device provided by the invention, such as... Figure 9 As shown, the device includes:

[0184] Audio determination unit 910 is used to determine the singing audio to be evaluated;

[0185] The pitch extraction unit 920 is used to extract the pitch of the singing audio to obtain pitch auxiliary information, which is used to reflect the distribution of the pitch of the singing audio in the pitch level;

[0186] The feature encoding unit 930 is used to encode the acoustic features of the singing audio to obtain music encoding features, which are used to reflect various musical attributes of the singing audio.

[0187] The music evaluation unit 940 is used to perform music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio.

[0188] The music evaluation device provided by this invention combines pitch auxiliary information reflecting the distribution of pitch on the basic pitch levels of the singing audio with music coding features characterizing various musical attributes of the singing audio to perform music evaluation and obtain a music score for the singing audio. This not only effectively improves the credibility and accuracy of the music evaluation process, but also reduces the dependence on the song content and improves the objectivity of the score. It overcomes the shortcomings of traditional schemes, such as low credibility, accuracy, and objectivity, poor evaluation effect, and poor evaluation stability, and makes the music scoring more robust, with better evaluation effect and stronger stability.

[0189] Based on the above embodiments, the music evaluation unit 940 is used for:

[0190] Based on the music encoding features, the song category of the singing audio is predicted to obtain the song category probability distribution of the singing audio;

[0191] Based on the probability distribution of the song category, the music encoding features, and the pitch auxiliary information, a music evaluation is performed to obtain a music score for the performance audio.

[0192] Based on the above embodiments, the music evaluation unit 940 is used for:

[0193] Based on the probability distribution of the song category, the song category features of the singing audio are determined, and based on the pitch auxiliary information, the pitch distribution features of the singing audio in terms of pitch levels are determined.

[0194] The pitch distribution features, the song category features, and the music encoding features are fused to obtain the fused features of the singing audio.

[0195] Based on the fusion features, a music evaluation is performed to obtain a music score for the singing audio.

[0196] The music score includes at least one of the following: music performance score, music fluency score, music pitch score, music rhythm score, music lyrics score, and overall music score.

[0197] Based on the above embodiments, the pitch extraction unit 920 is used for:

[0198] Pitch information of the singing audio is obtained by extracting the pitch of the singing audio.

[0199] The pitch information is extracted to determine the pitch information of the singing audio.

[0200] The pitch auxiliary information is obtained by counting the number of pitch cents corresponding to each note in the sung audio.

[0201] Based on the above embodiments, the feature encoding unit 930 is used for:

[0202] Musical attribute extraction is performed on the acoustic features to obtain various musical attribute features;

[0203] The various music attribute features are encoded to obtain initial music encoding features;

[0204] The initial music coding features are aligned to obtain the music coding features.

[0205] Based on the above embodiments, the music evaluation unit 940 is used for:

[0206] The music encoding features and the pitch auxiliary information are input into the music evaluation model to obtain the music score of the singing audio output by the music evaluation model.

[0207] The music evaluation model is trained based on the scoring error of each sample audio. The scoring error is the difference between the sample music score and the predicted music score of the corresponding sample audio. The predicted music score is determined by the music evaluation model based on the corresponding sample audio.

[0208] Based on the above embodiments, the device further includes a model training unit, used for:

[0209] Based on the initial music evaluation model, the predicted music score for each sample audio is determined;

[0210] The mean square loss is determined based on the scoring error of each audio sample and the number of audio samples.

[0211] Based on the scoring errors of different sample audio, determine the score difference consistency loss;

[0212] Based on the mean square loss and the difference consistency loss, the initial music evaluation model is iterated to obtain the music evaluation model.

[0213] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10 As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communications interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute a music evaluation method, which includes: determining a singing audio to be evaluated; extracting pitch from the singing audio to obtain pitch auxiliary information, the pitch auxiliary information reflecting the distribution of pitch in the singing audio across pitch levels; encoding the acoustic features of the singing audio to obtain music encoding features, the music encoding features reflecting various musical attributes of the singing audio; and performing music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio.

[0214] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0215] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the music evaluation method provided by the above methods, the method comprising: determining a singing audio to be evaluated; extracting pitch from the singing audio to obtain pitch auxiliary information, the pitch auxiliary information being used to reflect the distribution of pitch of the singing audio in pitch levels; encoding the acoustic features of the singing audio to obtain music encoding features, the music encoding features being used to reflect various musical attributes of the singing audio; and performing music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio.

[0216] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the music evaluation method provided by the methods described above. The method includes: determining a singing audio to be evaluated; extracting pitch from the singing audio to obtain pitch auxiliary information, the pitch auxiliary information reflecting the distribution of pitch in the singing audio across pitch levels; encoding the acoustic features of the singing audio to obtain music encoding features, the music encoding features reflecting various musical attributes of the singing audio; and performing music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio.

[0217] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0218] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0219] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A music evaluation method, characterized in that, include: Identify the singing audio to be evaluated; Pitch is extracted from the singing audio to obtain pitch auxiliary information, which is used to reflect the distribution of pitch in the singing audio across the pitch levels. The acoustic features of the singing audio are encoded to obtain music encoding features, which are used to reflect various musical attributes of the singing audio; Based on the music encoding features and the pitch auxiliary information, a music evaluation is performed to obtain a music score for the singing audio. The music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio includes: Based on the probability distribution of the song category of the singing audio, the song category feature of the singing audio is determined, and based on the pitch auxiliary information, the pitch distribution feature of the singing audio in terms of pitch levels is determined. The pitch distribution features, the song category features, and the music encoding features are fused to obtain the fused features of the singing audio. Based on the fusion features, a music evaluation is performed to obtain a music score for the singing audio. The music score includes at least one of the following: music performance score, music fluency score, music pitch score, music rhythm score, music lyrics score, and overall music score.

2. The music evaluation method according to claim 1, characterized in that, The probability distribution of the song category in the vocal audio is determined based on the following steps: Based on the music encoding features, the song category of the singing audio is predicted to obtain the song category probability distribution of the singing audio.

3. The music evaluation method according to claim 1 or 2, characterized in that, The process of extracting pitch from the singing audio to obtain pitch auxiliary information includes: Pitch information of the singing audio is obtained by extracting the pitch of the singing audio. The pitch information is extracted to determine the pitch information of the singing audio. The pitch auxiliary information is obtained by counting the number of pitch cents corresponding to each note in the sung audio.

4. The music evaluation method according to claim 1 or 2, characterized in that, The process of encoding the acoustic features of the singing audio to obtain music encoding features includes: Musical attribute extraction is performed on the acoustic features to obtain various musical attribute features; The various music attribute features are encoded to obtain initial music encoding features; The initial music coding features are aligned to obtain the music coding features.

5. The music evaluation method according to claim 1 or 2, characterized in that, The music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio includes: The music encoding features and the pitch auxiliary information are input into the music evaluation model to obtain the music score of the singing audio output by the music evaluation model. The music evaluation model is trained based on the scoring error of each sample audio. The scoring error is the difference between the sample music score and the predicted music score of the corresponding sample audio. The predicted music score is determined by the music evaluation model based on the corresponding sample audio.

6. The music evaluation method according to claim 5, characterized in that, The music evaluation model is trained based on the following steps: Based on the initial music evaluation model, the predicted music score for each sample audio is determined; The mean square loss is determined based on the scoring error of each audio sample and the number of audio samples. Based on the scoring errors of different sample audio, determine the score difference consistency loss; Based on the mean square loss and the difference consistency loss, the initial music evaluation model is iterated to obtain the music evaluation model.

7. A music evaluation device, characterized in that, include: The audio determination unit is used to determine the singing audio to be evaluated; The pitch extraction unit is used to extract the pitch of the singing audio to obtain pitch auxiliary information, which is used to reflect the distribution of the pitch of the singing audio in the pitch level. The feature encoding unit is used to encode the acoustic features of the singing audio to obtain music encoding features, which are used to reflect various musical attributes of the singing audio. The music evaluation unit is used to perform music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio. The music evaluation based on the music encoding features and the pitch auxiliary information to obtain a music score for the singing audio includes: Based on the probability distribution of the song category of the singing audio, the song category feature of the singing audio is determined, and based on the pitch auxiliary information, the pitch distribution feature of the singing audio in terms of pitch levels is determined. The pitch distribution features, the song category features, and the music encoding features are fused to obtain the fused features of the singing audio. Based on the fusion features, a music evaluation is performed to obtain a music score for the singing audio. The music score includes at least one of the following: music performance score, music fluency score, music pitch score, music rhythm score, music lyrics score, and overall music score.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the music evaluation method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the music evaluation method as described in any one of claims 1 to 6.

Images