Song Quality Evaluation Method, Device, and Storage Medium

Abstract

A song quality evaluation method, device, and storage medium, the method comprising: obtaining a song to be evaluated, and extracting quality evaluation features for the song, the quality evaluation features including main melody features, lyrics features, and rhythm features; inputting the quality evaluation features into a trained song quality evaluation network, and having the song quality evaluation network output a quality evaluation result for the song. The song quality evaluation method and device according to the embodiment of the present application extract the main melody features, lyrics features, and rhythm features of the song to be evaluated as quality evaluation features, and predict the quality evaluation result of the song through the trained song quality evaluation network, thereby enabling the application of the main melody, lyrics, and rhythm of the song in song quality prediction, thereby better utilizing information from different aspects of the song and improving the accuracy of quality prediction.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically to a song quality evaluation method, device and storage medium. Background Art

[0002] Song selection aims to select high-quality songs from a vast library for purchase, allowing music publishers to select songs that appeal to the public. Currently, manual song selection methods are still the mainstream approach.

[0003] Manual song selection methods assess a song's overall quality based on its melody, harmony, and the quality of its lyrics. Ultimately, a committee votes to decide whether to purchase the song. While this method can select high-quality songs, due to the differences and limitations of individual and committee aesthetics, as well as individual differences in aesthetic taste over time, the quality of the selected songs varies significantly over time, often declining compared to chart-based songs.

[0004] In the above-mentioned manual song selection process, the quality of the songs is also manually evaluated, which is time-consuming and labor-intensive. Therefore, there is currently a lack of methods that can automatically evaluate song quality. Summary of the Invention

[0005] This application is proposed to solve the above-mentioned problems. According to one aspect of this application, a song quality evaluation method is provided, the method comprising: obtaining a song to be evaluated, extracting quality evaluation features for the song, the quality evaluation features comprising main melody features, lyrics features, and rhythm features; inputting the quality evaluation features into a trained song quality evaluation network, and having the song quality evaluation network output a quality evaluation result for the song.

[0006] In one embodiment of the present application, the song quality evaluation network includes a main melody scoring model, a lyrics scoring model and a rhythm scoring model, wherein: the main melody scoring model outputs a main melody quality score based on the main melody feature; the lyrics scoring model outputs a lyrics quality score based on the lyrics feature; the rhythm scoring model outputs a rhythm quality score based on the rhythm feature; the quality evaluation result of the song is obtained by fusing the main melody quality score, the lyrics quality score and the rhythm quality score, or the main melody quality score, the lyrics quality score and the rhythm quality score can also be output separately as the quality evaluation result of the song.

[0007] In one embodiment of the present application, the song quality evaluation network includes a fusion model, which includes a main melody scoring model, a lyrics scoring model and a rhythm scoring model. The quality evaluation features are input into the fusion model, and the fusion model outputs the song quality score of the song as the quality evaluation result.

[0008] In one embodiment of the present application, the main melody scoring model includes a recurrent neural network or a self-attention neural network for capturing the temporal features of the song, and also includes a convolutional neural network for capturing local features.

[0009] In one embodiment of the present application, the lyrics scoring model includes a natural language processing model for capturing emotional information and syntactic structure information in the contextual features of the song.

[0010] In one embodiment of the present application, the natural language processing model includes a converter-based bidirectional encoding representation model or generates a pre-trained converter model.

[0011] In one embodiment of the present application, the rhythm scoring model includes a convolutional neural network, a recurrent neural network, or a network based on a self-attention mechanism for tracking the overall tempo and local tempo of the song.

[0012] In one embodiment of the present application, during the training process, the main melody scoring model, the lyrics scoring model and the rhythm scoring model update the model parameters by minimizing the cross entropy or mean square error between the predicted value and the labeled true value.

[0013] In one embodiment of the present application, the training data sets of the main melody scoring model, the lyrics scoring model and the rhythm scoring model are obtained by song crawling, data cleaning and manual data labeling.

[0014] In one embodiment of the present application, the song crawling includes using a web crawler to download released songs on a music platform, or obtaining sample songs from a database.

[0015] In one embodiment of the present application, the data cleaning includes filtering the crawled songs, and the filtering includes filtering out at least one of the following: songs with repeated identifiers, songs with repeated song names, non-Chinese songs, songs with audio or lyrics that are too short, and cover versions of released songs.

[0016] In one embodiment of the present application, the manually labeled data labels include: labeling of feature labels and labeling of quality labels.

[0017] In one embodiment of the present application, the feature tag includes at least one of the following: the position of the chorus, lyrics, and song type.

[0018] In one embodiment of the present application, the quality label includes a heat index or a qualitative evaluation label.

[0019] In one embodiment of the present application, the main melody features include the main fundamental frequency features, semitone features and Mel spectrum of the entire song or chorus.

[0020] In one embodiment of the present application, the lyric feature includes a word vector obtained by tokenizing the lyrics of the entire song or the chorus of the song.

[0021] In one embodiment of the present application, the rhythm feature includes the beat structure and beat point sequence of the entire song or chorus of the song.

[0022] According to another aspect of the present application, a song quality evaluation device is provided, which includes: a feature extraction module for obtaining a song to be evaluated and extracting quality evaluation features for the song, wherein the quality evaluation features include main melody features, lyrics features and rhythm features; a song quality evaluation module for inputting the quality evaluation features into a trained song quality evaluation network, and the song quality evaluation network outputs a quality evaluation result of the song.

[0023] According to another aspect of the present application, a song quality evaluation device is provided, which includes a memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the processor executes the above-mentioned song quality evaluation method.

[0024] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is run, the song quality evaluation method described above is executed.

[0025] According to the song quality evaluation method and device of the embodiment of the present application, the main melody features, lyrics features and rhythm features of the song to be evaluated are extracted as quality evaluation features, and the quality evaluation results of the song are predicted through a trained song quality evaluation network. The main melody, lyrics and rhythm of the song can be applied in song quality prediction, thereby better utilizing information from different aspects of the song and improving the accuracy of quality prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0027] Figure 1 A schematic flowchart of a song quality evaluation method according to an embodiment of the present application is shown.

[0028] Figure 2 A schematic diagram showing an example of a song quality evaluation network used in a song quality evaluation method according to an embodiment of the present application.

[0029] Figure 3 A schematic diagram showing another example of a song quality evaluation network used in a song quality evaluation method according to an embodiment of the present application.

[0030] Figure 4 A schematic diagram showing an example of a main melody scoring model used in a song quality evaluation method according to an embodiment of the present application.

[0031] Figure 5 A schematic structural block diagram of a song quality evaluation device according to an embodiment of the present application is shown.

[0032] Figure 6 A schematic structural block diagram of a song quality evaluation device according to another embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present application more apparent, example embodiments according to the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in this application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of this application.

[0034] First, refer to Figure 1 A song quality evaluation method according to one embodiment of the present application is described. Figure 1 FIG. 1 shows a schematic flow chart of a song quality evaluation method 100 according to an embodiment of the present application. Figure 1 As shown, the song quality evaluation method 100 may include the following steps:

[0035] In step S110 , a song to be evaluated is obtained, and quality evaluation features are extracted for the song. The quality evaluation features include main melody features, lyrics features, and rhythm features.

[0036] In step S120, the quality evaluation features are input into the trained song quality evaluation network, and the song quality evaluation network outputs the quality evaluation result of the song.

[0037] The four major elements of music are rhythm, melody, harmony, and timbre. Therefore, songs can be analyzed from four perspectives: main melody, lyrics, rhythm, and harmonic composition. First, the main melody, also known as primary frequency estimation or melody extraction, is extracted. This can be a recognizable vocal or the sound of the primary instrument. It represents the underlying emotional tendencies of the song and conveys the intended emotions to the listener. Next, the lyrics describe the specific content of the song and clearly present the values it aims to convey. Rhythm, which can be soothing or dynamic, is the backbone of music and plays a crucial role, reinforcing the emotions expressed by the main melody and setting the tone. Finally, harmony is the flowing, continuous connection of several chords, creating distinct musical colors with distinct shades of richness, thinness, and depth. The temporal variations in harmony enhance the main melody's simplicity, and good harmony can effectively enhance the listener's sense of connection. Therefore, traditionally, music evaluation is also carried out in accordance with the quality of the main melody, the quality of the lyrics, the matching of the main melody and lyrics, the matching of the main melody and rhythm, and the matching of the main melody and harmony. To this end, the song quality evaluation method of the present application extracts the main melody features, lyrics features and rhythm features of the song to be evaluated as quality evaluation features, and predicts the quality evaluation results of the song (such as the song quality score) through the trained song quality evaluation network. It can realize the application of the main melody, lyrics and rhythm of the song in the song quality prediction, thereby better utilizing the information of different aspects of the song, improving the accuracy of quality prediction, and thus filling the gap in the current automatic song quality evaluation scheme.

[0038] In one embodiment of the present application, the song quality evaluation network used in step S120 may include a main melody scoring model, a lyrics scoring model, and a rhythm scoring model. Figure 2 To describe. Figure 2 FIG. 2 is a schematic diagram showing an example of a song quality evaluation network 200 used in a song quality evaluation method according to an embodiment of the present application. Figure 2 As shown, the song quality evaluation network 200 includes a main melody scoring model, a lyrics scoring model and a rhythm scoring model. The main melody scoring model outputs a main melody quality score based on the main melody features; the lyrics scoring model outputs a lyrics quality score based on the lyrics features; and the rhythm scoring model outputs a rhythm quality score based on the rhythm features. Figure 2As shown, in the embodiment of the present application, the main melody quality score, the lyrics quality score, and the rhythm quality score can be output separately as the song quality evaluation result. In addition, the main melody quality score, the lyrics quality score, and the rhythm quality score can also be fused and output, that is, the song quality evaluation result is obtained by fusion of the three scores (such as weighted average of the three scores, etc.).

[0039] In another embodiment of the present application, the song quality evaluation network may include a fusion model. Figure 3 To describe. Figure 3 FIG. 3 is a schematic diagram showing an example of a song quality evaluation network 300 used in a song quality evaluation method according to an embodiment of the present application. Figure 3 As shown, the song quality evaluation network 300 includes a fusion model, which includes a main melody scoring model, a lyrics scoring model, and a rhythm scoring model. In this embodiment, the quality evaluation features (main melody features, lyrics features, and rhythm features) are input into the fusion model, and the fusion model outputs the song quality score as the quality evaluation result of the song. Figure 2 The fusion output is similar to that shown in the figure) or the fusion of model parameters. The fusion strategies include but are not limited to Boosting (a serial algorithm for reducing deviation), Stacking, Bagging, and joint training of multiple models (i.e., main melody scoring model, lyrics scoring model, and rhythm scoring model) to jointly predict the quality score of the song.

[0040] In another embodiment of the present application, the song quality evaluation network may include a main melody scoring model, a lyrics scoring model, a rhythm scoring model and the above-mentioned fusion model.

[0041] The following further describes the more specific contents of the main melody scoring model, lyrics scoring model and rhythm scoring model.

[0042] In the embodiment of the present application, before training the above-mentioned models (main melody scoring model, lyrics scoring model, and rhythm scoring model), it is necessary to collect and annotate a large number of songs in advance to form a training data set. In the embodiment of the present application, the training data set can be obtained through three steps: song crawling, data cleaning, and manual data labeling.

[0043] In the embodiments of the present application, song crawling involves using a web crawler to download released songs from music platforms, or obtaining sample songs from a database. In this embodiment, the source of songs is not limited to released songs, but also includes sample songs. For released songs, this step involves using a web crawler to download songs from major online music platforms. For sample songs, the song data is directly sourced from the internal database.

[0044] In an embodiment of the present application, data cleaning can include screening the crawled songs, and the screening includes screening out at least one of the following: songs with repeated identifiers (IDs), songs with repeated song titles, non-Chinese songs, songs with audio or lyrics that are too short, and cover versions of released songs. In this embodiment, since the data crawled down by the crawler may contain problems such as repeated IDs and repeated song titles, it is necessary to perform a first screening. Then, songs with lyrics in non-Chinese languages, audio or lyrics that are too short, etc. are screened a second time. For released songs, the cover versions of the songs can also be continued to be removed, and only the original versions are retained. After data cleaning, the audio data of popular songs can be obtained, wherein each song has a unique identifier ID.

[0045] In an embodiment of the present application, manually labeled data labels may include the labeling of feature labels and the labeling of quality labels. Among them, the feature labels may include at least one of the following: the position of the chorus, lyrics, and song type. The chorus is the most representative and most memorable important segment in a song. Therefore, the timestamps of the start and end of the chorus can be marked to make a faster judgment on the quality of the song. Regarding the lyrics, in addition to marking the lyrics of the entire song, the lyrics corresponding to the chorus can also be marked, as well as whether there are personal pronouns (you, me, and him) in the chorus, whether there are wordless parallelisms in the chorus, etc. Regarding the type of song, for example, it can be marked that the song belongs to the modern, folk, ancient style, etc. type of song. Regarding the labeling of quality labels, for released songs, its popularity index can be marked, including but not limited to the music index, the total number of comments on the music platform, the maximum number of instant searches on the music platform, and the highest number of searches on the input method on the day, etc. For sample songs, its qualitative evaluation labels, such as good, medium, and bad quality labels, can be manually marked.

[0046] In an embodiment of the present application, the main melody feature extracted in step S110 may include the main fundamental frequency feature of the entire song or chorus of the song. In order to take into account the chords in the music and the perception of the human ear, the semitone feature (Chroma) and the Mel spectrum may also be included in the main melody feature. Among them, the fundamental frequency directly reflects the evolution of the main melody, including but not limited to continuous fundamental frequency, discrete fundamental frequency, and characterization features with fundamental frequency characteristics. The semitone feature converts the frequency to the 12 musical notes corresponding to the twelve-tone equal temperament. This feature includes chord information, so it can be used to judge whether the chords of the song are appropriately matched with the main melody. The mel-spectrogram feature has been widely used in speech, and it is a feature that conforms to the perception of the human ear. The semitone feature and the Mel spectrum can describe the audio at the level of music and human ear perception, respectively.

[0047] Since the main melody feature is a feature that changes richly in time sequence, it is necessary to use a neural network encoding that can establish a temporal relationship, including but not limited to a recurrent neural network and a self-attention neural network. In order to capture local features, a convolutional neural network is also required. Therefore, in an embodiment of the present application, the main melody scoring model may include a recurrent neural network (located in the recurrent layer) or a self-attention neural network for capturing the temporal features of the song, and also include a convolutional neural network (located in the convolution layer) for capturing local features, such as Figure 4 As shown, the model can model the characteristic regularity within the short-term time series as well as the long-term dependence.

[0048] In an embodiment of the present application, after constructing the main melody scoring model, it can be trained. During training, the model input is the main melody feature, and the output is the main melody quality score. In the initial stage of training, the main melody scoring model needs to initialize the network parameters, for example, a random initialization method can be used; then the model parameters are updated according to the data, and the specific updating method is, for example, to minimize the cross entropy or mean square error between the parameter value predicted by the model and the true value in the song quality label data as the minimum target, and the model parameters are updated until the model parameter change is less than a preset threshold, at which point the model training ends.

[0049] In an embodiment of the present application, the lyric features extracted in step S110 may include the word vectors obtained after tokenization of the lyrics of the entire song or chorus of the song. The lyric features may be the lyrics of the entire song or chorus, but since the crawled lyrics contain some irrelevant information, such as the lyricist, composer, mixer, solo or chorus, etc., the lyrics may be normalized first, and the lyrics of the entire data set may be normalized into a unified format. In addition, in order to input the lyric text into the model, it is also necessary to tokenize it, that is, to divide it into a combination of characters or words, and then convert it into the input of the word vector. The word vector may be provided by other models, or learned by following the lyrics scoring model.

[0050] In an embodiment of the present application, the lyrics scoring model needs to capture the emotional information and syntactic structure information in the context features of the song, such as repetitive structure and rhyme information. Therefore, models commonly used in natural language processing can be adopted, including but not limited to the bidirectional encoder representation model based on transformers (Bidirectional Encoder Representation from Transformers, abbreviated as BERT), the generative pre-trained transformer model (Generative pre-trained transformer, abbreviated as GPT), etc., which are used to model and capture the context of emotions, syntactic structures and rhymes in lyrics.

[0051] In an embodiment of the present application, after constructing a lyrics scoring model, it can be trained. During training, the model input is the lyrics feature, and the output is the lyrics quality score. In the initial stage of training, the lyrics scoring model needs to initialize the network parameters. For example, the parameters of the pre-trained model Bert can be used as the initialization model parameters of the lyrics scoring model, and then the model parameters are updated according to the data; the specific updating method is, for example, to minimize the cross entropy or mean square error between the parameter value predicted by the model and the true value in the song quality label data as the minimum target, and the model parameters are updated until the model parameter change is less than the preset threshold, and the model training is completed.

[0052] In an embodiment of the present application, the rhythm features extracted in step S110 may include the beat structure and beat point sequence of the entire song or chorus of the song. For example, a music beat tracking algorithm may be used to extract the rhythm features, including but not limited to using a deterministic model, a probabilistic model, or a fusion model.

[0053] In the embodiments of the present application, the rhythm scoring model can be used to capture the overall tempo of a song, as well as track the local tempo of a song. This model still uses a neural network for beat tracking. Considering that rhythm changes occur on a larger time scale than changes in the main melody, larger units are required in the network to model rhythm changes, including but not limited to convolutional neural networks, recurrent neural networks, and networks based on self-attention mechanisms.

[0054] In an embodiment of the present application, after the rhythm scoring model is constructed, it can be trained. During training, the model input is the rhythm feature and the output is the rhythm quality score. In the initial stage of training, the rhythm scoring model needs to initialize the network parameters. Specifically, for example, the model parameters can be randomly initialized and then updated according to the data. The specific updating method is, for example, to minimize the cross entropy or mean square error between the parameter value predicted by the model and the true value in the song quality label data as the minimum target, and the model parameters are updated until the model parameter change is less than a preset threshold, at which point the model training ends.

[0055] After training the above model, combined with Figure 2 The song quality evaluation network training shown is completed.

[0056] Based on this, for the song to be evaluated, the concatenated vector of the main fundamental frequency features, semitone features and Mel spectrum features of the entire song or chorus can be input into the main melody scoring model. The output layer of the main melody scoring model predicts the quality score of the song in the main melody (main melody quality score). This score can be used alone or fused with other scores.

[0057] For the song to be evaluated, the lyrics of the entire song or chorus can be normalized first, then decomposed into word combinations, and the converted word vectors are input into the lyrics-based scoring model. The score output by the lyrics scoring model represents the quality score of the song in terms of lyrics (lyrics quality score). This score can be used alone or combined with other scores.

[0058] For the song to be evaluated, the beat of the entire song or chorus is first tracked to obtain the rhythm and beat sequence of the song, and then input into the rhythm-based scoring model. The score output by the rhythm scoring model represents the rhythm quality score of the song (rhythm quality score), which can be used alone or combined with other scores.

[0059] The above is a combination of the previous Figure 2 Operation of the song quality evaluation network.

[0060] When using Figure 3 The song quality assessment network shown in the figure first extracts the song's main melody, lyrics, and rhythm features. A fusion model is then constructed, including a main melody scoring model, a lyrics scoring model, and a rhythm scoring model. When training this fusion model, the inputs are the main melody, lyrics, and rhythm features, and the output is a song quality score. This song quality score can be a fusion of the scores from the three models in the fusion model, or a comprehensive score derived from the fusion of the three model parameters. Fusion strategies include, but are not limited to, boosting, stacking, bagging, and jointly training multiple systems to jointly predict song quality scores.

[0061] At this point, the structure of the multi-module collaborative song quality evaluation network has been established. Data is collected and processed using a web crawler, and the neural network parameters are trained by minimizing the sum of the loss functions of the neural network model on the training set. Finally, in the inference phase, after extracting the entire song or chorus, the main melody, lyrics, and rhythm features of the song to be evaluated are input into the trained song quality evaluation network to generate a corresponding quality score for the song.

[0062] In the embodiments of this application, a song quality evaluation network based on a fusion model can score songs directly based on their audio and lyrics, without resorting to multiple models. Its scores can be directly displayed to musicians, helping them quickly screen songs based on their overall quality, significantly reducing song procurement time. This fusion model also analyzes song quality from multiple perspectives, further helping to select high-quality songs during the song distribution process.

[0063] Based on the above description, the song quality evaluation method according to the embodiment of the present application extracts the main melody features, lyrics features and rhythm features of the song to be evaluated as quality evaluation features, and predicts the quality evaluation results of the song through a trained song quality evaluation network. It can realize the application of the main melody, lyrics and rhythm of the song in song quality prediction, thereby better utilizing information from different aspects of the song and improving the accuracy of quality prediction.

[0064] The following combination Figures 5 to 6 The song quality evaluation apparatuses 500 and 600 provided in another aspect of the present application are described, both of which can be used to execute the song quality evaluation method according to the embodiment of the present application described above.

[0065] Figure 5 FIG. 5 shows a schematic structural block diagram of a song quality evaluation device 500 according to an embodiment of the present application. Figure 5 As shown, the song quality evaluation device 500 includes a feature extraction module 510 and a song quality evaluation module 520. Among them, the feature extraction module 510 is used to obtain the song to be evaluated and extract quality evaluation features for the song, and the quality evaluation features include main melody features, lyrics features and rhythm features; the song quality evaluation module 520 is used to input the quality evaluation features into the trained song quality evaluation network, and the song quality evaluation network outputs the quality evaluation results of the song. The song quality evaluation device 500 according to the embodiment of the present application can be used to execute the song quality evaluation method 100 described above. Those skilled in the art can understand its structure and operation in combination with the above description. For the sake of brevity, it will not be repeated here.

[0066] Figure 6 FIG. 1 is a schematic structural block diagram of a song quality evaluation device according to another embodiment of the present application. Figure 6 As shown, the song quality evaluation device 600 may include a memory 610 and a processor 620. The memory 610 stores a computer program executed by the processor 620. When the computer program is executed by the processor 620, the processor 620 executes the song quality evaluation method 100 according to the embodiment of the present application described above. Those skilled in the art can understand the specific operation of the song quality evaluation device 600 according to the embodiment of the present application in combination with the above content. For the sake of brevity, the specific details are not repeated here.

[0067] In addition, according to an embodiment of the present application, a storage medium is further provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the corresponding steps of the song quality evaluation method of the embodiment of the present application are executed. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0068] Based on the above description, the song quality evaluation method and device according to the embodiment of the present application extract the main melody features, lyrics features and rhythm features of the song to be evaluated as quality evaluation features, and predict the quality evaluation results of the song through a trained song quality evaluation network. The main melody, lyrics and rhythm of the song can be applied in song quality prediction, thereby better utilizing information from different aspects of the song and improving the accuracy of quality prediction.

[0069] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely illustrative and are not intended to limit the scope of the present application. Various changes and modifications may be made therein by those skilled in the art without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0070] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units described is merely a logical function division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another device, or ignoring or not performing some features.

[0072] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0073] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this approach of the present application should not be interpreted as reflecting the intention that the application claimed for protection requires more features than those explicitly recited in each claim. More precisely, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with fewer features than all the features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present application.

[0074] It will be understood by those skilled in the art that, except where mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus disclosed herein may be combined in any combination. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature providing the same, equivalent, or similar purpose.

[0075] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of this application and to form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

[0076] The various component embodiments of the present application can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules according to the embodiments of the present application. The application can also be implemented as a part or all of a device program (e.g., a computer program and a computer program product) for performing the method described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0077] It should be noted that the above embodiments illustrate rather than limit the present application, and that a person skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0078] The above description is merely a specific embodiment or illustration of a specific embodiment of the present application, and the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. The scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A song quality evaluation method, characterized in that: The method comprises: Obtain a song to be evaluated, and extract quality evaluation features for the song, the quality evaluation features including main melody features, lyrics features, and rhythm features, wherein the main melody features include the main fundamental frequency features, semitone features, and mel spectrum of the entire song or chorus of the song, and the rhythm features include the beat structure and beat point sequence of the entire song or chorus of the song; Inputting the quality evaluation features into a trained song quality evaluation network, and having the song quality evaluation network output a quality evaluation result of the song; in: The song quality evaluation network includes a main melody scoring model, a lyrics scoring model and a rhythm scoring model, wherein: the main melody scoring model outputs a main melody quality score based on the main melody feature; the lyrics scoring model outputs a lyrics quality score based on the lyrics feature; the rhythm scoring model outputs a rhythm quality score based on the rhythm feature; the quality evaluation result of the song is obtained by fusing the main melody quality score, the lyrics quality score and the rhythm quality score, or the main melody quality score, the lyrics quality score and the rhythm quality score can also be output separately as the quality evaluation result of the song; or, the song quality evaluation network includes a fusion model, the fusion model includes a main melody scoring model, a lyrics scoring model and a rhythm scoring model, the quality evaluation feature is input into the fusion model, and the fusion model outputs the song quality score of the song as the quality evaluation result; And among them: The main melody scoring model includes a recurrent neural network or a self-attention neural network for capturing the temporal features of the song, and also includes a convolutional neural network for capturing local features; the rhythm scoring model includes a convolutional neural network, a recurrent neural network or a network based on a self-attention mechanism for tracking the overall tempo and local tempo of the song; the rhythm scoring model has larger units than the main melody scoring model to model rhythm changes.

2. The method according to claim 1, characterized in that The lyrics scoring model includes a natural language processing model for capturing emotional information and syntactic structure information in the contextual features of the song.

3. The method according to claim 2, characterized in that The natural language processing model includes a transformer-based bidirectional encoding representation model or generates a pre-trained transformer model.

4. The method according to claim 1, wherein During the training process, the main melody scoring model, the lyrics scoring model and the rhythm scoring model update model parameters by minimizing the cross entropy or mean square error between the predicted value and the labeled true value.

5. The method according to claim 1, wherein The training data sets of the main melody scoring model, the lyrics scoring model and the rhythm scoring model are obtained through song crawling, data cleaning and manual data labeling.

6. The method according to claim 5, characterized in that The song crawling includes downloading released songs on the music platform using a web crawler, or obtaining sample songs from a database.

7. The method according to claim 5, characterized in that The data cleaning includes screening the crawled songs, and the screening includes screening out at least one of the following: songs with repeated identifiers, songs with repeated song names, non-Chinese songs, songs with audio or lyrics that are too short, and cover versions of released songs.

8. The method according to claim 5, characterized in that The manually labeled data labels include: labeling of feature labels and labeling of quality labels.

9. The method according to claim 8, characterized in that The feature tag includes at least one of the following: the position of the chorus, the lyrics, and the song type.

10. The method according to claim 8, characterized in that The quality label includes a heat index or a qualitative evaluation label.

11. The method according to claim 1, wherein The lyric features include word vectors obtained by tokenizing the lyrics of the entire song or the chorus of the song.

12. A song quality evaluation device, characterized in that: The device comprises: A feature extraction module is used to obtain a song to be evaluated and extract quality evaluation features for the song, wherein the quality evaluation features include main melody features, lyrics features, and rhythm features, wherein the main melody features include the main fundamental frequency features, semitone features, and Mel spectrum of the entire song or chorus of the song, and the rhythm features include the beat structure and beat point sequence of the entire song or chorus of the song; A song quality evaluation module, configured to input the quality evaluation features into a trained song quality evaluation network, and have the song quality evaluation network output a quality evaluation result of the song; in: The song quality evaluation network includes a main melody scoring model, a lyrics scoring model and a rhythm scoring model, wherein: the main melody scoring model outputs a main melody quality score based on the main melody feature; the lyrics scoring model outputs a lyrics quality score based on the lyrics feature; the rhythm scoring model outputs a rhythm quality score based on the rhythm feature; the quality evaluation result of the song is obtained by fusing the main melody quality score, the lyrics quality score and the rhythm quality score, or the main melody quality score, the lyrics quality score and the rhythm quality score can also be output separately as the quality evaluation result of the song; or, the song quality evaluation network includes a fusion model, the fusion model includes a main melody scoring model, a lyrics scoring model and a rhythm scoring model, the quality evaluation feature is input into the fusion model, and the fusion model outputs the song quality score of the song as the quality evaluation result; And among them: The main melody scoring model includes a recurrent neural network or a self-attention neural network for capturing the temporal features of the song, and also includes a convolutional neural network for capturing local features; the rhythm scoring model includes a convolutional neural network, a recurrent neural network or a network based on a self-attention mechanism for tracking the overall tempo and local tempo of the song; the rhythm scoring model has larger units than the main melody scoring model to model rhythm changes.

13. A song quality evaluation device, characterized in that: The device includes a memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the processor executes the song quality evaluation method according to any one of claims 1 to 11.

14. A storage medium, characterized in that The storage medium stores a computer program, which, when running, executes the song quality evaluation method according to any one of claims 1 to 11.

Images