Song recommendation model processing method, computer equipment and readable storage medium

Abstract

The invention relates to a song recommendation model processing method, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a sample song and actual song annotation information associated with the sample song; inputting the sample songs into a music understanding feature extraction module in a to-be-trained song recommendation model to obtain music understanding features of the sample songs; inputting the music understanding features into a large language module in a song recommendation model to obtain predicted song annotation information; and training a music understanding feature extraction module by using the difference between the predicted song annotation information and the actual song annotation information to obtain a trained song recommendation model. According to the method, under the condition that the music understanding feature extraction module fully learns the music understanding content represented by the actual annotation information, the music understanding features which are output by the music understanding feature extraction module and have deep music understanding are used as a recommendation basis, so that accurate and effective song recommendation is realized; and the accuracy of song recommendation results is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for processing a song recommendation model. Background Art

[0002] Song recommendations can analyze users' listening history, preferences, and behavior patterns, and provide users with personalized music recommendation lists from a large number of song resources, making it easier for users to find songs that match their song preferences. Accurate song recommendations can help music platforms improve user satisfaction and retention rates, and optimize users' platform experience.

[0003] Traditional technology mainly uses the spectral features of songs for song recommendation. However, spectral features can only represent the basic audio information of songs, but cannot intuitively and comprehensively represent the deep musical understanding of songs, which leads to errors in the song recommendation results generated based on spectral features, which is not conducive to improving the accuracy of song recommendation results. Summary of the invention

[0004] Based on this, it is necessary to provide a processing method, device, computer equipment, computer-readable storage medium and computer program product for a song recommendation model that can improve the accuracy of song recommendation results in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for processing a song recommendation model, comprising:

[0006] Obtaining sample songs and actual song annotation information associated with the sample songs;

[0007] Inputting the sample song into a music understanding feature extraction module in a song recommendation model to be trained, and obtaining music understanding features of the sample song through the music understanding feature extraction module; the music understanding features are used to characterize the understanding of the music content of the sample song;

[0008] Inputting the music comprehension features and the question text matching the actual song annotation information into the big language module in the song recommendation model, and obtaining the predicted song annotation information through the big language module;

[0009] The difference between the predicted song tagging information and the actual song tagging information is used to train the music understanding feature extraction module in the song recommendation model to obtain a trained song recommendation model.

[0010] In one embodiment, the music understanding feature extraction module includes a pre-trained song audio feature extraction model and a feature conversion model to be trained, and the music understanding features of the sample song obtained by the music understanding feature extraction module include:

[0011] Inputting the sample song into the song audio feature extraction model to obtain the song audio features of the sample song;

[0012] The song audio features of the sample song are input into the feature conversion model to be trained to obtain the music understanding features of the sample song.

[0013] In one embodiment, the step of inputting the song audio features of the sample song into the feature conversion model to be trained to obtain the music understanding features of the sample song includes:

[0014] Performing convolution processing on the song audio features of the sample song through the feature conversion model to be trained to obtain a convolution result;

[0015] The convolution result is input into the conversion module in the feature conversion model to be trained to obtain the music understanding features of the sample song.

[0016] In one embodiment, the conversion module includes a first fully connected layer, a second fully connected layer and a third fully connected layer, and the step of inputting the convolution result into the conversion module in the feature conversion model to be trained to obtain the music understanding features of the sample song includes:

[0017] Inputting the convolution result into the first fully connected layer, and mapping the output result of the first fully connected layer according to a preset activation function to obtain first feature information;

[0018] Inputting the convolution result into the second fully connected layer to obtain second feature information;

[0019] Inputting the product of the first feature information and the second feature information into the third fully connected layer to obtain third feature information;

[0020] The third feature information and the convolution result are added to obtain the music understanding feature of the sample song.

[0021] In one embodiment, obtaining the sample song includes:

[0022] Acquire a song data set; the song data set includes a number of candidate songs;

[0023] Sorting the candidate songs in the song data set according to the song popularity information of the candidate songs to obtain a sorting result;

[0024] The sample songs are selected from the sorting results.

[0025] In a second aspect, the present application also provides a song recommendation method, comprising:

[0026] Inputting the song to be recommended into the music understanding feature extraction module in the pre-trained song recommendation model to obtain the music understanding features corresponding to the song to be recommended; wherein the song recommendation model is trained by the processing method of the song recommendation model;

[0027] A song recommendation result of the song to be recommended is determined based on the music understanding features corresponding to the song to be recommended and the sample songs corresponding to the user; the song recommendation result indicates whether the song to be recommended is recommended to the user.

[0028] In one embodiment, the sample songs include positive sample songs and negative sample songs, and determining the song recommendation result of the song to be recommended according to the music understanding features corresponding to the song to be recommended and the sample songs corresponding to the user includes:

[0029] The music understanding features of the positive sample songs and the music understanding features of the negative sample songs corresponding to the user are obtained through the music understanding feature extraction module in the pre-trained song recommendation model; the positive sample songs are songs of the first type of operation type in the listening behavior data of the user; the negative sample songs are songs of the second type of operation type in the listening behavior data of the user; the first type is different from the second type;

[0030] The song recommendation result of the song to be recommended is determined according to the music understanding features corresponding to the song to be recommended, the music understanding features of the positive sample song and the music understanding features of the negative sample song.

[0031] In a third aspect, the present application also provides a processing device for a song recommendation model, comprising:

[0032] An acquisition module, used to acquire sample songs and actual song annotation information associated with the sample songs;

[0033] An extraction module, used for inputting the sample song into a music understanding feature extraction module in a song recommendation model to be trained, and obtaining music understanding features of the sample song through the music understanding feature extraction module; the music understanding features are used to characterize the understanding of the music content of the sample song;

[0034] A prediction module, used for inputting the music understanding features and the question text matching the actual song annotation information into the large language module in the song recommendation model, and obtaining the predicted song annotation information through the large language module;

[0035] The training module is used to utilize the difference between the predicted song annotation information and the actual song annotation information to train the music understanding feature extraction module in the song recommendation model to obtain a trained song recommendation model.

[0036] In a fourth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above method are implemented.

[0037] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method when executed by a processor.

[0038] In a sixth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0039] The processing method, apparatus, computer equipment, computer-readable storage medium and computer program product of the song recommendation model described above utilize the music understanding feature extraction module in the song recommendation model to be trained to obtain the music understanding features of the sample song, and convert the music understanding features into prediction information through the large language module in combination with the question text that matches the actual song annotation information, and train the music understanding feature extraction module in combination with the prediction information and the difference between the actual annotation information associated with the sample song. The expressive power of the large language module can be utilized to enable the music understanding feature extraction module to fully learn the music understanding content represented by the actual annotation information, so that the music understanding feature extraction module can output accurate and intuitive music understanding features, thereby improving the accuracy of the music understanding features. Compared with song recommendations based on the audio features of the song, song recommendations based on the music understanding features can use music understanding features with in-depth music understanding as the basis for recommendation, thereby achieving accurate and effective song recommendations and improving the accuracy of the song recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 An application environment diagram of a processing method for a song recommendation model in one embodiment;

[0042] Figure 2 A schematic flow chart of a method for processing a song recommendation model in one embodiment;

[0043] Figure 3 A schematic diagram of a process for obtaining music understanding features in one embodiment;

[0044] Figure 4 A schematic diagram of a flow chart of a song recommendation method in one embodiment;

[0045] Figure 5 A structural block diagram of a processing device for a song recommendation model in one embodiment;

[0046] Figure 6 The figure is a diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The processing method of the song recommendation model provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the sample song and the actual song annotation information associated with the sample song; the server 104 inputs the sample song into the music understanding feature extraction module in the song recommendation model to be trained, and obtains the music understanding feature of the sample song through the music understanding feature extraction module; the music understanding feature is used to characterize the understanding of the music content of the sample song; the server 104 inputs the music understanding feature and the question text matching the actual song annotation information into the big language module in the song recommendation model, and obtains the predicted song annotation information through the big language module; the server 104 uses the difference between the predicted song annotation information and the actual song annotation information to train the music understanding feature extraction module in the song recommendation model to obtain the trained song recommendation model. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart car devices, projection devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0049] In an exemplary embodiment, Figure 2 As shown, a method for processing a song recommendation model is provided, and the method is applied to a server as an example for explanation, including the following steps S202 to S208. Among them:

[0050] Step S202, obtaining sample songs and actual song annotation information associated with the sample songs.

[0051] The sample songs may refer to songs used to train a song recommendation model. In practical applications, the sample songs may include any songs in a song database.

[0052] Among them, the actual song annotation information associated with the sample song may refer to information that describes the musical understanding of the sample song. In practical applications, the actual song annotation information may include but is not limited to song reviews for the song (such as the comment text in the comment area corresponding to the song, etc.) and music analysis (such as score analysis, chapter analysis, music style analysis, scene and artistic conception analysis, etc.).

[0053] As an example, the server can make song recommendations with the help of a song recommendation model. In order to optimize the performance of the song recommendation model, the server can train the song recommendation model with sample songs. First, the server can obtain several songs from a preset song database as sample songs, and obtain objective music analysis texts (detailed descriptions) for the sample songs and / or obtain song comment texts for the sample songs from the song comment area corresponding to the sample songs as actual song annotation information associated with the sample songs. For example, the content of the actual song annotation information associated with a song in the song database can be expressed as:

[0054] The first movement is in sonata form. The music is brilliant, magnificent, and colorful. The rich imagination is fully exerted. It starts with a long prelude of 106 bars, which occupies an important position in the music. The prelude theme is first sung by the orchestra, and the piano gives a strong and passionate chord to provide favorable coordination. Then the theme is handed over to the piano and the orchestra accompanies it. The prelude is magnificent, brilliant and exciting. It has a strong appeal and shocking power. The theme of the presentation part was originally a folk song collected by the composer from an artist, and then it was creatively used to make it humorous. The secondary part is gradually introduced by the orchestra and consists of two interconnected themes. The first theme is affectionate, sincere, and sweet like a dream. The second theme is played by muted string instruments, which is quiet and elegant.

[0055] Step S204, input the sample song into the music understanding feature extraction module in the song recommendation model to be trained, and obtain the music understanding features of the sample song through the music understanding feature extraction module; the music understanding features are used to characterize the understanding of the music content of the sample song.

[0056] Among them, the song recommendation model may refer to a model used to determine the recommendation results of songs. In actual applications, the recommendation results output by the song recommendation model can be used to determine whether to recommend songs to relevant users.

[0057] Among them, the music understanding feature extraction module may refer to a module for extracting music understanding features of songs. In practical applications, the music understanding feature extraction module may include a MERT (Music Encoding and Representation Transformer) model, wherein the MERT model may refer to a deep learning model for processing music data. The music understanding feature extraction module may also be composed of a MERT model and a feature conversion model, wherein the feature conversion model may perform multi-level feature conversion (such as feature dimension conversion and nonlinear conversion, etc.) on the song audio features output by the MERT model by performing convolution extraction, nonlinear mapping, linear projection, feature interaction, and residual connection operations on the song audio features (such as MERT representation) output by the MERT model.

[0058] The music understanding feature of the sample song may refer to information representing the understanding of the music content of the sample song.

[0059] As an example, the server can input sample songs into the music understanding feature extraction module in the song recommendation model to be trained, and obtain the music understanding features of the sample songs through the music understanding feature extraction module. In actual applications, when the music understanding feature extraction module is composed of the MERT model, the output results of the MERT model based on the sample songs can be used as music understanding features. When the music understanding feature extraction module is composed of the MERT model and the feature conversion model, the feature conversion model performs feature conversion on the song audio features (such as MERT representation) output by the MERT model based on the sample songs, and the result obtained can be used as the music understanding feature.

[0060] Step S206: input the music understanding features and the question text matching the actual song annotation information into the big language module in the song recommendation model, and obtain the predicted song annotation information through the big language module.

[0061] Among them, the large language module may refer to a large language model (Large Language Model, LLM for short), which may include a deep learning model trained using a large amount of text data, and the large language module may generate natural language text or understand the meaning of natural language text.

[0062] The question text may refer to information used to control the large language module to generate text output.

[0063] Among them, the predicted song annotation information may refer to the information output by the large language module based on the music understanding features of the sample song.

[0064] As an example, the server inputs the music understanding feature into the pre-trained big language module in the song recommendation model, and obtains the predicted song tagging information through the big language module. In actual applications, the big language module can analyze the music understanding features of the sample song in combination with the question text input by the user, and output the predicted song tagging information corresponding to the sample song. Since the music understanding feature is obtained after feature conversion of the song audio feature (such as MERT representation), the music understanding feature can contain the song emotion, song rhythm, song melody and other information contained in the song audio feature (such as MERT representation). At this time, the big language module can perform data processing (such as feature mapping, etc.) on the music understanding feature, and then analyze the song emotion, song rhythm, song melody and other information of the song based on the music understanding feature, and then generate the predicted song tagging information corresponding to the sample song based on the analysis results. It can be understood that the music understanding feature can have a deeper music understanding than the song audio feature (such as MERT representation).

[0065] Step S208, using the difference between the predicted song annotation information and the actual song annotation information, the music understanding feature extraction module in the song recommendation model is trained to obtain a trained song recommendation model.

[0066] As an example, the server can calculate the model loss function value of the song recommendation model based on the predicted song annotation information and the actual song annotation information. When the predicted song annotation information is close enough to the actual song annotation information, the model loss function value of the song recommendation model is less than a preset threshold. At this time, the server determines that the training of the music understanding feature extraction module is completed and obtains a trained song recommendation model. In a specific implementation, when the music understanding feature extraction module is composed of a MERT model, the server can calculate the model loss function value of the song recommendation model based on the predicted song annotation information and the actual song annotation information, and use the model loss function value to adjust the model parameters of the MERT model until the predicted song annotation information is close enough to the actual song annotation information. At this time, the training of the music understanding feature extraction module is completed.

[0067] In the processing method of the above-mentioned song recommendation model, the music understanding feature extraction module in the song recommendation model to be trained is used to obtain the music understanding features of the sample songs, and the music understanding features are converted into prediction information through the big language module. The music understanding feature extraction module is trained by combining the difference between the prediction information and the actual annotation information associated with the sample songs. The expressive power of the big language module can be utilized to enable the music understanding feature extraction module to fully learn the music understanding content represented by the actual annotation information, so that the music understanding feature extraction module can output accurate and intuitive music understanding features, thereby improving the accuracy of the music understanding features. Compared with song recommendations based on the audio features of the songs, song recommendations based on the music understanding features can use music understanding features with in-depth music understanding as the basis for recommendation, thereby achieving accurate and effective song recommendations and improving the accuracy of the song recommendation results.

[0068] In some embodiments, the song recommendation model includes a music understanding feature extraction module and a large language module, wherein the music understanding feature extraction module can be divided into two cases: including only a MERT model and including a MERT model and an Adapter module (i.e., a feature conversion model). The large language model is pre-trained. When the music understanding feature extraction module only includes the MERT model, the trained song recommendation model can be obtained after the music understanding feature extraction module is trained. When the music understanding feature extraction module includes the MERT model and the Adapter module, the MERT model is pre-trained. After the Adapter module is trained, the trained song recommendation model can be obtained.

[0069] In an exemplary embodiment, the music understanding feature extraction module includes a pre-trained song audio feature extraction model and a feature conversion model to be trained. The music understanding features of the sample song are obtained through the music understanding feature extraction module, including: inputting the sample song into the song audio feature extraction model to obtain the song audio features of the sample song; inputting the song audio features of the sample song into the feature conversion model to be trained to obtain the music understanding features of the sample song.

[0070] Among them, the song audio feature extraction model may refer to a deep learning model used to process music data. In practical applications, the song audio feature extraction model may include but is not limited to the MERT model, MULE model, LAION-CLAP model, Hubert model, etc.

[0071] Among them, the feature conversion model may refer to a model used to perform feature conversion on the output results of the song audio feature extraction model. In practical applications, the feature conversion model may include an Adapter module.

[0072] As an example, when the music understanding feature extraction module is composed of a song audio feature extraction model and a feature conversion model to be trained, in order to obtain the music understanding features of a sample song, the server can input the sample song into the song audio feature extraction model to obtain the song audio features of the sample song, and then the server can input the song audio features of the sample song into the feature conversion model to be trained. The feature conversion model to be trained can perform feature conversion on the song audio features of the sample song and convert the song audio features of the sample song into input suitable for the large language module to obtain the music understanding features of the sample song.

[0073] In a specific implementation, when the music understanding feature extraction module is composed of a MERT model and a feature conversion model to be trained, in order to train the music understanding feature extraction module, the server can input the music understanding features output by the music understanding feature extraction module based on sample songs into the large language module to obtain the predicted song annotation information of the sample songs. Then, the server can combine the predicted song annotation information and the actual song annotation information of the sample songs to calculate the model loss function value of the song recommendation model without modifying the model parameters of the MERT model (that is, the MERT model does not participate in the training). The model loss function value is used to adjust the model parameters of the feature conversion model to be trained, thereby avoiding the training complexity problem caused by adjusting the model parameters of the MERT model and the feature conversion model at the same time, and improving the training efficiency, until the predicted song annotation information is close enough to the actual song annotation information and the feature conversion model training is completed, at which time the music understanding feature extraction module training is completed.

[0074] In this embodiment, when the music understanding feature extraction module includes a song audio feature extraction model and a feature conversion model to be trained, the feature conversion model to be trained is used to convert the song audio features output by the song audio feature extraction model based on the sample song into the music understanding features of the sample song. The song audio features can be used as preliminary music understanding representations, and the feature conversion model can be used to convert the song audio features into music understanding features used as input to the large language module. The interpretability of the music understanding feature extraction module is improved, which helps to understand and debug the feature conversion model, facilitates the subsequent training of the music understanding feature extraction module based on actual song annotation information and predicted song annotation information, and improves the training efficiency of the music understanding feature extraction module.

[0075] In some embodiments, the song audio features of the sample song are input into the feature conversion model to be trained to obtain the music understanding features of the sample song, including: performing convolution processing on the song audio features of the sample song through the feature conversion model to be trained to obtain the convolution result; inputting the convolution result into the conversion module in the feature conversion model to be trained to obtain the music understanding features of the sample song.

[0076] Among them, the convolution result may refer to the result obtained after the feature conversion model to be trained performs convolution processing on the song audio features of the sample song. In practical applications, the convolution result may include a feature map.

[0077] The conversion module may refer to a module for converting the convolution result into a music understanding feature. In practical applications, the conversion module may include a Projection module (a model for data mapping). In a specific implementation, Figure 3 As shown, a flow chart for obtaining music understanding features is provided, and the conversion module (Projection module) may include three fully connected layers (such as a first fully connected layer, a second fully connected layer, and a third fully connected layer), wherein the first fully connected layer may be used to perform a linear transformation on the input data and introduce nonlinearity through an activation function, the second fully connected layer may be used to perform a linear transformation on the input data, and the third fully connected layer may be used to integrate the output results of the first fully connected layer and the second fully connected layer.

[0078] As an example, the server can perform convolution processing on the song audio features of the sample song through the feature conversion model to be trained to obtain the convolution result, and then the server can input the convolution result into the conversion module in the feature conversion model to be trained to obtain the music understanding feature of the sample song. In actual applications, the conversion module can regularize the convolution result and convert the regularized convolution result into a music understanding feature through a fully connected layer.

[0079] In this embodiment, a feature conversion model is used to perform convolution processing on song audio features, and the convolution results are converted into music understanding features through a conversion module. This can comprehensively capture song audio features while preventing overfitting and introducing nonlinear features, and convert abstract song audio features into music understanding features, thereby improving the accuracy of music understanding features and facilitating subsequent song recommendations based on music understanding features, thereby improving the accuracy of song recommendation results.

[0080] In some embodiments, the conversion module includes a first fully connected layer, a second fully connected layer and a third fully connected layer, and the convolution result is input into the conversion module in the feature conversion model to be trained to obtain the music understanding features of the sample song, including: inputting the convolution result into the first fully connected layer, and mapping the output result of the first fully connected layer according to a preset activation function to obtain the first feature information; inputting the convolution result into the second fully connected layer to obtain the second feature information; inputting the product of the first feature information and the second feature information into the third fully connected layer to obtain the third feature information; adding the third feature information and the convolution result to obtain the music understanding features of the sample song.

[0081] Among them, the fully connected layer can be represented as Linear.

[0082] The preset activation function may include but is not limited to the SiLU activation function.

[0083] As an example, the server may perform convolution processing on the song audio features of the sample song through the convolution layer in the feature conversion model to be trained, and perform channel dimension compression on the song audio features to obtain the convolution result X i-1 , and then the server will convolve the result X i-1 Input into the transformation module (Projection module), such as Figure 3 As shown in FIG. 1 , a schematic diagram of a process for obtaining music understanding features is provided. The conversion module (Projection module) may include three fully connected layers (such as a first fully connected layer, a second fully connected layer, and a third fully connected layer). The conversion module first performs convolution on the result X i-1 The first fully connected layer processes the convolution result after the regularization processing, and uses the SiLU activation function to map the output result of the first fully connected layer to obtain the first feature information. The second fully connected layer processes the convolution result after the regularization processing to obtain the second feature information. The conversion module calculates the product between the first feature information and the second feature information, and inputs the product to the third fully connected layer. The conversion module converts the output result of the third fully connected layer with the convolution result X i-1 Add together to get the music understanding feature X i .

[0084] In this embodiment, the convolution results are processed by three fully connected layers respectively, and the convolution results are converted into music understanding features, which can integrate multi-level song features to form a comprehensive feature representation as the music understanding feature, thereby improving the accuracy of the music understanding feature and facilitating the subsequent song recommendation based on the music understanding feature, thereby improving the accuracy of the song recommendation results.

[0085] In some embodiments, obtaining predicted song tagging information through a large language module includes: obtaining a question text that matches the actual song tagging information; and inputting the question text and music understanding features into the large language module to obtain predicted song tagging information.

[0086] The question text may refer to information used to control the large language module to generate text output. In actual applications, the question text may include content such as "describe this song" and "evaluate this song".

[0087] As an example, in order to obtain predicted song tagging information for a sample song, the server may first obtain a question text that matches the actual song tagging information. In actual applications, when the actual song tagging information is music analysis, the server may use a text in a preset text library that contains the content of "describing this song" as the question text that matches the actual song tagging information; when the actual song tagging information is a song review, the server may use a text in a preset text library that contains the content of "evaluating this song" as the question text that matches the actual song tagging information. Then, the server may input the question text and the music understanding features into the big language module, and the big language module may output the predicted song tagging information corresponding to the sample song based on the question content of the question text.

[0088] In this embodiment, the question text that matches the actual song tagging information is used to control the large language module to output the predicted song tagging information corresponding to the sample song. This can promptly respond to the user's question content and accurately output the corresponding prediction information, thereby improving the accuracy of the predicted song tagging information and facilitating the subsequent training of the song recommendation model based on the difference between the predicted song tagging information and the actual song tagging information, thereby improving the accuracy of the song recommendation results output by the song recommendation model.

[0089] In some embodiments, obtaining sample songs includes: obtaining a song data set; the song data set includes a number of candidate songs; sorting the candidate songs in the song data set according to song popularity information of the candidate songs to obtain a sorting result; and filtering out sample songs from the sorting result.

[0090] The song data set may refer to a pre-set collection of several songs. In practical applications, the song data set may be used as candidate songs, and each candidate song may have corresponding actual song annotation information.

[0091] Among them, song popularity information may refer to information that characterizes the popularity and / or popularity of a song.

[0092] Among them, the sorting result may refer to the result obtained after sorting the song popularity information of the song in a specific order (such as from large to small or from small to large). For example: after sorting the song popularity information of the song in order from large to small, the song with a higher position or order in the sorting result has a greater value of the song popularity information.

[0093] As an example, the server obtains a song data set from a music platform, and the song data set contains several candidate songs. Then the server can obtain the song popularity information of each candidate song, and sort the candidate songs in the song data set according to the song popularity information of the candidate songs to obtain the sorting results. Then the server can filter out sample songs from the sorting results. For example: the song data set contains m candidate songs, and the song popularity information of the m candidate songs is sorted in descending order. The server can select the first N (such as top-N) songs in the sorting results as sample songs for training the song recommendation model.

[0094] In this embodiment, the size of song popularity information is used to filter out sample songs from a song data set. Since the sample songs have high song popularity and a large amount of actual song annotation information, a song recommendation model can be trained using a large amount of actual song annotation information corresponding to the sample songs, thereby improving the accuracy of the output results of the song recommendation model and thus improving the accuracy of the song recommendation results output by the song recommendation model.

[0095] In some embodiments, a song recommendation method is provided, including: inputting the song to be recommended into a music understanding feature extraction module in a pre-trained song recommendation model to obtain the music understanding features corresponding to the song to be recommended; determining the song recommendation result of the song to be recommended based on the music understanding features corresponding to the song to be recommended and the sample songs corresponding to the user; the song recommendation result indicates whether the song to be recommended is recommended to the user.

[0096] Among them, the song recommendation model can be trained by the above-mentioned song recommendation model processing method.

[0097] As an example, the server can input the song to be recommended into the music understanding feature extraction module in the pre-trained song recommendation model. The music understanding feature extraction module can generate music understanding features corresponding to the song to be recommended. Then the server can determine the song recommendation result of the song to be recommended based on the music understanding features corresponding to the song to be recommended. In actual applications, the server can determine the song recommendation result of the song to be recommended by comparing the degree of difference between the music understanding features corresponding to the song to be recommended and the music understanding features of the songs liked by the user.

[0098] In this embodiment, the music understanding feature extraction module in the pre-trained song recommendation model is used to extract the music understanding features of the song to be recommended, and the music understanding features of the song to be recommended are used to determine the song recommendation results. Based on the music understanding feature extraction module in the pre-trained song recommendation model, accurate music understanding features with in-depth music understanding can be obtained, which facilitates song recommendations based on the music understanding features, thereby improving the accuracy of the song recommendation results.

[0099] In some embodiments, the sample songs include positive sample songs and negative sample songs, and the song recommendation results of the songs to be recommended are determined according to the music understanding features corresponding to the songs to be recommended and the sample songs corresponding to the user, including: obtaining the music understanding features of the positive sample songs and the music understanding features of the negative sample songs corresponding to the user through the music understanding feature extraction module in the pre-trained song recommendation model; the positive sample songs are songs whose operation type is the first type in the user's listening behavior data; the negative sample songs are songs whose operation type is the second type in the user's listening behavior data; the first type is different from the second type; the song recommendation results of the songs to be recommended are determined according to the music understanding features corresponding to the songs to be recommended, the music understanding features of the positive sample songs, and the music understanding features of the negative sample songs.

[0100] Among them, positive sample songs can refer to songs with the first type of operation type in the user's listening behavior data. In actual applications, the user's listening behavior data can represent the operations performed by the user on the songs. The first type may include like operations, and positive sample songs can be used as songs that the user likes.

[0101] Among them, negative sample songs can refer to songs whose operation type is the second type in the user's listening behavior data. In actual applications, the second type may include dislike operations, and negative sample songs can be songs that the user does not like.

[0102] As an example, the server can obtain positive sample songs and negative sample songs from the user's historical listening records. For example, songs liked by the user can be used as positive sample songs, and songs disliked by the user can be used as negative sample songs. The server obtains the music understanding features of the positive sample songs and the music understanding features of the negative sample songs corresponding to the user through the music understanding feature extraction module in the pre-trained song recommendation model. Then, the server can determine the song recommendation results of the song to be recommended based on the music understanding features corresponding to the song to be recommended, the music understanding features of the positive sample songs, and the music understanding features of the negative sample songs.

[0103] In actual applications, the server can calculate the similarity between the music understanding features corresponding to the song to be recommended and the music understanding features of the positive sample songs, and calculate the similarity between the music understanding features corresponding to the song to be recommended and the music understanding features of the negative sample songs. When the similarity between the music understanding features corresponding to the song to be recommended and the music understanding features of the positive sample songs is greater than a first threshold, and the similarity between the music understanding features corresponding to the song to be recommended and the music understanding features of the negative sample songs is less than a second threshold, the server can determine that the song recommendation result of the song to be recommended is that the song to be recommended needs to be recommended to the user; when the similarity between the music understanding features corresponding to the song to be recommended and the music understanding features of the positive sample songs is less than or equal to the first threshold, and the similarity between the music understanding features corresponding to the song to be recommended and the music understanding features of the negative sample songs is greater than or equal to the second threshold, the server can determine that the song recommendation result of the song to be recommended is that the song to be recommended does not need to be recommended to the user.

[0104] In this embodiment, based on the music understanding features of the positive sample songs and the negative sample songs corresponding to the user, the difference between the music understanding features of the song to be recommended and the music understanding features of the positive sample songs and the negative sample songs is analyzed. This can ensure that the song to be recommended has similar music understanding to the positive sample songs, and at the same time ensure that the music understanding of the song to be recommended and the negative sample songs are greatly different, thereby obtaining accurate song recommendation results and improving the accuracy of the song recommendation results.

[0105] In one of the exemplary embodiments, the music understanding feature can also be used for other song analysis tasks besides song recommendation. In practical applications, the song analysis tasks can also include song genre classification, singer timbre recognition, and song language recognition. When the song analysis task is song genre classification, the server can input the question text for the song genre and the music understanding feature of the song to be classified into the large language model, and the large language model can output the song genre category of the song to be classified; when the song analysis task is singer timbre recognition, the server can input the question text for the singer timbre and the music understanding feature of the song to be recognized into the large language model, and the large language model can output the singer timbre type of the song to be recognized; when the song analysis task is song language recognition, the server can input the question text for the song language and the music understanding feature of the song to be recognized into the large language model, and the large language model can output the song language of the song to be recognized.

[0106] In this embodiment, the music understanding features of the song are used to implement various song analysis tasks. The music understanding features with in-depth music understanding can be used as the analysis basis for the song analysis tasks, thereby accurately and efficiently completing the corresponding song analysis tasks and improving the accuracy of the song analysis results.

[0107] In one embodiment, if Figure 4 As shown, a song recommendation method is provided, and the song recommendation model may include a music understanding feature extraction module and a large language model module, wherein the music understanding feature extraction module may be composed of a MERT model and a feature conversion model (such as an Adapter module), the server inputs the song audio of the song to be recommended into the MERT model, and the MERT model extracts the song audio features of the song to be recommended (such as embedding representation), and the song audio features of the song to be recommended can be used as the preliminary music representation of the song to be recommended, and the server inputs the song audio features of the song to be recommended into the feature conversion model, and the feature conversion model can convert the song audio features of the song to be recommended into the music understanding features of the song to be recommended (such as MERT-Adapter embedding) through the convolution layer and the fully connected layer in the conversion module (Projection module). The music understanding features of the song to be recommended can be used to determine whether to recommend the song to be recommended to the user. Specifically, when the distance between the music understanding features of the song to be recommended and the music understanding features corresponding to the positive sample song of the user is less than the first distance threshold, and the distance between the music understanding features of the song to be recommended and the music understanding features corresponding to the negative sample song of the user is greater than the first distance threshold, the server can recommend the song to be recommended to the above-mentioned user.

[0108] In the process of training the song recommendation model using sample songs, the MERT model may not participate in the training. The feature conversion model may serve as a bridge between the MERT model and the large language model. The server inputs the music understanding features of the sample songs and the corresponding question text Q (question) into the large language model. The large language model may output the predicted song annotation information of the sample songs. The server may also obtain the actual song annotation information of the sample songs. The training goal of the song recommendation model may be to make the predicted song annotation information close enough to the actual song annotation information. Through the large language model, it may be intuitively discovered that the required music understanding feature MERT-Adapter embedding needs to include the music understanding information described by the actual song annotation information. When the output of the large language model is closer to the music understanding information described by the actual song annotation information, the model loss function value is smaller, thereby continuously optimizing the model parameters of the music understanding feature extraction module (such as the feature conversion model).

[0109] It is understandable that the music understanding base model can refer to a network that uses a large amount of music data for self-supervisory training, which can be used to extract music representations and train models for downstream tasks. The music understanding feature extraction module can be used as a music understanding base model, which only uses the song audio itself or spectral features for downstream task training. Although the song audio itself or spectral features retain a large amount of original information of the song, the part of the music downstream task understanding is entirely handed over to the small model in the downstream task. The small model has limited learning ability and cannot deeply understand the music, resulting in poor task completion of the music downstream task. On the basis of the MERT model, the feature conversion model is used as a bridge between the audio representation and the large language model, which can be used to guide the MERT representation to deeply and intuitively understand the music content. Compared with the MERT representation, the music understanding features with in-depth music understanding output by the feature conversion model are used as the analysis basis for the music downstream task, which is conducive to optimizing the completion effect of the music downstream task.

[0110] In this embodiment, the optimization of the music understanding base model is guided by the big language model, so that the music base model optimization based on the big language teacher model can be achieved. The expressive power of the big language model can be used to optimize the MERT model as the base model, so that the goal of music understanding can be more intuitive and controllable. Moreover, the obtained music understanding feature MERT-Adapter embedding has a deeper understanding of music than the song audio feature MERT embedding, which brings better improvement to downstream tasks.

[0111] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0112] Based on the same inventive concept, the embodiment of the present application also provides a device for processing a song recommendation model for implementing the processing method of the song recommendation model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the processing device for one or more song recommendation models provided below can refer to the limitations of the processing method for the song recommendation model above, and will not be repeated here.

[0113] In an exemplary embodiment, Figure 5 As shown, a processing device for a song recommendation model is provided, comprising: an acquisition module 502, an extraction module 504, a prediction module 506 and a training module 508, wherein:

[0114] The acquisition module 502 is used to acquire sample songs and actual song annotation information associated with the sample songs.

[0115] The extraction module 504 is used to input the sample song into the music understanding feature extraction module in the song recommendation model to be trained, and obtain the music understanding features of the sample song through the music understanding feature extraction module; the music understanding features are used to characterize the understanding of the music content of the sample song.

[0116] The prediction module 506 is used to input the music understanding features and the question text matching the actual song annotation information into the big language module in the song recommendation model, and obtain the predicted song annotation information through the big language module.

[0117] The training module 508 is used to utilize the difference between the predicted song tagging information and the actual song tagging information to train the music understanding feature extraction module in the song recommendation model to obtain a trained song recommendation model.

[0118] In one of the exemplary embodiments, the music understanding feature extraction module includes a song audio feature extraction model and a feature conversion model to be trained, and the extraction module 504 is specifically used to input the sample song into the song audio feature extraction model to obtain the song audio features of the sample song; input the song audio features of the sample song into the feature conversion model to be trained to obtain the music understanding features of the sample song.

[0119] In one of the exemplary embodiments, the extraction module 504 is specifically used to perform convolution processing on the song audio features of the sample song through the feature conversion model to be trained to obtain a convolution result; and input the convolution result into the conversion module in the feature conversion model to be trained to obtain the music understanding features of the sample song.

[0120] In one of the exemplary embodiments, the extraction module 504 is specifically used to input the convolution result into the first fully connected layer, and map the output result of the first fully connected layer according to a preset activation function to obtain first feature information; input the convolution result into the second fully connected layer to obtain second feature information; input the product of the first feature information and the second feature information into the third fully connected layer to obtain third feature information; add the third feature information and the convolution result to obtain the music understanding feature of the sample song.

[0121] In one of the exemplary embodiments, the prediction module 506 is further specifically used to obtain a question text that matches the actual song annotation information; input the question text and the music understanding features into the large language module to obtain the predicted song annotation information.

[0122] In one of the exemplary embodiments, the acquisition module 502 is specifically used to acquire a song data set; the song data set includes a number of candidate songs; the candidate songs in the song data set are sorted according to the song popularity information of the candidate songs to obtain a sorting result; and the sample song is filtered out from the sorting result.

[0123] Each module in the processing device of the song recommendation model can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0124] In an exemplary embodiment, a song recommendation device is provided, comprising: a feature extraction module and a result generation module, wherein:

[0125] A feature extraction module is used to input the song to be recommended into the music understanding feature extraction module in the pre-trained song recommendation model to obtain the music understanding features corresponding to the song to be recommended; wherein the song recommendation model is trained by the processing method of the song recommendation model.

[0126] The result generation module is used to determine the song recommendation result of the song to be recommended according to the music understanding features corresponding to the song to be recommended and the sample songs corresponding to the user; the song recommendation result indicates whether the song to be recommended is recommended to the user.

[0127] In one of the exemplary embodiments, the sample songs include positive sample songs and negative sample songs, and the result generation module is specifically used to obtain the music understanding features of the positive sample songs and the music understanding features of the negative sample songs corresponding to the user through the music understanding feature extraction module in the pre-trained song recommendation model; the positive sample songs are songs whose operation type is the first type in the user's listening behavior data; the negative sample songs are songs whose operation type is the second type in the user's listening behavior data; the first type is different from the second type; the song recommendation result of the song to be recommended is determined according to the music understanding features corresponding to the song to be recommended, the music understanding features of the positive sample songs and the music understanding features of the negative sample songs.

[0128] Each module in the above song recommendation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0129] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a processing method for a song recommendation model and / or a group of song recommendation methods are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0130] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0131] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0132] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0133] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0134] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0135] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0137] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for processing a song recommendation model, characterized in that: The method comprises: Obtaining sample songs and actual song annotation information associated with the sample songs; Inputting the sample song into a music understanding feature extraction module in a song recommendation model to be trained, and obtaining music understanding features of the sample song through the music understanding feature extraction module; the music understanding features are used to characterize the understanding of the music content of the sample song; Inputting the music comprehension features and the question text matching the actual song annotation information into the big language module in the song recommendation model, and obtaining the predicted song annotation information through the big language module; The difference between the predicted song tagging information and the actual song tagging information is used to train the music understanding feature extraction module in the song recommendation model to obtain a trained song recommendation model.

2. The method according to claim 1, characterized in that The music understanding feature extraction module includes a pre-trained song audio feature extraction model and a feature conversion model to be trained, and the music understanding features of the sample song are obtained by the music understanding feature extraction module, including: Inputting the sample song into the song audio feature extraction model to obtain the song audio features of the sample song; The song audio features of the sample song are input into the feature conversion model to be trained to obtain the music understanding features of the sample song.

3. The method according to claim 2, characterized in that The step of inputting the song audio features of the sample song into the feature conversion model to be trained to obtain the music understanding features of the sample song includes: Performing convolution processing on the song audio features of the sample song through the feature conversion model to be trained to obtain a convolution result; The convolution result is input into the conversion module in the feature conversion model to be trained to obtain the music understanding features of the sample song.

4. The method according to claim 3, characterized in that The conversion module includes a first fully connected layer, a second fully connected layer and a third fully connected layer, and the conversion module inputs the convolution result into the feature conversion model to be trained to obtain the music understanding features of the sample song, including: Inputting the convolution result into the first fully connected layer, and mapping the output result of the first fully connected layer according to a preset activation function to obtain first feature information; Inputting the convolution result into the second fully connected layer to obtain second feature information; Inputting the product of the first feature information and the second feature information into the third fully connected layer to obtain third feature information; The third feature information and the convolution result are added to obtain the music understanding feature of the sample song.

5. The method according to claim 1, characterized in that The step of obtaining a sample song comprises: Acquire a song data set; the song data set includes a number of candidate songs; Sorting the candidate songs in the song data set according to the song popularity information of the candidate songs to obtain a sorting result; The sample songs are selected from the sorting results.

6. A song recommendation method, characterized in that: The method comprises: Input the song to be recommended into the music understanding feature extraction module in the pre-trained song recommendation model to obtain the music understanding feature corresponding to the song to be recommended; wherein the song recommendation model is trained by the processing method of the song recommendation model described in any one of claims 1 to 5; A song recommendation result of the song to be recommended is determined based on the music understanding features corresponding to the song to be recommended and the sample songs corresponding to the user; the song recommendation result indicates whether the song to be recommended is recommended to the user.

7. The method according to claim 6, characterized in that The sample songs include positive sample songs and negative sample songs, and determining the song recommendation result of the song to be recommended according to the music understanding features corresponding to the song to be recommended and the sample songs corresponding to the user includes: The music understanding features of the positive sample songs and the music understanding features of the negative sample songs corresponding to the user are obtained through the music understanding feature extraction module in the pre-trained song recommendation model; the positive sample songs are songs of the first type of operation type in the listening behavior data of the user; the negative sample songs are songs of the second type of operation type in the listening behavior data of the user; the first type is different from the second type; The song recommendation result of the song to be recommended is determined according to the music understanding features corresponding to the song to be recommended, the music understanding features of the positive sample song and the music understanding features of the negative sample song.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images