Model training method, song recommendation method, device, storage medium, and program product

Abstract

The application relates to a model training method, which comprises the following steps: obtaining a plurality of song triplets; each song triplet comprises two sample songs collected by a sample user and one sample song not collected by a sample user; inputting the plurality of song triplets into a trained music understanding large model to extract first audio features of the sample songs in the plurality of song triplets, and determining a first feature similarity matrix based on the first audio features; inputting the plurality of song triplets into a to-be-trained song recommendation model to extract second audio features of the sample songs in the plurality of song triplets, and determining a second feature similarity matrix based on the second audio features; determining a relationship distillation loss according to the first feature similarity matrix and the second feature similarity matrix, and determining a triple loss according to the second audio features; training the song recommendation model based on the relationship distillation loss and the triple loss; and using the trained song recommendation model for song recommendation. The method can improve the accuracy of song recommendation.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model training method, a song recommendation method, an apparatus, a storage medium, and a program product. Background Technology

[0002] With the rapid rise and development of large-scale model technology, the bridge between machines and humans is becoming increasingly clear. Human-computer dialogue has reached an unprecedented level of naturalness. We can ask questions to large-scale models, and they can quickly return high-quality answers, eliminating the need to painstakingly sift through search results for useful information as before. Although large-scale models can already meet people's diverse needs in life and work, there is still much room for development in deeper content understanding, such as in the music field. While some music understanding foundational large-scale models are currently active in the industry and can bring better results to downstream music tasks, these foundational models each have their own strengths and weaknesses. Some perform better in genre classification, while others perform better in timbre tasks. However, no single foundational representation large-scale model is suitable for song recommendation scenarios because song recommendation is related to user preferences. The model needs to learn from users' song preferences in order to acquire the ability to understand those preferences.

[0003] Currently, such as Figure 1 As shown, after the large music understanding model is trained, the downstream recommendation model is mainly trained in an unsupervised manner. The learned features can well represent the audio characteristics. However, when used in the song recommendation scenario, only the original large music understanding model is frozen and its representation is extracted. The downstream recommendation model only learns the recommendation features. This leads to a disconnect between the two learning stages. When learning the recommendation task, the downstream recommendation model "forgets" the learning of song understanding, resulting in an unsatisfactory performance of the final trained recommendation model and a low accuracy of song recommendation. Summary of the Invention

[0004] Therefore, it is necessary to provide a model training method, song recommendation method, device, storage medium, and program product that can improve the accuracy of song recommendation in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a model training method, the method comprising:

[0006] Obtain multiple song triples; each song triple contains two sample songs collected by the sample user and one sample song not collected by the sample user;

[0007] The multiple song triples are input into the trained music understanding big model to extract the first audio features of the sample songs in the multiple song triples respectively, and the first feature similarity matrix corresponding to all sample songs in the multiple song triples is determined based on the first audio features of each sample song.

[0008] The multiple song triples are input into the song recommendation model to be trained, so as to extract the second audio features of the sample songs in the multiple song triples respectively, and determine the second feature similarity matrix corresponding to all sample songs in the multiple song triples based on the second audio features of each sample song.

[0009] The relation distillation loss corresponding to the plurality of song triples is determined based on the first feature similarity matrix and the second feature similarity matrix, and the triplet loss corresponding to the plurality of song triples is determined based on the second audio features of each of the sample songs.

[0010] The song recommendation model is trained based on the relation distillation loss and the ternary loss to obtain a trained song recommendation model; the trained song recommendation model is used for song recommendation.

[0011] In one embodiment, both the first feature similarity matrix and the second feature similarity matrix are symmetric matrices with the same dimension; the elements in the first feature similarity matrix represent the similarity between any two first audio features; and the elements in the second feature similarity matrix represent the similarity between any two second audio features.

[0012] The step of determining the relation distillation loss corresponding to the plurality of song triples based on the first feature similarity matrix and the second feature similarity matrix includes:

[0013] Determine the mean square error between the positions of each element in the first feature similarity matrix and the second feature similarity matrix;

[0014] Based on the mean squared error between each element position in the first feature similarity matrix and the second feature similarity matrix, the relation distillation loss corresponding to the multiple song triples is determined.

[0015] In one embodiment, determining the triplet loss corresponding to the plurality of song triplets based on the second audio features of each of the sample songs includes:

[0016] For each song triplet, the positive sample similarity of the song triplet is determined based on the similarity between the second audio feature of the sample song collected by the sample user in the song triplet and the anchor feature of the preset anchor sample.

[0017] For each of the song triples, the negative sample similarity of the song triple is determined based on the similarity between the second audio feature of the sample song collected by the non-sample user in the song triple and the anchor feature.

[0018] Based on the positive and negative sample similarities of each song triplet, the triplet loss corresponding to the multiple song triplets is determined.

[0019] In one embodiment, the positive sample similarity of the song triples includes a first positive sample similarity and a second positive sample similarity;

[0020] The step of determining the triplet loss corresponding to the plurality of song triplets based on the positive sample similarity and negative sample similarity of each song triplet includes:

[0021] For each song triplet, the first loss parameter value is determined based on the difference between the first positive sample similarity and the negative sample similarity of the song triplet.

[0022] The second loss parameter value is determined based on the difference between the second positive sample similarity and the negative sample similarity of the song triples.

[0023] The larger of the first loss parameter value and the second loss parameter value is taken as the target loss parameter value of the song triplet.

[0024] Based on the target loss parameter value of each song triplet, the triplet loss corresponding to the plurality of song triplets is determined.

[0025] In one embodiment, training the song recommendation model based on the relation distillation loss and the ternary loss to obtain the trained song recommendation model includes:

[0026] A first weighting parameter for the relational distillation loss is determined, a second weighting parameter for the ternary loss is determined, and the relational distillation loss and the ternary loss are weighted and fused based on the first weighting parameter and the second weighting parameter to obtain a fused loss;

[0027] The song recommendation model is trained based on the fusion loss to obtain the trained song recommendation model.

[0028] In one embodiment, the method further includes:

[0029] Extract the first spectral features of the sample songs from the multiple song triples, and convert the first spectral features of the first dimension into the second spectral features of the second dimension;

[0030] The second spectral feature is randomly masked at a fixed ratio to obtain the masked feature;

[0031] The mask features are encoded to obtain the first encoded feature of the second dimension;

[0032] The first encoded feature is expanded in dimension to obtain a second encoded feature in the first dimension.

[0033] Decode the second encoded feature to obtain the first embedded feature;

[0034] The sample songs in the multiple song triples are encoded to obtain the second embedding feature;

[0035] The training loss is determined based on the first embedding feature and the second embedding feature;

[0036] Based on the training loss, the large music understanding model to be trained is trained to obtain the trained large music understanding model.

[0037] Secondly, this application provides a song recommendation method, the method comprising:

[0038] Obtain at least one candidate song to be recommended;

[0039] Obtain at least one preferred song from the target user's collection;

[0040] The at least one candidate song and the at least one preferred song are input into a trained song recommendation model to extract the candidate audio features of the at least one candidate song and the preferred audio features of the at least one preferred song, respectively; the song recommendation model is trained by the model training method of this application.

[0041] For each candidate song, the similarity between the candidate audio features of the candidate song and the preferred audio features of the at least one preferred song is determined to obtain the feature similarity corresponding to the candidate song.

[0042] Based on the feature similarity of each candidate song, a target song is selected from the at least one candidate song and recommended to the target user.

[0043] Thirdly, this application provides a model training apparatus, the apparatus comprising:

[0044] The first acquisition module is used to acquire multiple song triples; each song triple contains two sample songs collected by the sample user and one sample song not collected by the sample user.

[0045] The first extraction module is used to input the multiple song triples into a trained music understanding model to extract the first audio features of the sample songs from the multiple song triples.

[0046] The first determining module is used to determine the first feature similarity matrix corresponding to all sample songs in the plurality of song triplets based on the first audio features of each of the sample songs.

[0047] The first extraction module is also used to input the multiple song triples into the song recommendation model to be trained, so as to extract the second audio features of the sample songs in the multiple song triples respectively;

[0048] The first determining module is further configured to determine the second feature similarity matrix corresponding to all sample songs in the plurality of song triples based on the second audio features of each of the sample songs; determine the relation distillation loss corresponding to the plurality of song triples based on the first feature similarity matrix and the second feature similarity matrix; and determine the triple loss corresponding to all sample songs in the plurality of song triples based on the second audio features of each of the sample songs.

[0049] The training module is used to train the song recommendation model based on the relation distillation loss and the ternary loss to obtain the trained song recommendation model; the trained song recommendation model is used for song recommendation.

[0050] Fourthly, this application provides a song recommendation device, the device comprising:

[0051] The first acquisition module is used to acquire at least one candidate song to be recommended; and to acquire at least one preferred song collected by the target user.

[0052] The second extraction module is used to input the at least one candidate song and the at least one preferred song into a trained song recommendation model, so as to extract the candidate audio features of the at least one candidate song and the preferred audio features of the at least one preferred song respectively; the song recommendation model is trained by the model training method of this application;

[0053] The second determining module is used to determine the similarity between the candidate audio features of each candidate song and the preferred audio features of each of the at least one preferred song for each candidate song, and to obtain the feature similarity corresponding to the candidate song.

[0054] The recommendation module is used to filter target songs from the at least one candidate song based on the feature similarity of each candidate song, and recommend the target song to the target user.

[0055] Fifthly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the various method embodiments of this application.

[0056] In a sixth aspect, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the various method embodiments of this application.

[0057] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the various method embodiments of this application.

[0058] The aforementioned model training method, song recommendation method, device, storage medium, and program products construct triplet samples containing user preference information and introduce a joint training mechanism of relation distillation loss and triplet loss. This allows the song recommendation model to retain the original audio feature extraction capabilities of the large-scale music understanding model while specifically learning the personalized preference features of users. This method solves the problem of the separation between the recommendation task and the music understanding task in traditional two-stage training, avoids the recommendation model "forgetting" key audio representation information during the learning process, improves the depth of the recommendation model's understanding of user preferences, and thus improves the accuracy of song recommendation. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating a traditional model training method in one embodiment;

[0061] Figure 2 This is a diagram illustrating the application environment of a model training method in one embodiment;

[0062] Figure 3 This is a flowchart illustrating a model training method in one embodiment;

[0063] Figure 4 This is a schematic diagram of the training process of a large music understanding model in one embodiment;

[0064] Figure 5 This is a flowchart illustrating the model training method of this application in one embodiment;

[0065] Figure 6This is a flowchart illustrating a song recommendation method in one embodiment;

[0066] Figure 7 This is a structural block diagram of a model training device in one embodiment;

[0067] Figure 8 This is a structural block diagram of the model training device in another embodiment;

[0068] Figure 9 This is an internal structural diagram of a computer device in one embodiment;

[0069] Figure 10 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0071] The model training method provided in this application can be applied to, for example... Figure 2 In the application environment shown, terminal 202 communicates with server 204 via a network. The data storage system can be set up separately and can store the data that server 204 needs to process. The data storage system can be integrated into server 204 or placed in the cloud or on other servers. Terminal 202 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, vehicle terminals, etc. Server 204 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, cloud security, host security and other network security services, CDN, and basic cloud computing services such as big data and artificial intelligence platforms. Terminal 202 and server 204 can be directly or indirectly connected via wired or wireless communication; this application does not impose any restrictions.

[0072] Server 204 can obtain multiple song triples; each song triple contains two sample songs collected by sample users and one sample song not collected by sample users; Server 204 can input multiple song triples into a trained music understanding model to extract the first audio features of the sample songs in the multiple song triples, and determine the first feature similarity matrix corresponding to all sample songs in the multiple song triples based on the first audio features of each sample song; Server 204 can input multiple song triples into a song recommendation model to be trained to extract the second audio features of the sample songs in the multiple song triples, and determine the second feature similarity matrix corresponding to all sample songs in the multiple song triples based on the second audio features of each sample song; Server 204 can determine the relation distillation loss corresponding to the multiple song triples based on the first feature similarity matrix and the second feature similarity matrix, and determine the triple loss corresponding to the multiple song triples based on the second audio features of each sample song; Server 204 can train the song recommendation model based on the relation distillation loss and the triple loss to obtain a trained song recommendation model; The trained song recommendation model is used to recommend songs to terminal 202.

[0073] It is understood that this embodiment does not impose any limitations on this. Figure 2 The application scenarios shown are for illustrative purposes only and are not limited to these.

[0074] In one embodiment, such as Figure 3 As shown, a model training method is provided. This method can be applied to a computer device, which can be a terminal or a server. That is, the method can be executed independently by the terminal or the server, or it can be implemented through interaction between the terminal and the server. This embodiment illustrates the application of this method to a computer device as an example, including the following steps:

[0075] Step 302: Obtain multiple song triples; each song triple contains two sample songs collected by sample users and one sample song collected by non-sample users.

[0076] It is understandable that song triples can represent a user's preference for songs. Preference association is represented by two songs that are simultaneously collected. That is, if a user collects both songs A and B, it indicates that there is a preference association between the two songs. If a user does not collect song C, it indicates that this song is not a song that the user prefers.

[0077] Step 304: Input multiple song triples into the trained music understanding big model to extract the first audio features of the sample songs in the multiple song triples respectively, and determine the first feature similarity matrix corresponding to all sample songs in the multiple song triples based on the first audio features of each sample song.

[0078] The first feature similarity matrix is ​​a symmetric matrix; the elements in the first feature similarity matrix represent the similarity between any two first audio features.

[0079] In one embodiment, a computer device can input multiple song triples into a music understanding big model to be trained, so as to perform self-supervised training on the music understanding big model using sample songs from the multiple song triples, and obtain a trained music understanding big model.

[0080] Step 306: Input multiple song triples into the song recommendation model to be trained, so as to extract the second audio features of the sample songs in the multiple song triples respectively, and determine the second feature similarity matrix corresponding to all sample songs in the multiple song triples based on the second audio features of each sample song.

[0081] The second feature similarity matrix is ​​a symmetric matrix; the first and second feature similarity matrices have the same dimension; and the elements in the second feature similarity matrix represent the pairwise similarity between the second audio features. This can be understood as the trained music understanding model serving as the teacher model, and the song recommendation model serving as the student model.

[0082] Step 308: Based on the first feature similarity matrix and the second feature similarity matrix, determine the relational distillation loss corresponding to multiple song triples; and based on the second audio features of each sample song, determine the triplet loss corresponding to multiple song triples.

[0083] In one embodiment, a computer device can determine the relational distillation loss corresponding to multiple song triples based on the similarity between pairs of first audio features in a first feature similarity matrix and the similarity between pairs of second audio features in a second feature similarity matrix.

[0084] In one embodiment, for each song triplet, a triplet loss is determined based on the second audio features of each sample song in the song triplet.

[0085] Step 310: Train a song recommendation model based on relation distillation loss and ternary loss to obtain a trained song recommendation model; the trained song recommendation model is used for song recommendation.

[0086] Compared to traditional model training methods, the above-described model training method constructs triplet samples containing user preference information and introduces a joint training mechanism of relation distillation loss and triplet loss. This allows the song recommendation model to retain the original audio feature extraction capabilities of the large-scale music understanding model while specifically learning the personalized preference features of users. This method solves the problem of the separation between the recommendation task and the music understanding task in traditional two-stage training, avoids the recommendation model "forgetting" key audio representation information during the learning process, improves the depth of the recommendation model's understanding of user preferences, and thus improves the accuracy of song recommendations.

[0087] In one embodiment, both the first feature similarity matrix and the second feature similarity matrix are symmetric matrices with the same dimension; the elements in the first feature similarity matrix represent the similarity between any two first audio features; the elements in the second feature similarity matrix represent the similarity between any two second audio features; and the relation distillation loss corresponding to multiple song triples is determined based on the first feature similarity matrix and the second feature similarity matrix, including: determining the mean squared error between each element position in the first feature similarity matrix and the second feature similarity matrix; and determining the relation distillation loss corresponding to multiple song triples based on the mean squared error between each element position in the first feature similarity matrix and the second feature similarity matrix.

[0088] Among them, relational distillation loss allows the student model (i.e., the song recommendation model) to understand the correlation and differences between different songs from the teacher model (i.e., the music understanding big model).

[0089] In one embodiment, the relationship distillation loss It can be calculated using the following formula:

[0090]

[0091] in, This represents the first feature similarity matrix. This represents the second feature similarity matrix. In matrix T, ... Let T represent the cosine similarity between the i-th and j-th songs among all songs input into the teacher model (i.e., the large music understanding model). Matrix T exhibits a symmetric relationship, i.e. = And when i=j, = =1. In matrix S Let represent the cosine similarity between the i-th and j-th songs among all songs input into the student model (i.e., the song recommendation model). The matrix exhibits a symmetric relationship. = And when i=j, = =1.

[0092] In the above embodiments, by constructing symmetric similarity matrices of the same dimension and calculating the mean squared error of corresponding element positions in the two matrices to determine the relationship distillation loss, precise alignment of the music understanding model and the song recommendation model at the feature relationship level is achieved. This method can effectively "distill" the internal structural relationships of audio features learned by the large model and pass them to the recommendation model, ensuring that the recommendation model strictly maintains the relative similarity structure of the original audio representation when learning user preferences, thereby avoiding feature distortion and improving recommendation accuracy.

[0093] In one embodiment, determining the triplet loss corresponding to multiple song triplets based on the second audio features of each sample song includes: for each song triplet, determining the positive sample similarity of the song triplet based on the similarity between the second audio features of the sample songs collected by the sample user in the song triplet and the anchor features of the preset anchor samples; for each song triplet, determining the negative sample similarity of the song triplet based on the similarity between the second audio features of the sample songs collected by non-sample users in the song triplet and the anchor features; and determining the triplet loss corresponding to multiple song triplets based on the positive and negative sample similarities of each song triplet.

[0094] In one embodiment, ternary loss It can be calculated using the following formula:

[0095]

[0096] in, This represents the second audio feature of the sample songs (i.e., positive samples) collected by the sample user in the song triplet. This represents the second audio feature of sample songs collected by non-sample users (i.e., negative samples) in the song triplet. This represents the anchor point features of the preset anchor point sample. The cosine similarity of negative samples is represented by... Let M represent the cosine similarity of the positive samples, M represent the number of song triples, and m represent the index value of the song triples. It is a preset fixed value, such as 0.4.

[0097] In the above embodiments, a ternary loss is constructed by calculating the similarity between the anchor point and positive and negative samples. This allows the recommendation model to bring user-favorited songs closer together in the feature space, while simultaneously distancing them from non-favorited songs. This mechanism accurately characterizes user personalized preferences, strengthens the model's ability to identify preference features, and thus improves the accuracy and matching effect of song recommendations.

[0098] In one embodiment, the positive sample similarity of the song triples includes a first positive sample similarity and a second positive sample similarity; according to the positive sample similarity and negative sample similarity of each song triple, determining the triple loss corresponding to multiple song triples, including: for each song triple respectively, determining a first loss parameter value according to the difference between the first positive sample similarity and the negative sample similarity of the song triple, determining a second loss parameter value according to the difference between the second positive sample similarity and the negative sample similarity of the song triple, and taking the larger one of the first loss parameter value and the second loss parameter value as the target loss parameter value of the song triple; determining the triple loss corresponding to multiple song triples according to the target loss parameter values of each song triple.

[0099] In the above embodiment, by selecting the larger value of the differences between the first and second positive sample similarities and the negative sample similarity as the target loss parameter, it is possible to focus on the most difficult-to-distinguish sample pairs in the training process and strengthen the model's identification ability for boundary samples. This mechanism effectively improves the robustness and accuracy of the recommendation model in complex preference scenarios, ensuring that the model can more stably learn the user's personalized features.

[0100] In one embodiment, training a song recommendation model based on the relational distillation loss and the triple loss to obtain a trained song recommendation model, including: determining a first weight parameter of the relational distillation loss, determining a second weight parameter of the triple loss, and weighted-fusing the relational distillation loss and the triple loss according to the first weight parameter and the second weight parameter to obtain a fused loss; training the song recommendation model based on the fused loss to obtain a trained song recommendation model.

[0101] In one embodiment, the fused loss Loss can be calculated by the following formula:

[0102] Loss = a * Loss_mae + (1 - a) * Loss_recommend

[0103] Where a represents the first weight parameter, (1 - a) represents the second weight parameter, and 0 < a < 1 is used to adjust the proportion of the two in the process of training the student model (i.e., the song recommendation model).

[0104] In the above embodiment, by weighted-fusing the relational distillation loss and the triple loss, a dynamic balance between audio feature preservation and user preference learning is achieved. By adjusting the weight parameters, different application scenarios can be flexibly adapted, ensuring that the recommendation model can accurately capture personalized needs while inheriting the music understanding ability, thereby improving the accuracy and scenario adaptability of song recommendations.

[0105] In one embodiment, the method further includes: extracting first spectral features of sample songs from multiple song triples, and converting the first spectral features of the first dimension into second spectral features of the second dimension; performing a fixed-ratio random masking operation on the second spectral features to obtain masked features; performing an encoding operation on the masked features to obtain first encoded features of the second dimension; performing a dimensional expansion operation on the first encoded features to obtain second encoded features of the first dimension; decoding the second encoded features to obtain first embedding features; performing an encoding operation on sample songs from multiple song triples to obtain second embedding features; determining a training loss based on the first embedding features and the second embedding features; and training the large music understanding model to be trained based on the training loss to obtain a trained large music understanding model.

[0106] In one embodiment, such as Figure 4 As shown, the computer device can input sample songs from multiple song triples into a large-scale music understanding model to be trained. The feature extraction layer in the music understanding model extracts the first spectral features of the sample songs from the multiple song triples. These first-dimensional spectral features are then input into a linear transformation layer in the music understanding model to transform them into second-dimensional spectral features. The second spectral features are then masked with a fixed ratio of randomness to obtain masked features. These masked features are then input into the MAE encoder in the music understanding model to encode the masked features, obtaining the first-dimensional encoded features. The first-dimensional encoded features are then expanded to obtain the second-dimensional encoded features. These second-dimensional encoded features are then input into the MAE decoder in the music understanding model to decode the second-dimensional encoded features, obtaining the first embedding features. The sample songs from the multiple song triples are then input into the EnCodec encoder in the music understanding model to encode the sample songs and perform residual vector quantization, obtaining the second embedding features. The training loss is determined based on the first and second embedding features.

[0107] In the above embodiments, by transforming the spectral feature dimension and using a random masking mechanism, the model is forced to learn the deep structure of audio under conditions of missing information. Combining the encoder-decoder architecture with embedded feature comparison allows the model to fully explore the potential features of the audio during reconstruction, enhancing robustness and generalization ability. Ultimately, a large-scale music understanding model with more accurate feature representation is obtained, providing a high-quality audio representation foundation for downstream recommendation tasks.

[0108] In one embodiment, such as Figure 5As shown, this application can use the ResNet18 model as the song recommendation model (i.e., the student model). A fully connected network is connected after the ResNet18 model to perform dimensionality transformation. The purpose of dimensionality transformation is to keep the output dimension consistent with the teacher model (i.e., the music understanding big model) to facilitate loss calculation alignment. Specifically, the computer device can acquire multiple song triples (e.g., n song triples); each song triple contains two sample songs collected by sample users (i.e., collected song 1 and collected song 2) and one sample song not collected by sample users (i.e., a non-collected song); the multiple song triples are input into a trained music understanding model (i.e., the teacher model) to extract the first audio features of the sample songs in the multiple song triples, and a first feature similarity matrix (i.e., the teacher model cosine similarity matrix T) is determined based on each first audio feature; the multiple song triples are input into a song recommendation model to be trained (i.e., the student model) to extract the second audio features of the sample songs in the multiple song triples, and a second feature similarity matrix (i.e., the student model cosine similarity matrix S) is determined based on each second audio feature; the relation distillation loss is determined based on the first feature similarity matrix and the second feature similarity matrix, and a triple loss is determined based on each second audio feature; the song recommendation model is trained based on the relation distillation loss and the triple loss to obtain the trained song recommendation model.

[0109] In one embodiment, such as Figure 6 As shown, a song recommendation method is provided. This method can be applied to computer devices, which can be terminals or servers. That is, the method can be executed independently by the terminal or the server, or it can be implemented through interaction between the terminal and the server. This embodiment illustrates the application of this method to a computer device as an example, including the following steps:

[0110] Step 602: Obtain at least one candidate song to be recommended.

[0111] Step 604: Obtain at least one favorite song from the target user's collection.

[0112] Step 606: Input at least one candidate song and at least one preferred song into the trained song recommendation model to extract the candidate audio features of the at least one candidate song and the preferred audio features of the at least one preferred song respectively; the song recommendation model is trained by the model training method of this application.

[0113] Step 608: For each candidate song, determine the similarity between the candidate audio features of the candidate song and the preferred audio features of at least one preferred song, and obtain the feature similarity corresponding to the candidate song.

[0114] Step 610: Based on the feature similarity of each candidate song, select the target song from at least one candidate song and recommend the target song to the target user.

[0115] The aforementioned song recommendation method constructs triplet samples containing user preference information and introduces a joint training mechanism combining relation distillation loss and triplet loss. This allows the song recommendation model to retain the original audio feature extraction capabilities of the large-scale music understanding model while specifically learning the user's personalized preference features. This method solves the problem of the separation between the recommendation task and the music understanding task in traditional two-stage training, preventing the recommendation model from "forgetting" key audio representation information during the learning process, improving the depth of the recommendation model's understanding of user preferences, and thus improving the accuracy of song recommendations.

[0116] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially, these steps are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the above embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0117] Based on the same inventive concept, this application also provides a model training apparatus for implementing the model training method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more model training apparatus embodiments provided below can be found in the limitations of the model training method described above, and will not be repeated here.

[0118] In one embodiment, such as Figure 7 As shown, a model training device 700 is provided, which specifically includes:

[0119] The first acquisition module 702 is used to acquire multiple song triples; each song triple contains two sample songs collected by sample users and one sample song not collected by sample users.

[0120] The first extraction module 704 is used to input multiple song triples into a trained music understanding model to extract the first audio features of sample songs from the multiple song triples.

[0121] The first determining module 706 is used to determine the first feature similarity matrix corresponding to all sample songs in multiple song triplets based on the first audio features of each sample song;

[0122] The first extraction module 704 is also used to input multiple song triples into the song recommendation model to be trained, so as to extract the second audio features of the sample songs in the multiple song triples respectively.

[0123] The first determining module 706 is further configured to determine the second feature similarity matrix corresponding to all sample songs in the multiple song triplet based on the second audio features of each sample song; determine the relation distillation loss corresponding to the multiple song triplet based on the first feature similarity matrix and the second feature similarity matrix; and determine the triplet loss corresponding to the multiple song triplet based on the second audio features of each sample song.

[0124] Training module 708 is used to train a song recommendation model based on relation distillation loss and ternary loss to obtain a trained song recommendation model; the trained song recommendation model is used for song recommendation.

[0125] In one embodiment, both the first feature similarity matrix and the second feature similarity matrix are symmetric matrices with the same dimension; the elements in the first feature similarity matrix represent the similarity between any two first audio features; the elements in the second feature similarity matrix represent the similarity between any two second audio features; the first determining module 706 is further configured to determine the mean square error between each element position in the first feature similarity matrix and the second feature similarity matrix; and to determine the relational distillation loss corresponding to multiple song triples based on the mean square error between each element position in the first feature similarity matrix and the second feature similarity matrix.

[0126] In one embodiment, the first determining module 706 is further configured to, for each song triplet, determine the positive sample similarity of the song triplet based on the similarity between the second audio features of the sample songs collected by the sample users in the song triplet and the anchor features of the preset anchor samples; determine the negative sample similarity of the song triplet based on the similarity between the second audio features of the sample songs collected by non-sample users in the song triplet and the anchor features; and determine the triplet loss corresponding to multiple song triplets based on the positive and negative sample similarities of each song triplet.

[0127] In one embodiment, the positive sample similarity of the song triplet includes a first positive sample similarity and a second positive sample similarity; the first determining module 706 is further configured to, for each song triplet, determine a first loss parameter value based on the difference between the first positive sample similarity and the negative sample similarity of the song triplet, determine a second loss parameter value based on the difference between the second positive sample similarity and the negative sample similarity of the song triplet, and take the larger of the first loss parameter value and the second loss parameter value as the target loss parameter value of the song triplet; and determine the triplet loss corresponding to each song triplet based on the target loss parameter value of each song triplet.

[0128] In one embodiment, the training module 708 is further configured to determine a first weight parameter of the relation distillation loss, determine a second weight parameter of the ternary loss, and, based on the first weight parameter and the second weight parameter, perform a weighted fusion of the relation distillation loss and the ternary loss to obtain a fusion loss; and train a song recommendation model based on the fusion loss to obtain a trained song recommendation model.

[0129] In one embodiment, the training module 708 is further configured to extract first spectral features of sample songs from multiple song triplets, and convert the first spectral features of the first dimension into second spectral features of the second dimension; perform a fixed-ratio random masking on the second spectral features to obtain masked features; perform an encoding operation on the masked features to obtain first encoded features of the second dimension; perform a dimensional expansion operation on the first encoded features to obtain second encoded features of the first dimension; decode the second encoded features to obtain first embedding features; perform an encoding operation on sample songs from multiple song triplets to obtain second embedding features; determine the training loss based on the first embedding features and the second embedding features; and train the large music understanding model to be trained based on the training loss to obtain the trained large music understanding model.

[0130] The aforementioned model training device constructs triplet samples containing user preference information and introduces a joint training mechanism of relation distillation loss and triplet loss. This allows the song recommendation model to retain the original audio feature extraction capabilities of the large-scale music understanding model while specifically learning the personalized preference features of users. This method solves the problem of the separation between the recommendation task and the music understanding task in traditional two-stage training, avoids the recommendation model "forgetting" key audio representation information during the learning process, improves the depth of the recommendation model's understanding of user preferences, and thus improves the accuracy of song recommendations.

[0131] In one embodiment, such as Figure 8 As shown, a song recommendation device 800 is provided, which specifically includes:

[0132] The second acquisition module 802 is used to acquire at least one candidate song to be recommended; and to acquire at least one preferred song collected by the target user.

[0133] The second extraction module 804 is used to input at least one candidate song and at least one preferred song into the trained song recommendation model, so as to extract the candidate audio features of the at least one candidate song and the preferred audio features of the at least one preferred song respectively; the song recommendation model is trained by the model training method of this application;

[0134] The second determining module 806 is used to determine the similarity between the candidate audio features of each candidate song and the preferred audio features of at least one preferred song for each candidate song, and to obtain the feature similarity corresponding to the candidate songs.

[0135] The recommendation module 808 is used to select target songs from at least one candidate song based on the feature similarity of each candidate song, and recommend the target song to the target user.

[0136] The aforementioned model training device constructs triplet samples containing user preference information and introduces a joint training mechanism of relation distillation loss and triplet loss. This allows the song recommendation model to retain the original audio feature extraction capabilities of the large-scale music understanding model while specifically learning the personalized preference features of users. This method solves the problem of the separation between the recommendation task and the music understanding task in traditional two-stage training, avoids the recommendation model "forgetting" key audio representation information during the learning process, improves the depth of the recommendation model's understanding of user preferences, and thus improves the accuracy of song recommendations.

[0137] Each module in the aforementioned model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0138] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a model training method.

[0139] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a model training method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0140] Those skilled in the art will understand that Figure 9 and Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0142] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0143] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0145] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0146] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0147] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model training method, characterized in that, The method includes: Obtain multiple song triples; each song triple contains two sample songs collected by the sample user and one sample song not collected by the sample user; The multiple song triples are input into the trained music understanding big model to extract the first audio features of the sample songs in the multiple song triples respectively, and the first feature similarity matrix corresponding to all sample songs in the multiple song triples is determined based on the first audio features of each sample song. The multiple song triples are input into the song recommendation model to be trained, so as to extract the second audio features of the sample songs in the multiple song triples respectively, and determine the second feature similarity matrix corresponding to all sample songs in the multiple song triples based on the second audio features of each sample song. The relation distillation loss corresponding to the plurality of song triples is determined based on the first feature similarity matrix and the second feature similarity matrix, and the triplet loss corresponding to the plurality of song triples is determined based on the second audio features of each of the sample songs; The song recommendation model is trained based on the relation distillation loss and the ternary loss to obtain a trained song recommendation model; the trained song recommendation model is used for song recommendation.

2. The method according to claim 1, characterized in that, Both the first feature similarity matrix and the second feature similarity matrix are symmetric matrices with the same dimension; the elements in the first feature similarity matrix represent the similarity between any two first audio features; the elements in the second feature similarity matrix represent the similarity between any two second audio features. The step of determining the relation distillation loss corresponding to the plurality of song triples based on the first feature similarity matrix and the second feature similarity matrix includes: Determine the mean square error between the positions of each element in the first feature similarity matrix and the second feature similarity matrix; Based on the mean squared error between each element position in the first feature similarity matrix and the second feature similarity matrix, the relation distillation loss corresponding to the multiple song triples is determined.

3. The method according to claim 1, characterized in that, The step of determining the triplet loss corresponding to the plurality of song triplets based on the second audio features of each of the sample songs includes: For each song triplet, the positive sample similarity of the song triplet is determined based on the similarity between the second audio feature of the sample song collected by the sample user in the song triplet and the anchor feature of the preset anchor sample. For each of the song triples, the negative sample similarity of the song triples is determined based on the similarity between the second audio feature of the sample songs collected by non-sample users in the song triples and the anchor feature. Based on the positive and negative sample similarities of each song triplet, the triplet loss corresponding to the multiple song triplets is determined.

4. The method according to claim 3, characterized in that, The positive sample similarity of the song triples includes the first positive sample similarity and the second positive sample similarity; The step of determining the triplet loss corresponding to the plurality of song triplets based on the positive sample similarity and negative sample similarity of each song triplet includes: For each song triplet, a first loss parameter value is determined based on the difference between the first positive sample similarity and the negative sample similarity of the song triplet. A second loss parameter value is determined based on the difference between the second positive sample similarity and the negative sample similarity of the song triplet. The larger of the first loss parameter value and the second loss parameter value is taken as the target loss parameter value of the song triplet. Based on the target loss parameter value of each song triplet, the triplet loss corresponding to the plurality of song triplets is determined.

5. The method according to claim 1, characterized in that, The process of training the song recommendation model based on the relation distillation loss and the ternary loss to obtain the trained song recommendation model includes: A first weighting parameter for the relational distillation loss is determined, a second weighting parameter for the ternary loss is determined, and the relational distillation loss and the ternary loss are weighted and fused based on the first weighting parameter and the second weighting parameter to obtain a fused loss; The song recommendation model is trained based on the fusion loss to obtain the trained song recommendation model.

6. The method according to claim 1, characterized in that, The method further includes: Extract the first spectral features of the sample songs from the multiple song triples, and convert the first spectral features of the first dimension into the second spectral features of the second dimension; The second spectral feature is randomly masked at a fixed ratio to obtain the masked feature; The mask features are encoded to obtain the first encoded feature of the second dimension; The first encoded feature is expanded in dimension to obtain a second encoded feature in the first dimension. Decode the second encoded feature to obtain the first embedded feature; The sample songs in the multiple song triples are encoded to obtain the second embedding feature; The training loss is determined based on the first embedding feature and the second embedding feature; Based on the training loss, the large music understanding model to be trained is trained to obtain the trained large music understanding model.

7. A song recommendation method, characterized in that, The method includes: Obtain at least one candidate song to be recommended; Obtain at least one preferred song from the target user's collection; The at least one candidate song and the at least one preferred song are input into a trained song recommendation model to extract the candidate audio features of the at least one candidate song and the preferred audio features of the at least one preferred song, respectively; the song recommendation model is trained by the model training method of any one of claims 1 to 6. For each candidate song, the similarity between the candidate audio features of the candidate song and the preferred audio features of the at least one preferred song is determined to obtain the feature similarity corresponding to the candidate song. Based on the feature similarity of each candidate song, a target song is selected from the at least one candidate song and recommended to the target user.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images