Music recommendation method, electronic device, storage medium, and program product

Abstract

The present disclosure relates to the technical field of computers, and provides a music recommendation method, an electronic device, a storage medium and a program product. The music recommendation method comprises: obtaining a user feature of a user to be recommended; determining a matching degree between each index node and the user to be recommended layer by layer in a pre-constructed hierarchical index structure based on the user feature, and screening the index nodes, so as to obtain a target song used for recommending music to the user to be recommended; and the hierarchical index structure organizes a plurality of songs in a music library through a plurality of levels, each level contains a plurality of index nodes, and at least part of the index nodes of one level in two adjacent levels cover a plurality of index nodes of the other level. The present disclosure can improve the accuracy of music recommendation.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to music recommendation methods, electronic devices, storage media, and program products. Background Technology

[0002] Music recommendation systems typically employ a two-stage architecture of recall and refinement. The recall stage is responsible for efficiently filtering a small number of candidate songs from a massive music library. In related technologies, hierarchical index-based recall schemes usually first construct a static hierarchical index structure based on the content features of the songs using clustering algorithms, and then train a recommendation model on this fixed index structure to predict user preferences for index nodes. However, in the above schemes, the construction of the index structure and the training of the recommendation model are two independent stages. The criteria for dividing the index structure are inconsistent with the optimization objectives of the recommendation model, resulting in a hierarchical index structure that is not optimal for recommendation tasks. This leads to problems such as insufficient accuracy in candidate song selection and wasted computational resources during the layer-by-layer retrieval process.

[0003] Therefore, there is an urgent need for a music recommendation method to solve the problem of insufficient retrieval accuracy and wasted computing resources caused by the inconsistency between the hierarchical index structure and the recommendation target in related technologies. Summary of the Invention

[0004] This disclosure provides a music recommendation method, electronic device, storage medium, and program product to solve the problem of poor music recommendation quality in related technologies.

[0005] Firstly, this disclosure provides a music recommendation method, which includes: Obtain user characteristics of users to be recommended; In the pre-built hierarchical index structure, the matching degree between the index node and the user to be recommended is determined layer by layer based on user characteristics, and the index node is filtered layer by layer according to the matching degree to obtain the target song for music recommendation to the user to be recommended. The hierarchical index structure comprises multiple levels, which are used to organize multiple songs in the music library. Each level includes multiple index nodes, and in any two adjacent levels, at least some index nodes in one level cover multiple index nodes in the other level. Each song is assigned to an index node in each level of the hierarchical index structure based on an assignment probability. The assignment parameters used to determine the assignment probability and the matching parameters used to determine the matching degree are obtained through joint optimization. The optimization objective of the joint optimization includes a recommendation loss, which is constructed based on the behavioral data of sample users.

[0006] In one alternative implementation, joint optimization includes: Song representation is determined based on allocation probability; The prediction results are determined based on the user characteristics of the sample users and the song representation. The prediction results are used to characterize the sample users' preference for the songs. The recommendation loss is constructed based on the prediction results and the behavioral data of the sample users, and the matching parameters and assignment parameters are updated based on the recommendation loss.

[0007] In one alternative implementation, determining the song representation based on the assignment probability includes: The song representation is obtained by weighting the node representations of each index node at the song's level according to the assigned probability.

[0008] In one alternative implementation, the allocation probability is obtained as follows: The song's allocation parameters for each index node in the corresponding level are scaled according to the temperature parameter. Perform exponential operations on each of the scaled allocation parameters; The results of each exponent operation are normalized to obtain the probability of the song being assigned to each index node. The temperature parameter is used to control the smoothness of the distribution of the allocation probability; during the joint optimization process, the temperature parameter decreases as the joint optimization progresses.

[0009] In one alternative implementation, the optimization objective of the joint optimization also includes at least one of index balancing loss and hierarchy consistency loss; Index balancing loss is used to constrain the balance of the number of songs allocated to each index node; Hierarchical consistency loss is used to constrain the consistency between the allocation probability of a song in a hierarchy and the allocation probability obtained by aggregating the allocation probabilities in adjacent hierarchy levels.

[0010] In one alternative implementation, the matching degree is determined by a recommendation model, which includes a user encoding subnetwork, a music encoding subnetwork, and an interaction subnetwork. The user coding subnetwork is used to encode user features to obtain user representations; The music coding subnetwork is used to encode the music features of the index nodes to obtain music representations; The interaction subnetwork is used to determine the interaction representation that represents the interaction relationship between the user and the music based on the intermediate layer features of the user coding subnetwork and the music coding subnetwork. The matching degree is determined based on user representation, music representation, and interaction representation.

[0011] In one alternative implementation, the recommendation models corresponding to different levels share at least some parameters in the user coding subnetwork and the music coding subnetwork, and the recommendation models corresponding to different levels have their own independent interaction subnetworks.

[0012] In one optional implementation, the index nodes are filtered layer by layer based on the matching degree, including: The current level of filtering is performed on the index nodes covered by the selected index node in the previous level, in order of increasing number of index nodes in the hierarchy. In each level, index nodes whose matching degree meets preset conditions are selected.

[0013] In one alternative implementation, the method further includes: In response to the addition of a new song to the music library, the representation of the new song is determined based on its content characteristics, and the index node to which the new song belongs in each level of the hierarchical index structure is determined based on the similarity between the representation and the node representation of each index node.

[0014] In one alternative implementation, the method further includes: Obtain the metric information for each index node, including at least one of the following: the number of songs belonging to the index node and the exposure of songs belonging to the index node. If the index information of any index node meets the preset redistribution conditions, the index node to which the song belongs is reassigned is determined.

[0015] In one alternative implementation, each song belongs to a unique index node at each level of the hierarchical index structure, and the unique index node is the index node with the highest probability of being assigned to the song at that level.

[0016] In a second aspect, this disclosure provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the music recommendation method described in the first aspect or any corresponding embodiment thereof.

[0017] Thirdly, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the music recommendation method described in the first aspect or any corresponding embodiment.

[0018] Fourthly, this disclosure provides a computer program product, including computer instructions for causing a computer to execute the music recommendation method described in the first aspect or any corresponding embodiment thereof.

[0019] The music recommendation method provided in this disclosure has two aspects. Firstly, multiple songs in a music library are organized using a hierarchical index structure with multiple levels. Each song is assigned to an index node at each level based on an allocation probability. During music recommendation, the matching degree between the index node and the user to be recommended is determined layer by layer based on the user characteristics of the user to be recommended, and the index nodes are filtered layer by layer. This ensures that the matching degree calculation at each level only needs to be performed on the range covered by the filtered and retained index nodes at that level, without needing to calculate it for each song in the music library individually. This reduces the computational load in determining the target song and improves the efficiency of music recommendation. Secondly, the index nodes to which the songs belong at each level... The points are determined based on the allocation probability, and the allocation parameters used to determine the allocation probability and the matching parameters used to determine the matching degree are obtained through joint optimization. The optimization objective of the joint optimization includes the recommendation loss constructed based on the behavioral data of sample users. This ensures that the song's affiliation in the hierarchical index structure and the matching degree calculation between the index node and the user are adjusted together according to the same optimization objective. The resulting hierarchical index structure is consistent with the recommendation objective, avoiding the problem that the index structure is not optimal for the recommendation task due to different criteria for song affiliation and recommendation objectives. This improves the matching degree between the target songs selected based on the hierarchical index structure and the preferences of the users to be recommended, thus improving the accuracy of music recommendation. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure; Figure 2 This is a schematic flowchart of a first method for recommending music according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of a second process for a music recommendation method according to an embodiment of the present disclosure; Figure 4 This is a schematic diagram of a hierarchical retrieval reasoning process according to an embodiment of the present disclosure; Figure 5 This is a structural block diagram of a music recommendation device according to an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0023] It should be noted that the information (including but not limited to user-inputted information, such as search keywords entered by users in music playback clients), data (including but not limited to data used for analysis, stored data, and displayed data, such as user listening history data, collection behavior data, and user profile feature data), and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. For example, the user listening history, collection behavior data, and user profile feature data involved in this disclosure were all obtained with full authorization.

[0024] Before providing a detailed description of the embodiments of this disclosure, some of the nouns and terms involved in the embodiments of this disclosure will be explained.

[0025] (1) Recall: A processing stage in a recommendation system, used to initially filter out a smaller subset of candidates from the full candidate set for further processing in subsequent ranking and other stages.

[0026] (2) Hierarchical index structure: A tree-like data structure that organizes a large-scale data set into multiple levels from coarse to fine. The upper-level nodes cover multiple lower-level nodes, and the search range can be narrowed layer by layer during retrieval.

[0027] (3) Index node: The basic unit of each level in the hierarchical index structure. An index node corresponds to a set of data with similar attributes.

[0028] (4) Soft assignment: An assignment method that associates an object with multiple categories or nodes simultaneously in the form of a probability distribution. The assignment result is continuous and differentiable.

[0029] (5) Hard allocation: An allocation method that uniquely and definitively assigns an object to a certain category or node. The allocation result is discrete.

[0030] (6) logits: The raw, unnormalized output scores in a neural network, which are usually converted into a probability distribution by a normalization function.

[0031] (7) softmax (normalized exponential function): a function that converts a set of real values ​​into a probability distribution. It first performs an exponential operation on each value, and then divides each result by the sum of all results, so that all output terms are positive and the sum is one.

[0032] (8) argmax (maximum index operation): an operation that selects the position or category corresponding to the maximum value from a set of values.

[0033] (9) Temperature parameter: The scaling factor in the softmax function, used to control the smoothness of the output probability distribution. The higher the temperature, the closer the distribution is to uniformity, and the lower the temperature, the more concentrated the distribution is.

[0034] (10) Annealing: A strategy that gradually reduces a certain control parameter as the training process progresses.

[0035] (11) Dual-tower model: a common recall model structure that uses two independent sub-networks to encode users and items as vectors, and measures the degree of matching by the dot product or cosine similarity between vectors.

[0036] (12) MLP (Multi-Layer Perceptron): A basic neural network structure consisting of multiple fully connected layers.

[0037] (13) Sampled softmax loss: a loss function used for large-scale classification or retrieval tasks. It approximates the full normalization calculation by sampling some negative samples, thereby reducing training overhead.

[0038] (14) KL divergence (Kullback-Leibler divergence, relative entropy): an index that measures the degree of difference between two probability distributions. The closer the two distributions are, the smaller the KL divergence.

[0039] (15) Beam Search: A heuristic search strategy that retains several optimal candidates layer by layer in a hierarchical or sequence space and expands step by step.

[0040] (16) Top-K: The operation of selecting the top K objects after sorting them by score from highest to lowest.

[0041] (17) ANN (Approximate Nearest Neighbor) search: A retrieval technique that quickly finds the vector that is closest to the query vector in a vector space in an approximate manner.

[0042] (18) Gradient descent and backpropagation: The basic training method of neural networks. Based on the gradient of the loss function with respect to the parameters, the parameters are iteratively updated in the direction that reduces the loss.

[0043] (19) K-means clustering: a commonly used clustering algorithm that divides samples into the nearest cluster based on the cluster center.

[0044] (20) Hierarchical clustering: A clustering method that aggregates or divides samples into clusters of different granularities in a hierarchical manner.

[0045] (21) TDM (Tree-based Deep Model): A recall method that uses a predefined tree structure to organize candidate items and uses a deep model to perform layer-by-layer retrieval.

[0046] (22) DSI (Differentiable Search Index): A generative retrieval method that directly generates item identifiers through a sequence-to-sequence (seq2seq) model.

[0047] (23) DeepFM and DIN: Two commonly used recommendation model structures. The former combines factorization machine and deep network, while the latter introduces attention mechanism for user history behavior.

[0048] (24) RQ-VAE (Residual Quantized Variational AutoEncoder): A model that converts continuous vectors into discrete codes layer by layer through multi-level residual quantization, with each level maintaining a codebook.

[0049] (25) FSQ (Finite Scalar Quantization): A method for deterministic finite scalar quantization of each dimension of a vector without the need to maintain an explicit codebook.

[0050] (26) Codebook: A set of candidate vectors in a quantization method. The quantization process is to select the vector that is closest to the input from the codebook.

[0051] (27) Straight-through estimator: A technique for approximating gradients in networks containing non-differentiable discrete operations. The forward pass is calculated based on discrete results, and the backward pass is approximated as a continuous quantity.

[0052] As one optional application scenario of this disclosure embodiment, such as Figure 1As shown, this music recommendation method can be applied to an implementation environment that includes at least one terminal device and at least one server. Figure 1 The example shows that the implementation environment includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through the network 110.

[0053] The terminal device can specifically be a smartphone, tablet computer, laptop computer, PDA, desktop computer, smart TV, smart speaker, smart wearable device, in-vehicle terminal, or other device capable of playing music. The terminal device may have a music playback client installed, through which the user listens to music. The terminal device sends relevant information about the user to be recommended to the server 103 via network 110 and receives the recommendation results returned by server 103. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Server 103 may maintain a music library and a hierarchical index structure for organizing multiple songs in the music library, and execute the music recommendation method provided in this embodiment based on the hierarchical index structure. Network 110 can be a wired network or a wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0054] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this disclosure. The music recommendation method provided in this disclosure can be executed by server 103, by a terminal device with corresponding computing capabilities, or by a terminal device and server 103 in collaboration. No limitation is made in this disclosure.

[0055] In related technologies, the song classification in the music library is pre-defined and fixed in the index structure. The construction of the index structure is separate from the training of the recommendation model. To address the technical problem that the classification criteria for song classification are inconsistent with the optimization goals of the recommendation model, resulting in the index structure not being optimal for the recommendation task and thus affecting the accuracy of the recommendation, this disclosure provides a music recommendation method to adapt the hierarchical index structure to the recommendation goal and improve the accuracy of music recommendation.

[0056] According to an embodiment of this disclosure, a music recommendation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0057] This embodiment provides a music recommendation method, which can be used in the aforementioned server, or in terminal devices with corresponding computing capabilities, such as mobile phones and tablets. Figure 2 This is a flowchart of a music recommendation method according to an embodiment of the present disclosure, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the user characteristics of the users to be recommended.

[0058] Users to be recommended to are those who currently need music recommendations. For example, they could be users who are using a music player client, or users who have triggered recommendation scenarios such as playlist retrieval requests or radio playback requests.

[0059] User features refer to characteristic information that reflects the attributes and / or preferences of the user to be recommended. For example, user features may include one or more of the following: user profile features (such as relevant attributes in registration information), features corresponding to the user's historical music listening behavior (such as information related to songs involved in historical playback, favorites, etc.), and the user's current context features (such as the time period and scenario in which the recommendation request occurred). The specific form of user features is not limited in this embodiment. For example, the above feature information can be encoded into the form of feature vectors to facilitate subsequent matching degree calculation.

[0060] There are various ways to obtain user characteristics. For example, in response to a recommendation request sent by a terminal device, the user characteristics of the user to be recommended can be extracted from the information carried in the recommendation request and the pre-stored user data. Alternatively, when a preset recommendation time is reached, the relevant data of the user to be recommended can be actively read and user characteristics can be generated.

[0061] Step S202: In the pre-built hierarchical index structure, the matching degree between the index node and the user to be recommended is determined layer by layer based on user characteristics, and the index node is filtered layer by layer according to the matching degree to obtain the target song for music recommendation to the user to be recommended.

[0062] First, let's explain the hierarchical index structure. The hierarchical index structure consists of multiple levels, which are used to organize multiple songs in a music library. In other words, the songs in the music library are organized and managed hierarchically through this hierarchical index structure. Each level includes multiple index nodes, which are the basic units in the hierarchical index structure. An index node can be understood as corresponding to a set of songs. In adjacent levels, at least some index nodes in one level cover multiple index nodes in the other level; that is, there is an overlay relationship between the index nodes of adjacent levels. This forms a hierarchical organization from coarse to fine. For example, an index node in a higher level can cover multiple index nodes in a lower level. The range of songs corresponding to index nodes in a higher level is relatively larger, and the range of songs corresponding to index nodes in a lower level is relatively smaller.

[0063] Each song is assigned to an index node in each level of the hierarchical index structure based on its allocation probability. The allocation probability represents the probability that a song will be assigned to any index node in the corresponding level. For example, for a given song, the allocation probability of it being assigned to each index node in a certain level can be determined, and the index node to which the song belongs can be determined from multiple index nodes in that level based on the allocation probability. The method for determining the assignment based on the allocation probability can be varied. For example, the index node with the highest allocation probability in that level can be determined as the index node to which the song belongs; another example is that the index nodes in that level can be sampled according to the allocation probability, and the sampled index nodes can be determined as the index nodes to which the song belongs; yet another example is that the index nodes whose allocation probability reaches a preset probability threshold can be determined as the index nodes to which the song belongs. This embodiment does not impose any limitations on this method.

[0064] Furthermore, the allocation parameters used to determine the allocation probability and the matching parameters used to determine the matching degree are obtained through joint optimization. The allocation parameters are learnable parameters used to determine the allocation probability of a song in each level, and the matching parameters are learnable parameters used to determine the matching degree between the index node and the user; for example, they could be the model parameters of the model used to calculate the matching degree. Joint optimization means that the allocation parameters and matching parameters are adjusted together in the same optimization process, rather than fixing one and then optimizing the other separately. For example, during training, the allocation parameters and matching parameters can be updated with the same optimization objective, so that the song's affiliation in the hierarchical index structure and the calculation of the matching degree evolve together based on the same objective.

[0065] The optimization objective of this joint optimization includes a recommendation loss, which is constructed based on the behavioral data of sample users. For example, the behavioral data of sample users may include data corresponding to their actions such as playing, favorited, and skipping songs. Based on this behavioral data, the preferences of sample users for songs can be determined, and a recommendation loss can be constructed to measure the difference between the predicted results and the actual preferences of sample users. Because the optimization objective includes a recommendation loss constructed based on behavioral data, the allocation parameters can be adjusted in a direction that aligns with the user preference prediction objective during the joint optimization process, ensuring that the song attribution determined based on the allocation probability is consistent with the recommendation objective.

[0066] Based on the aforementioned hierarchical index structure, the process of determining the target song in this step is explained. Determining the matching degree between index nodes and the users to be recommended, layer by layer, based on user characteristics, means calculating the matching degree between index nodes at each level and the users to be recommended, following the hierarchical order of the index structure. The matching degree represents the degree of matching between the song set corresponding to the index node and the preferences of the users to be recommended. For example, the matching degree can be expressed as a score; a higher score indicates a higher degree of preference for the song set corresponding to that index node by the users to be recommended.

[0067] Filtering index nodes layer by layer based on matching degree means retaining a portion of index nodes at each level according to their matching degree. For example, at a certain level, a preset number of index nodes can be selected in descending order of matching degree. In the next level, matching degree is calculated and filtered only for the index nodes covered by the selected index nodes. This process is repeated layer by layer, gradually narrowing the candidate range until the target song for music recommendation to the user is obtained. It is understood that the target song is the song determined after layer-by-layer filtering and used to recommend to the user. The number of target songs can be one or more. After obtaining the target song, it can be directly recommended to the user, or it can be provided as a candidate for subsequent recommendation processing. This embodiment does not impose any limitations.

[0068] In summary, the music recommendation method provided in this disclosure organizes multiple songs in a music library through a hierarchical index structure with multiple levels. It determines the matching degree between index nodes and the users to be recommended layer by layer based on the user characteristics of the users to be recommended, and filters index nodes layer by layer. This limits the calculation of the matching degree to the range covered by the retained index nodes during the layer-by-layer filtering process, eliminating the need to calculate it for each song in the entire music library, thus reducing the computational load required to determine target songs and improving the efficiency of music recommendation. Furthermore, the index node to which a song belongs in each level is determined based on allocation probabilities. The allocation parameters used to determine the allocation probabilities and the matching parameters used to determine the matching degree are jointly optimized using an optimization objective including a recommendation loss. The recommendation loss is constructed based on the behavioral data of sample users, ensuring that the song's affiliation in the hierarchical index structure and the calculation of the matching degree are jointly optimized based on the same recommendation objective. The hierarchical index structure is consistent with the recommendation objective, thereby improving the matching degree between the target songs obtained through layer-by-layer filtering and the preferences of the users to be recommended, and improving the accuracy of music recommendation.

[0069] This embodiment provides a music recommendation method, which can be used in the aforementioned server, or in terminal devices with corresponding computing capabilities, such as mobile phones and tablets. Figure 3 This is a flowchart of a music recommendation method according to an embodiment of the present disclosure, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the user characteristics of the users to be recommended. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0070] Step S302: In the pre-built hierarchical index structure, the matching degree between the index node and the user to be recommended is determined layer by layer based on user characteristics, and the index node is filtered layer by layer according to the matching degree to obtain the target song for music recommendation to the user to be recommended.

[0071] Each song is assigned to an index node in each level of the hierarchical index structure based on its allocation probability. The allocation probability represents the probability that a song will be assigned to any index node in the corresponding level. For example, for a given song, the allocation probability of it being assigned to each index node in a certain level can be determined, and the index node to which the song belongs can be determined from multiple index nodes in that level based on the allocation probability. For instance, the index node to which the song belongs can be selected based on the magnitude of the allocation probability, or it can be determined by sampling according to the allocation probability; this is not limited in this embodiment.

[0072] In some embodiments, each song belongs to a unique index node in each level of the hierarchical index structure, and the unique index node is the index node with the highest probability of being assigned to the song in the level.

[0073] Belonging to a unique index node means that, in any given level, a song belongs to only one index node at that level, and not to multiple index nodes at the same level. The index node with the highest allocation probability is the one with the highest allocation probability among multiple index nodes at that level. For example, if a level includes several index nodes, and a song has its own allocation probability on each of these index nodes, then the index node with the highest allocation probability is determined as the unique index node to which the song belongs at that level.

[0074] With this implementation method, each song has a definite and unique index position in each level. The range of songs covered by the selected index node during layer-by-layer filtering is clear, avoiding ambiguity in song ownership and improving the certainty of the layer-by-layer filtering process and the execution efficiency of online recommendation.

[0075] The determination of the matching degree will be explained next. The matching degree is used to represent the degree of matching between the song set corresponding to the index node and the preferences of the user to be recommended, and can be expressed as a score.

[0076] In some embodiments, the matching degree is determined by a recommendation model, which includes a user encoding subnetwork, a music encoding subnetwork, and an interaction subnetwork.

[0077] The user encoding subnetwork is used to encode user features to obtain a user representation; the music encoding subnetwork is used to encode the music features of the index nodes to obtain a music representation. The music features of the index nodes may include, for example, the overall features of the song set corresponding to the index node; the interaction subnetwork is used to determine the interaction representation representing the interaction relationship between the user and the music based on the intermediate layer features of the user encoding subnetwork and the music encoding subnetwork. The intermediate layer features refer to the intermediate outputs generated by the subnetwork during the encoding process. For example, the interaction subnetwork can perform one or more operations such as concatenating, element-wise multiplying, and element-wise subtracting on the intermediate layer features of the two subnetworks to construct interaction information and output the interaction representation based on the interaction information; the matching degree is determined based on the user representation, music representation, and interaction representation. For example, the three can be fused and mapped to a matching degree score through a multi-layer network.

[0078] Through this implementation method, in addition to relying on the respective representations of the user side and the music side, the determination of the matching degree also introduces an interactive representation constructed based on the features of the intermediate layers on both sides. This can reflect the interactive relationship between the user and the music in the matching degree, which is conducive to improving the matching degree's ability to characterize user preferences, thereby improving the accuracy of layer-by-layer screening.

[0079] Furthermore, considering that the hierarchical index structure includes multiple levels and the matching degree needs to be calculated for each level, in some embodiments, the recommendation models corresponding to different levels share at least some parameters in the user encoding subnetwork and the music encoding subnetwork, and the recommendation models corresponding to different levels have their own independent interaction subnetworks.

[0080] Sharing at least some parameters, for example, could mean that recommendation models at different levels share some network parameters from two encoding subnetworks near the input side. Independent interaction subnetworks mean that each level sets its own interaction subnetwork, and the parameters of each interaction subnetwork are independent of each other. Through this implementation, the same user and the same music can obtain relatively consistent basic representations across different levels. The shared parameters can be updated jointly using training data from each level, which helps improve the efficiency of training data utilization. Simultaneously, the independent interaction subnetworks at each level can adapt to the filtering tasks at each level, taking into account the differences in filtering granularity between different levels.

[0081] The process of layer-by-layer filtering will be explained next. As an optional implementation method, filtering index nodes layer by layer based on matching degree includes: performing filtering of the current level among the index nodes covered by the selected index nodes in the previous filtering level, in order of increasing number of index nodes in the level; wherein, in each level, index nodes whose matching degree meets the preset conditions are selected.

[0082] Following the order of increasing number of index nodes in the hierarchy means starting with the hierarchy with fewer index nodes and progressively filtering towards the hierarchy with more index nodes. The previous filtering hierarchy refers to the hierarchy that completed filtering before the current hierarchy. Preset conditions could be, for example, matching scores sorted from highest to lowest and falling within a preset number, matching scores reaching a preset threshold, or matching scores ranking within a preset proportion. For instance, if two index nodes were selected in the previous filtering hierarchy, the current hierarchy only determines the matching score of the index nodes covered by those two index nodes and selects the index nodes whose matching scores meet the preset conditions. This process continues layer by layer until the target song is obtained.

[0083] With this implementation method, the matching degree calculation of each level is limited to the range covered by the selected results of the previous filtered level. The range of index nodes participating in the calculation is reduced layer by layer. It is not necessary to calculate for all songs in the music library one by one, thereby reducing the amount of calculation to determine the target song and improving the efficiency of music recommendation.

[0084] In some embodiments, the method further includes the step of maintaining and updating the hierarchical index structure.

[0085] Step S303: In response to the addition of a new song to the music library, the representation of the new song is determined based on the content features of the new song, and the index node to which the new song belongs in each level of the hierarchical index structure is determined according to the similarity between the representation and the node representation of each index node.

[0086] Among them, new songs refer to songs newly added to the music library; content features refer to features obtained based on the song's own content, which may include one or more of the following, such as audio features, lyric features, and singer-related information; node representation refers to the feature expression of the index node.

[0087] The process involves determining the representation of a new song, for example, by inputting content features into an encoding network for forward computation to obtain a representation vector; similarity can be measured using cosine similarity or distance metrics between vectors; and determining the assigned index node. This can be done by selecting the index node with the highest similarity as the new song's assigned index node at the corresponding level, or by weighting a preset number of candidate index nodes based on similarity and selecting the index node with the highest weighting result. It is understood that new songs often lack user behavior data when added to a music library. Through step S303, new songs can obtain their affiliation at each level based on their content features without waiting for behavioral data to accumulate, thus enabling them to participate in layer-by-layer filtering and gain the opportunity to be recommended. This helps alleviate the problem of new songs being difficult to recommend due to a lack of behavioral data and also allows the hierarchical index structure to adapt to changes in the music library.

[0088] Step S304: Obtain the indicator information of each index node. The indicator information includes at least one of the number of songs belonging to the index node and the exposure of songs belonging to the index node. If the indicator information of any index node meets the preset redistribution conditions, redetermine the index node to which the songs belonging to that index node belong.

[0089] Here, exposure refers to the number of songs displayed to users within an index node, such as the average number of times songs in a node are displayed within a statistical period; obtaining indicator information, such as periodically statistically analyzing each index node with a preset duration window; redistribution conditions refer to the conditions used to determine whether an index node needs to redistribute songs. For example, when the number of songs and / or exposure of a certain index node is too high or too low relative to the average level of the same level, it can be determined that the redistribution conditions are met. When a certain index node is not selected by any user within a consecutive preset period, it can also be determined that the redistribution conditions are met; reassigning the index node to which it belongs, for example, can be migrating some songs in the node to index nodes with lower load in the same level, or migrating songs in nodes that have lacked exposure for a long time to other similar index nodes based on their representational similarity.

[0090] Step S304 allows for adjustments to song attribution based on the actual load of index nodes, alleviating situations where some index nodes are overloaded with songs or lack exposure for extended periods. This ensures the hierarchical index structure maintains a reasonable organizational state as the music library and song popularity change, thus contributing to the efficiency of layer-by-layer filtering and the stability of recommendation results.

[0091] In some embodiments, the method further includes a joint optimization process. It is understood that this joint optimization process is executed before step S302; that is, the allocation parameters and matching parameters are first obtained through joint optimization, and then music recommendation is performed based on the jointly optimized hierarchical index structure. Specifically, the joint optimization process may include the following steps: The first step is to determine the song representation based on the allocation probability.

[0092] In this context, song representation refers to the representation of song features in the form of vectors or similar methods. As an optional implementation, the node representations of each index node at the song's level can be weighted according to assigned probabilities to obtain the song representation.

[0093] Weighting refers to multiplying the node representation of each index node by the allocation probability of the song on the corresponding index node and then summing the results. In other words, the song representation is a weighted sum of the node representations, each weighted by its corresponding allocation probability. Through this implementation, the song representation is continuously correlated with the allocation probability of each index node. Changes in the allocation probability are continuously reflected in the song representation and thus in the recommendation loss, facilitating gradient-based updates to the allocation parameters and providing a foundation for the allocation parameters to participate in joint optimization.

[0094] Furthermore, the method for obtaining the allocation probabilities is explained. In some embodiments, the allocation probabilities are obtained as follows: the allocation parameters of the song for each index node in the corresponding level are scaled according to the temperature parameter; exponential operations are performed on each scaled allocation parameter; the results of each exponential operation are normalized to obtain the allocation probability of the song on each index node; wherein, the temperature parameter is used to control the smoothness of the distribution of allocation probabilities, and the temperature parameter decreases as the joint optimization process progresses.

[0095] Specifically, scaling can be achieved by dividing the allocation parameter by the temperature parameter; exponential operations can be performed using the natural constant as the base and the scaled allocation parameter as the exponent; normalization can be achieved by dividing the result of each exponential operation by the sum of the results, so that the sum of the allocation probabilities is one. It is understood that when the temperature parameter value is large, the differences between the allocation probabilities are reduced, the distribution is smoother, and songs can be associated with multiple index nodes with relatively close probabilities; when the temperature parameter value is small, the distribution tends to be concentrated. The temperature parameter can be decreased, for example, by decreasing with the number of training steps according to a preset decay coefficient, or by adjusting it in stages according to preset phases. Through this implementation method, the distribution of allocation probabilities in the early stage of joint optimization is relatively smooth, and the song assignment has a large adjustment space, making it easier to explore more suitable assignments during the optimization process; as the temperature parameter decreases, the distribution gradually tends to be concentrated, gradually approaching the method of determining the index node of the song assignment based on the allocation probability, which helps to reduce the difference between the optimization stage and the recommendation stage.

[0096] The second step is to determine the prediction results based on the user characteristics of the sample users and the song representations. The prediction results are used to characterize the sample users' preference for the songs.

[0097] Here, "sample user" refers to the user whose behavioral data is used for joint optimization. The type of user characteristics of the sample user can be the same as the user characteristics described in step S301. The prediction result can be, for example, a prediction score, where a higher score indicates a higher degree of preference for the song by the sample user. Determining the prediction result can be, for example, by inputting the user characteristics of the sample user and the song representation into a model used to calculate the matching degree to obtain the prediction score.

[0098] The third step is to construct a recommendation loss based on the prediction results and the behavioral data of the sample users, and then update the matching parameters and assignment parameters based on the recommendation loss.

[0099] The process involves constructing a recommendation loss, for example, by determining positive and negative samples of actual user preferences based on behavioral data, and measuring the difference between the predicted results and the actual preferences; the greater the difference, the greater the recommendation loss. The update process can be achieved, for example, using gradient descent to iteratively adjust the matching and assignment parameters based on the recommendation loss. It's understandable that since song representations are determined based on assignment probabilities, and assignment probabilities are determined by assignment parameters, the adjustment signal for the recommendation loss can affect both the matching and assignment parameters.

[0100] Through the above joint optimization process, the criteria for determining song affiliation are aligned with the optimization objectives of preference prediction, ensuring that the hierarchical index structure obtained through joint optimization is consistent with the recommendation objectives. This helps to improve the matching degree between target songs selected based on the hierarchical index structure and user preferences.

[0101] Furthermore, in some embodiments, the optimization objective of the joint optimization includes, in addition to the recommendation loss, at least one of index balancing loss and hierarchical consistency loss. The index balancing loss constrains the balance of the number of songs assigned to each index node. For example, it can be constructed based on the deviation between the statistical probability of allocation on each index node and a uniform distribution; the greater the deviation, the greater the loss. Introducing the index balancing loss can suppress the excessive concentration of songs on a few index nodes, making the number of songs corresponding to each index node relatively balanced, which is beneficial for balancing the computational load on each index node during layer-by-layer filtering. The hierarchical consistency loss constrains the consistency between the allocation probability of a song in a level and the allocation probability obtained by aggregating the allocation probabilities in adjacent levels. Aggregation, for example, can be the summation of the allocation probabilities on multiple index nodes in the adjacent levels covered by an index node. The hierarchical consistency loss can be constructed, for example, by measuring the difference between two probability distributions. Introducing the hierarchical consistency loss can constrain the logical consistency of the song's affiliation in adjacent levels, reducing the possibility of missing songs during layer-by-layer filtering due to contradictory affiliations in adjacent levels.

[0102] In summary, the music recommendation method provided in this disclosure organizes multiple songs in a music library through a hierarchical index structure with multiple levels. It determines the matching degree layer by layer based on the user characteristics of the user to be recommended and filters index nodes layer by layer to obtain target songs, reducing the computational burden of determining target songs. Each song is assigned to an index node at each level based on its allocation probability. The allocation parameters and matching parameters are jointly optimized by an optimization objective including recommendation loss, ensuring the hierarchical index structure aligns with the recommendation objective and improving the matching degree between target songs and the preferences of the user to be recommended. Furthermore, the index node with the highest allocation probability at each level is determined as the unique index node to which each song belongs, giving each song a definite index position. Based on this, by combining the decreasing adjustment of temperature parameters, constraints of index balance loss and hierarchical consistency loss, a recommendation model including an interactive subnetwork, and an update mechanism for adding new songs and reassigning index nodes, the hierarchical index structure can adapt to the continuous changes in the music library while maintaining both the accuracy and efficiency of music recommendation.

[0103] To better illustrate the music recommendation method of this disclosure, a preferred embodiment will be provided below. This embodiment is intended to describe the implementation process of this disclosure in detail, but is not intended to limit the scope of protection of this disclosure.

[0104] In this embodiment, a music retrieval and recommendation scenario in a large-scale online music platform is used as a preferred example for illustration. This scenario is chosen because the music library of online music platforms typically reaches tens of millions or even hundreds of millions of songs. It requires filtering candidate songs that match the user's preferences from a massive library within a millisecond-level response time, placing high demands on both retrieval efficiency and recommendation accuracy. This scenario effectively demonstrates the combined optimization effect of hierarchical indexing and the recommendation model in this embodiment. It is understood that this embodiment is not limited to this example; it is applicable to any recommendation scenario that requires efficient filtering of personalized content from a large-scale candidate set. Furthermore, this embodiment can be implemented as a music recommendation method, or correspondingly as a recommendation system, electronic device, or storage medium that executes the method. The steps of the method can, for example, be executed by the music platform's server. The following description focuses on the method as the main thread.

[0105] In this scenario, music retrieval solutions in related technologies mainly fall into three categories: First, vector retrieval methods based on a dual-tower model, where the user side and the music side are encoded into vectors and then retrieved using an ANN (Approximate Nearest Neighbor) search. However, the two sides are completely decoupled during the inference stage, relying solely on dot product or cosine similarity matching, which is insufficient to characterize the complex interaction between users and music. Second, methods based on tree structures or hierarchical indexes, such as TDM (Tree-Based Deep Model), which performs layer-by-layer retrieval through a pre-built tree structure. However, index building and model training are two separate stages. Static indexes are first built by clustering songs based on their own features, and then the model is trained based on the fixed indexes. This results in inconsistencies between the index structure construction basis and the recommendation target. Furthermore, static indexes are difficult to adapt to the continuous changes in the music library, and the hard allocation method also limits the opportunity for songs with multi-dimensional attributes to be discovered by different types of users. Third, generative indexing methods, such as DSI (Differentiable Search Index), which directly generates item identifiers through a seq2seq model. However, its training efficiency, inference latency, and integration with multimodal features of music in large-scale industrial scenarios are still immature. To address the aforementioned issues, this preferred embodiment provides a method for constructing and recommending a dynamic hierarchical index for a music library based on joint optimization.

[0106] The overall process of the music library dynamic hierarchical index construction and recommendation method based on joint optimization in this embodiment is divided into an offline training stage and an online service stage, which are described below.

[0107] Step 1: Construct a multi-level music index structure.

[0108] The entire music library is organized into an L-level hierarchical index structure, where level 1 is the coarsest level and level L is the finest level (i.e., song level). Each level contains several index nodes, with upper-level index nodes covering multiple sub-index nodes of lower levels, forming a tree-like hierarchical relationship. Specifically, level 1 contains N1 index nodes, each representing a major music category (such as a major genre category); level 2 contains N2 index nodes, each representing a sub-music category (such as a sub-style or mood category), and N2 is greater than N1; level L contains N... L There are 10 index nodes, each corresponding to a specific song or a small group of songs; the layers satisfy the containment relationship, that is, the upper-level node is an aggregation of the lower-level nodes.

[0109] The number of layers L and the number of nodes per layer can be pre-defined based on the size of the music library: assuming the total number of songs in the library is M, and the number of nodes per layer (i.e., the codebook size) is C, then the constraint C^L ≥ M must be satisfied. This constraint means that the number of leaf node combinations in the entire tree can cover the entire music library. The initial index structure can be constructed based on the multidimensional features of the songs (including audio acoustic features, lyric semantic features, and tag metadata features) using a hierarchical clustering algorithm.

[0110] Step 2: Construct a multi-level modular recommendation model.

[0111] For each level of the hierarchical index, a corresponding modular neural network (MoNN) is constructed. Each modular neural network contains three sub-modules: User Tower, which receives user features (such as listening history, collection behavior, user profile, etc.) and outputs a user representation vector; Music Tower, which receives the music features corresponding to the index node (i.e., the music group features aggregated by that node) and outputs a music representation vector; and Interaction Tower, which receives the concatenation or intersection of intermediate layer features from the User Tower and Music Tower, models the complex interaction relationship between users and music, and outputs an interaction representation vector. The final prediction score is jointly determined by the outputs of the three towers, i.e.: score(u,m)=f(e u ,e m ,e interact ); Among them, e u For user representation, e m For musical representation, e interact For interactive representation, f is the fusion function.

[0112] The fusion function f can be any model structure (MLP) or fusion function (inner product), for example: score(u, m) = MLP(eu ⊙e m ∥e interact ), where ⊙ represents element-wise product, capturing the feature-level interaction between the user and the music; ∥ represents the concatenation operation; MLP maps the concatenated vector to the final scalar score.

[0113] The complexity of models can vary across different layers: coarse-grained layers (such as layer 1) have fewer index nodes and can use more complex models (wider networks, deeper interaction layers); fine-grained layers have more index nodes and use lighter models to balance inference efficiency. Furthermore, modular neural networks at different layers can adopt a parameter architecture of shared parameters at the bottom and private parameters at the top: the shared part consists of the shared parameters of the user feature encoder and music feature encoder across all layers, ensuring consistent basic representations of the same user and the same music across different layers, while improving the efficiency of training data utilization; the private part consists of independent interaction networks and prediction heads for each layer. The private networks in coarse-grained layers are deeper and wider, responsible for capturing global category distinctions, while the private networks in fine-grained layers are shallower and narrower, focusing on accurate ranking within the same category. In terms of training strategy, joint multi-task training is adopted, with the loss functions of each layer weighted and summed before unified backpropagation. Shared parameters receive gradient updates from all layers, while private parameters only receive gradients from their own layer. In the early stages of training, layer-by-layer pre-training can also be used to initialize the parameters for each layer.

[0114] Step 3: Learning the joint index based on gradient descent.

[0115] The allocation of songs to index nodes at each level is modeled as learnable parameters and jointly optimized with the recommendation model, specifically including the following:

[0116] For each song m in the music library, an index assignment probability vector is introduced at the l-th layer. ,in [k] represents the probability that song m is assigned to the k-th index node of the l-th level: ; The meaning of this formula is: The learnable logits parameter vector of song m at layer l. After scaling by the temperature parameter τ, the result is normalized to a probability distribution using the softmax function. This is a learnable vector of logits parameters. This refers to the temperature parameter.

[0117] During the training phase, a soft allocation strategy is adopted. The effective representation of song m at layer l is the weighted sum of the central representations of all index nodes in that layer according to the allocation probability: ; The meaning of this formula is: N of the l-th layer l The central representation of each index node is c k (l) Multiply by the probability π of song m being assigned to the corresponding node. m (l) After summing [k], we obtain the representation of song m at this layer, ê. m (l) Since this representation is a continuous function of the allocation probability, the gradient of the recommendation loss can be backpropagated to the allocation parameters through it, which is the principle behind soft allocation guaranteeing differentiability.

[0118] Here, the node representation of the k-th index node in the l-th layer is used, and the node representation of each index node is also a learnable parameter. In the initialization phase, the music library is hierarchically clustered based on the pre-trained music representation, and the cluster center vector is used as the initial value of the corresponding node representation. In the training phase, the node representation is used as a model parameter in end-to-end training, and gradient updates are performed through backpropagation of the recommendation loss, gradually optimizing it into a discriminative representation for user preference matching. In the online service phase, when the music library changes dynamically, the node representation of the affected nodes is updated in real time using incremental aggregation, and overall correction is performed periodically through full retraining.

[0119] During the service phase, a hard allocation strategy is used, and song m is assigned to the index node with the highest probability: [k]; The meaning of this formula is: take the index node number k with the highest probability of song m being assigned to layer l as its affiliation a. m (l) This ensures that each song has a specific index position during retrieval, meeting the efficiency requirements of online services.

[0120] Design a temperature scheduling strategy to gradually reduce the temperature parameter during training: ; The formula means: the temperature at the t-th training step. equal to the initial temperature Multiply by the attenuation coefficient γ raised to the power of t. The initial temperature, ∈(0,1) is the decay coefficient, and t is the number of training steps. The annealing process, with the temperature decreasing from high to low, gradually makes the assignments approach discrete hard assignments from uniform soft assignments, ensuring the consistency between training and service.

[0121] Step 4: Jointly optimize the objective function training.

[0122] The overall loss function consists of two parts: recommendation loss and index regularization loss. ; in, For the total loss, To recommend losses, To balance the loss of the index, The loss is the hierarchical consistency loss, where α and β are the weighting coefficients of the two regularization loss terms, respectively.

[0123] Recommended loss Based on user behavior data (play, favorite, skip, etc.), a sampled softmax loss function is used to optimize the prediction accuracy of the recommendation model at each level. ; The meaning of this formula is: for each level l and for each pair of user u and positive samples... Calculate the logarithm of the ratio of the exponent of the positive sample score to the sum of the exponents of the positive and negative sample scores, and sum over all levels and sample pairs, taking the negative. Positive samples are songs that users interact with positively (such as playing or adding to favorites); These are the negative samples obtained through sampling. The higher the score of the positive samples, the smaller the loss, thereby driving the model to learn the user's preference for music.

[0124] Index Balance Loss To prevent songs from being overly concentrated on a few index nodes, and to ensure a relatively balanced number of songs across all nodes: ; The formula means that for each index node k at each level l, first sum the distribution probabilities of all M songs at that node and divide by M to obtain the average distribution probability of that node. Then, calculate its value compared to the ideal value 1 / N under a uniform distribution. l The square of the deviation between them is summed over all nodes and levels. Where M is the total number of songs in the music library, and N... l Let be the number of index nodes in the l-th layer. This loss can suppress index degradation (i.e., songs are concentrated in a few popular nodes), making the computational load of each node more balanced during retrieval.

[0125] Hierarchical consistency loss : Ensure that index allocation between layers satisfies the containment relationship, that is, the allocation in the upper layer is semantically consistent with the allocation in the lower layer: ; The formula means: for each level l (except the last level) and each song m, calculate the direct assignment probability of the song in level l. The KL divergence between the probabilities allocated in layer (l+1) and the probabilities obtained by aggregating them to layer l using the aggregation function Aggregate is summed. Let KL divergence be denoted as KL divergence, and Aggregate be a mapping function that aggregates lower-level assignments to upper-level assignments based on the hierarchical structure. This loss can reduce the possibility of missed or missed responses during layer-by-layer retrieval due to conflicting assignments between upper and lower levels.

[0126] Regarding the implementation logic of the aggregation function Aggregate: In this embodiment, the index path of each song can be represented as an L-dimensional code code(m) = (c1, c2, ..., cL), where cl represents the node affiliation of the song at level l. The Aggregate function utilizes the hierarchical encoding structure of the tree to achieve bottom-up aggregation through prefix truncation, i.e., a_m(l) = (c1, c2, ..., cl). The affiliation of a song at level l is uniquely determined by the first l bits of its index code. This ensures that the affiliation of a song at each level forms a complete and valid path from root to leaf, satisfying the path consistency constraint of the tree structure. When soft allocation is used during the training phase, Aggregate degenerates into a layer-by-layer summation of probabilities, i.e., the affiliation probability of node k at level l is equal to the sum of the affiliation probabilities of all its child nodes. ; Wherein, children(k) represents the set of child nodes covered by node k in layer l in layer l+1.

[0127] Step 5: Dynamic index update.

[0128] We designed an online incremental update strategy to adapt to the dynamic changes in the music library, which includes three mechanisms.

[0129] New song index initialization: When a new song is released, its initial representation is calculated through forward propagation of the music tower based on the content features of the new song (audio features, lyrics features, singer information, etc.). Initial soft assignment is then performed based on the similarity between this representation and the central representation of each index node, so that the new song can obtain index ownership and participate in recall even in the absence of user behavior data.

[0130] Index periodic re-optimization: Set an update period T, and every T time windows, based on the user behavior data accumulated within that period, allocate parameter z to the index. m (l) Incremental gradient updates are performed to adapt the index structure to changes in user interests and song popularity.

[0131] Popularity-aware indexing adjustment: This involves monitoring the average exposure and click-through rate of songs within each index node. When a node becomes overheated (due to excessive popular songs causing a retrieval bottleneck) or underheated (songs within the node have long-term low exposure), a re-indexing process is triggered. The criteria for determining overheating or underheating are: the system continuously monitors the capacity ratio (the ratio of the number of songs in a node to the average number of songs in that layer) and traffic ratio (the ratio of the average exposure of a node to the average exposure of that layer) of each index node using a sliding window of N days. If the product of the capacity ratio and traffic ratio is greater than a corresponding threshold, it is considered overheated; if it is less than the threshold, it is considered underheated. The specific re-indexing process is as follows: For overheated nodes, songs within the node are re-clustered using K-means based on their representation vectors, and some sub-clusters are migrated to idle or low-load nodes in the same layer to achieve load balancing. For underheated nodes, remaining songs within the node are migrated to the nearest active node based on representation similarity, and empty nodes are reclaimed for future use. After migration, path maintenance is performed, i.e., the node number in the corresponding layer of the index encoding is updated synchronously, and the consistency of the upper and lower layer paths is maintained through the Aggregate function.

[0132] Step six: Hierarchical retrieval reasoning.

[0133] Figure 4 This is a schematic diagram of the hierarchical retrieval reasoning process according to an embodiment of the present disclosure, such as... Figure 4 As shown in the diagram, this illustration uses a hierarchical index structure containing L1, L2, and song layers as an example to demonstrate the process of performing beamsearch layer by layer during online service: In L1 layer retrieval, the user node preference score is calculated for all four index nodes in L1 layer, and the two nodes with the higher scores are selected (Top-K1=2), while the remaining nodes are not selected; In L2 layer retrieval, the score is calculated only for the six L2 child nodes covered by the two selected L1 nodes, and three of them are selected (Top-K2=3); In song layer retrieval, the score is calculated only for the songs covered by the three selected L2 nodes, resulting in the final Top-K songs, and a candidate song list is output. The computational workload decreases layer by layer throughout the process.

[0134] Specifically, the search process for online services is as follows: In the first level, the model of the first level is used to calculate the user's preference score for all N1 index nodes, and the Top-K1 nodes are selected.

[0135] In the second level, preference scores are calculated using the second-level model only within the child nodes covered by the selected K1 first-level nodes, and the Top-K2 nodes are selected.

[0136] The process proceeds layer by layer until the final layer selects the top-K candidate songs, which are then sent to the subsequent fine-tuning stage. By narrowing down the candidate range layer by layer, the total computational complexity is reduced from O(M) to traversing the entire music library to O(K1·N2 / N1+ K2·N3 / N2+…), where O(·) represents the order of magnitude of computational complexity, K1·N2 / N1 approximately represents the number of nodes that need to be calculated in the second layer (i.e., the average total number of child nodes covered by the K1 selected upper-level nodes), and so on, thereby achieving sublinear inference efficiency.

[0137] In addition to the above implementation methods, this embodiment can also be implemented in the following ways: First, the modular neural network can be replaced with various model structures, such as recommendation models like DeepFM and DIN. Second, the establishment of the model hierarchy structure can adopt multiple quantization schemes such as RQ-VAE and FSQ to uniformly realize the establishment of the index structure (corresponding to step S1) and the allocation of the index path (corresponding to step S3). Specifically, the RQ-VAE (Residual Quantization Variational Autoencoder) scheme uses multi-level residual quantization to quantize the continuous representation of a song layer by layer into discrete codes code(m) = (c1, c2, ..., cL). The codebook vector of each level is the central representation of the corresponding level's index node, and the codebook size is the number of nodes in that level. The quantization encoder and the recommendation model are jointly trained, and the codebook and encoder parameters are updated through backpropagation of the gradient of the recommendation loss (in conjunction with a straight-through estimator, used to approximate the gradient of the discrete quantization operation), achieving end-to-end learning of the index structure. The FSQ (Finite Scalar Quantization) scheme projects the song representation into an L-dimensional space and performs deterministic finite scalar quantization on each dimension to directly obtain the index codes of each layer. This scheme does not require maintaining an explicit codebook, avoiding the codebook collapse problem and making it simpler to implement. The common point of both schemes is that they unify tree construction and song allocation to tree nodes into a differentiable encoding learning process, enabling the index structure to be optimized end-to-end for the recommendation goal.

[0138] In summary, this embodiment models the allocation of songs to index nodes at each level as a differentiable parameter, enabling end-to-end joint training of the hierarchical index structure and the recommendation model with recommendation loss as the objective. This eliminates the inconsistency between index construction and model training objectives in the two-stage method, making the index structure more adaptable to recommendation tasks. The soft allocation during the training phase allows songs to be associated with multiple index nodes with varying probabilities, increasing the chances of long-tail songs being discovered by different types of users. Combined with the temperature annealing strategy, the soft allocation gradually approaches the hard allocation during the service phase, reducing the difference between training and inference. The online incremental update mechanism allows the index structure to continuously adapt to changes in the music library, such as new song releases and popularity shifts, alleviating the cold start problem for new songs. The hierarchical index, combined with layer-by-layer beam search retrieval, can meet the online real-time recommendation needs of large-scale music libraries.

[0139] This embodiment also provides a music recommendation device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0140] This embodiment provides a music recommendation device, such as... Figure 5 As shown, it includes: The acquisition module 501 is used to acquire the user characteristics of the users to be recommended. The recommendation module 502 is used to determine the matching degree between the index node and the user to be recommended in the pre-built hierarchical index structure based on user characteristics, and to filter the index node layer by layer according to the matching degree to obtain the target song for music recommendation to the user to be recommended. The hierarchical index structure comprises multiple levels, which are used to organize multiple songs in the music library. Each level includes multiple index nodes, and in any two adjacent levels, at least some index nodes in one level cover multiple index nodes in the other level. Each song is assigned to an index node in each level of the hierarchical index structure based on an assignment probability. The assignment parameters used to determine the assignment probability and the matching parameters used to determine the matching degree are obtained through joint optimization. The optimization objective of the joint optimization includes a recommendation loss, which is constructed based on the behavioral data of sample users.

[0141] In one alternative embodiment, the device further includes: Module 503 is used for: Song representation is determined based on allocation probability; The prediction results are determined based on the user characteristics of the sample users and the song representation. The prediction results are used to characterize the sample users' preference for the songs. The recommendation loss is constructed based on the prediction results and the behavioral data of the sample users, and the matching parameters and assignment parameters are updated based on the recommendation loss.

[0142] In one alternative implementation, the construction module 503 is configured to: The song representation is obtained by weighting the node representations of each index node at the song's level according to the assigned probability.

[0143] In one alternative implementation, the construction module 503 is configured to: The song's allocation parameters for each index node in the corresponding level are scaled according to the temperature parameter. Perform exponential operations on each of the scaled allocation parameters; The results of each exponent operation are normalized to obtain the probability of the song being assigned to each index node. The temperature parameter is used to control the smoothness of the distribution of the allocation probability; during the joint optimization process, the temperature parameter decreases as the joint optimization progresses.

[0144] In one alternative implementation, the optimization objective of the joint optimization also includes at least one of index balancing loss and hierarchy consistency loss; Index balancing loss is used to constrain the balance of the number of songs allocated to each index node; Hierarchical consistency loss is used to constrain the consistency between the allocation probability of a song in a hierarchy and the allocation probability obtained by aggregating the allocation probabilities in adjacent hierarchy levels.

[0145] In one alternative implementation, the matching degree is determined by a recommendation model, which includes a user encoding subnetwork, a music encoding subnetwork, and an interaction subnetwork. The user coding subnetwork is used to encode user features to obtain user representations; The music coding subnetwork is used to encode the music features of the index nodes to obtain music representations; The interaction subnetwork is used to determine the interaction representation that represents the interaction relationship between the user and the music based on the intermediate layer features of the user coding subnetwork and the music coding subnetwork. The matching degree is determined based on user representation, music representation, and interaction representation.

[0146] In one alternative implementation, the recommendation models corresponding to different levels share at least some parameters in the user coding subnetwork and the music coding subnetwork, and the recommendation models corresponding to different levels have their own independent interaction subnetworks.

[0147] In one alternative implementation, module 502 is recommended for: The current level of filtering is performed on the index nodes covered by the selected index node in the previous level, in order of increasing number of index nodes in the hierarchy. In each level, index nodes whose matching degree meets preset conditions are selected.

[0148] In an alternative implementation, the construction module 503 is further configured to: In response to the addition of a new song to the music library, the representation of the new song is determined based on its content characteristics, and the index node to which the new song belongs in each level of the hierarchical index structure is determined based on the similarity between the representation and the node representation of each index node.

[0149] In an alternative implementation, the construction module 503 is further configured to: Obtain the metric information for each index node, including at least one of the following: the number of songs belonging to the index node and the exposure of songs belonging to the index node. If the indicator information of any index node meets the preset redistribution conditions, the index node to which the song belongs is reassigned is determined.

[0150] In one alternative implementation, each song belongs to a unique index node at each level of the hierarchical index structure, and the unique index node is the index node with the highest probability of being assigned to the song at that level.

[0151] The music recommendation device provided in this disclosure can execute the music recommendation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0152] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0153] The following is a detailed reference. Figure 6 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0154] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0155] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the music recommendation method of embodiments of this disclosure.

[0156] Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0157] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the music recommendation method shown in the above embodiments.

[0158] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0159] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A music recommendation method, characterized in that, The method includes: Obtain user characteristics of users to be recommended; In the pre-built hierarchical index structure, the matching degree between the index node and the user to be recommended is determined layer by layer based on the user characteristics, and the index node is filtered layer by layer according to the matching degree to obtain the target song for music recommendation to the user to be recommended. The hierarchical index structure includes multiple levels for organizing multiple songs in a music library. Each level includes multiple index nodes, and in adjacent levels, at least some index nodes in one level cover multiple index nodes in another level. Each song is assigned to an index node in each level of the hierarchical index structure based on an allocation probability. The allocation parameter used to determine the allocation probability and the matching parameter used to determine the matching degree are obtained through joint optimization. The optimization objective of the joint optimization includes a recommendation loss, which is constructed based on the behavioral data of sample users.

2. The method according to claim 1, characterized in that, The joint optimization includes: The song representation is determined based on the allocation probability; The prediction result is determined based on the user characteristics of the sample users and the song representation, and the prediction result is used to represent the degree of preference of the sample users for the song; The recommendation loss is constructed based on the prediction results and the behavioral data of the sample users, and the matching parameters and the allocation parameters are updated based on the recommendation loss.

3. The method according to claim 2, characterized in that, The process of determining the song representation based on the allocation probability includes: The node representations of each index node at the level where the song is located are weighted according to the allocated probabilities to obtain the song representation.

4. The method according to any one of claims 1 to 3, characterized in that, The allocation probability is obtained in the following way: The song is scaled according to the temperature parameter for each index node in the corresponding level, based on the allocation parameters of each node. Perform exponential operations on each of the scaled allocation parameters; The results of each exponential operation are normalized to obtain the song's allocation probability on each index node. The temperature parameter is used to control the smoothness of the distribution of the allocation probability; during the joint optimization process, the temperature parameter decreases as the joint optimization progresses.

5. The method according to claim 1, characterized in that, The optimization objective of the joint optimization also includes at least one of index balancing loss and hierarchical consistency loss; The index balance loss is used to constrain the balance of the number of songs assigned to each index node; The hierarchical consistency loss is used to constrain the consistency between the allocation probability of a song in a level and the allocation probability obtained by aggregating the allocation probabilities in adjacent levels.

6. The method according to claim 1, characterized in that, The matching degree is determined by a recommendation model, which includes a user encoding subnetwork, a music encoding subnetwork, and an interaction subnetwork. The user coding subnetwork is used to encode the user features to obtain a user representation; The music coding subnetwork is used to encode the music features of the index nodes to obtain music representations; The interaction subnetwork is used to determine the interaction representation that represents the interaction relationship between the user and the music based on the intermediate layer features of the user coding subnetwork and the music coding subnetwork. The matching degree is determined based on the user representation, the music representation, and the interaction representation.

7. The method according to claim 6, characterized in that, The recommendation models corresponding to different levels share at least some parameters of the user encoding subnetwork and the music encoding subnetwork, and the recommendation models corresponding to different levels have their own independent interaction subnetworks.

8. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the music recommendation method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the music recommendation method according to any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the music recommendation method according to any one of claims 1 to 7.

Images