A music recommendation method based on double-path heterogeneous graph link prediction

Abstract

The application discloses a music recommendation method based on double-path heterogeneous graph link prediction, comprising the following steps: S1, based on a graph neural network, adding a random label feature matrix to a music label feature matrix representing each node to form a locally enhanced feature matrix as a basic model of a path; S2, taking an improved graph attention network GATv2 based on a dynamic attention mechanism as a basic model of another path, and introducing residual connection and L2 normalization on the two paths to prevent model overfitting; S3, after obtaining node embedding through the double-path structure, introducing a correction module and combining it with self-supervised auxiliary learning of the original model, and using the correction module to better balance the weight of the auxiliary task, so as to perform a downstream link prediction task. The link prediction based on the heterogeneous graph music recommendation realizes better effects in accuracy and recall rate.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of link prediction, and particularly relates to a music recommendation method based on double-path heterogeneous graph link prediction. BACKGROUND

[0002] Link prediction is to predict the possibility of the link between two nodes in a network, which can be used in recommendation systems, knowledge graphs and other directions. The rapid development of networking technology and digital music provides an open and fast music listening platform for users. Users can search and download digital music through the Internet. However, due to the huge number of daily released digital music, users need to spend a lot of time to find the songs they are interested in from the vast amount of digital music. The emergence of music personalized recommendation technology solves the above problem. Generally, link prediction methods can be divided into three categories. One is the heuristic method based on scoring. All these methods have a premise that the more similar the nodes are, the greater the possibility of generating a link. For example, for Node2vec, it defines a heuristic score according to a pair of node neighbors, and then sets a threshold according to the score, so that the edges between node pairs can be predicted. The other is the method based on deep learning. For example, the method proposed by Liben-Nowell et al.: first obtain node features through a deep learning-based method, and then use a classifier for prediction. The last one is the method based on graph representation learning. Specifically, these methods usually use graph neural networks (GNN) to obtain node embeddings, and then obtain edge features by binary operation on node embeddings. In this case, various classification methods can be applied to edge features for prediction. Among the three categories, the method based on graph representation learning is the most commonly used method in link prediction. Therefore, how to obtain more meaningful node embeddings through GNN becomes particularly important.

[0003] The concept of GNN was first proposed by Scarselli et al. GNN is a branch of neural networks that processes data on graphs, where each node in the graph is represented by its features and adjacent nodes. In recent years, as GNN has gradually become a research hotspot, researchers have also begun to study GNN in different directions. Among them, heterogeneous graphs are the most commonly encountered graphs in daily life, so HGNN (Heterogeneous Graph Neural Network) has also developed greatly in this period. In order to make GNN better apply to heterogeneous graphs, researchers have proposed various HGNNs (Heterogeneous Graph Neural Network), such as HAN (Heterogeneous Graph Attention Network), SELAR (SELf-supervised Auxiliary LeaRning), and so on. These models have greatly promoted the development of HGNN. Among them, HAN is a model that applies attention mechanisms to heterogeneous graphs, and these models can also be understood as variants of GAT. Graph attention mechanisms have always been an important branch of GNN. GAT can assign different importance coefficients to neighbor nodes of different sizes and does not require knowledge of the entire graph structure, thereby making up for the many shortcomings of previous spectral-based methods.

[0004] As a branch of GNN, GAT has always been an important research direction. However, in heterogeneous graphs, due to the inclusion of different types of nodes and edges, the original GAT cannot perform well. Therefore, HAN (Heterogeneous Graph Attention Network) has emerged, combining node-level attention and semantic-level attention. Node-level attention is used to learn the importance of the central node and its different types of neighbors, and semantic-level attention is used to learn the importance of different meta-paths. HAN can capture rich semantic information on heterogeneous graphs, so its performance on heterogeneous graphs is better than GAT. GATv2, which was proposed later, fundamentally solves the problems that GAT as a static attention has, and can also be well applied to heterogeneous graphs.

[0005] GCN is a classic graph neural network. There are currently two ways to interpret GCN, one is based on the spatial domain, which can be understood as directly convolving the pixels of the graph. The other is the spectral domain method, which can be understood as the convolution of the Fourier transform of the graph. These two methods are understood from different angles. The spatial domain-based method is simpler, while the spectral method is more rigorous in derivation. GCN is a subtle method of extracting features from graph data so that they can be used for link prediction on graph data.

[0006] Local augmentation is a form of data augmentation. Data augmentation has been a popular research area in computer vision and natural language processing, but its application in graph neural networks (GNNs) has been limited. Existing graph data augmentation methods can be divided into two categories: topology-level augmentation and feature-level augmentation. Topology-level augmentation perturbs the adjacency matrix, generating different graph structures; on the other hand, existing feature-level augmentation utilizes adversarial training-guided node attribute perturbations to improve generalization ability. However, both have a drawback: they tend to consider global information of the graph rather than the attribute augmentation of individual nodes, thus ignoring the importance of local information. To address this issue, Liu et al. proposed LAGNN (Local Augmentation Graph Neural Networks), which effectively utilizes local information from the neighborhood.

[0007] Auxiliary learning is a variant of multi-task learning, the main difference being that multi-task learning is parallel, with results influencing each other, while auxiliary learning only focuses on the performance of the primary task. Auxiliary learning has been widely applied to many tasks; for example, meta-auxiliary learning proposes a method to generate new auxiliary tasks by folding existing classes. However, it is not suitable for certain tasks, such as link prediction with only one positive class. SELAR treats meta-path prediction as a self-supervised auxiliary task, leveraging meta-learning to automatically select the optimal combination of meta-paths, thereby improving the performance of link prediction on heterogeneous graphs. Summary of the Invention

[0008] The technical problem to be solved by this invention is: how to quickly find songs of interest from a massive amount of digital music when users search for and download digital music on the Internet. This invention provides a music recommendation method based on dual-path heterogeneous graph link prediction.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a music recommendation method based on dual-path heterogeneous graph link prediction, based on the nodes containing the tag features of each type of music, including the following steps: S1, based on graph neural network, add random tag feature matrix to the music tag feature matrix representing each node to form a local enhanced feature matrix;

[0010] S2. Based on this locally enhanced feature matrix, combined with the adjacency matrix and degree matrix after adding self-loops, and the learnable weight matrix, a generative model is constructed; based on the graph convolutional network, the generative model is connected after the first graph convolutional layer of the graph convolutional network to construct a locally enhanced graph convolutional neural network.

[0011] S3. Connect residual connections at the output of the local augmentation graph convolutional neural network, and then perform L2 normalization to obtain the first update node for each node.

[0012] S4. Calculate the importance of the music tag features of the neighboring nodes representing each node to the music tag features of that node, and then perform normalization to obtain the normalized attention coefficient.

[0013] S5. Combining the normalized attention coefficient, the music tag features of the neighboring nodes are weighted and summed to construct a graph attention network based on the dynamic attention mechanism. A residual connection is then connected at the output of the graph attention network based on the dynamic attention mechanism, followed by L2 normalization to obtain the second update node for each node.

[0014] S6. After splicing and merging the first update node and the second update node, the final update node is obtained. Using the auxiliary learning model and the correction module, the music recommendation link prediction results between nodes are obtained.

[0015] Furthermore, the aforementioned step S1 specifically includes:

[0016] The local enhancement feature matrix is ​​obtained using the following formula.

[0017]

[0018] Here, X' is a subset of features extracted from the music tag feature matrix X of the node, and the dimension of X'||R is the same as that of the node. The dimensions are consistent, || represents the join operation, and R represents the random label feature.

[0019] Furthermore, step S2 described above specifically involves constructing a local augmentation graph convolutional neural network according to the following formula:

[0020]

[0021] Among them, H (1) This is the first graph convolutional layer, where X is the node music tag feature matrix. For local enhancement feature matrix, The model representing GCN, Represents a generative model. Let A and D represent the adjacency matrix A and degree matrix D after adding self-loops, respectively; || denotes the join operation; σ denotes the activation function; W is the learnable weight matrix, with its subscripts and superscripts representing the layer number and parameter index, respectively; and let... dim represents dimension.

[0022] Furthermore, in step S3 above, a residual connection is made at the output of the local enhancement map convolutional neural network, followed by L2 normalization, as shown in the following equation:

[0023]

[0024] Among them, H (0) This is the initial enhanced feature matrix. ||·|| represents the L2 norm.

[0025] Furthermore, the aforementioned step S4 includes the following sub-steps:

[0026] S4.1 Calculate the importance of the music tag features of the neighboring node j of node i to the music tag features of node i using the following formula:

[0027] e(h i ,h j ) = a T LeakyReLU(W[h i h j ]),

[0028] in, It is the learning coefficient, the concatenation operation, h i and h j Let i and j represent the music tag features of nodes i and j, respectively. LeakyReLU is a non-linear activation function.

[0029] S4.2. Normalize according to the following formula:

[0030]

[0031] Among them, N i Let h represent the set of neighboring nodes of node i. j and h k Let j and k represent the music tag features of nodes j and k, respectively, and softmax represents the normalization operation.

[0032] Furthermore, the aforementioned step S5 includes the following sub-steps:

[0033] S5.1. Combining the normalized attention coefficient, the features of neighboring nodes are weighted and summed as follows:

[0034]

[0035] Where σ is the nonlinear activation ReLU function;

[0036] S5.2 Obtain the second update node corresponding to each node according to the following formula:

[0037]

[0038] Furthermore, in step S6 above, the correction module is used to better balance the weights of the auxiliary tasks. This module uses an enhanced graph based on virtual nodes, and obtains the updated virtual nodes according to the following formula:

[0039]

[0040] Among them, f θ Represents a linear layer and ReLU, AGG represents aggregating neighbor features in an additive manner, h m It is a virtual node.

[0041] Furthermore, in step S6 above, the music tag features of node c after introducing the virtual node are obtained according to the following formula:

[0042] h c '=f θ (h c +h' m ,AGG({h j |j∈N i})),

[0043] Among them, h c and h c 'represents the music tag features of node c before and after introducing virtual nodes in the augmented graph; h c The weighting function, used to balance auxiliary tasks, is employed in this invention. Compared with existing technologies, the technical effects of the above-mentioned technical solution are as follows:

[0044] This invention proposes a link prediction method based on a dual-path heterogeneous graph. This method obtains node embeddings through two paths. One path is GATv2, which incorporates residual connections and L2 normalization based on a dynamic attention mechanism. The other path is an improved LAGCN, which also incorporates residual connections and L2 normalization. After obtaining node embeddings using the dual-path structure, a correction module is introduced and combined with the self-supervised auxiliary learning of the original model. This correction module is used to better balance the weights of the auxiliary task, thus enabling downstream link prediction. This invention achieves superior accuracy and recall in link prediction, particularly on heterogeneous graphs composed of different types of nodes and edges. This invention not only solves the problem of how to more fully and effectively aggregate neighbor information to learn node embeddings when the number of neighbor nodes is small or even different in type, but also addresses the issue that the attention function is always monotonic with respect to neighbor scores in traditional attention mechanisms for any query node.

[0045] By analyzing users' listening preferences, this approach not only helps users save time searching for songs and find those they're interested in, but also increases user loyalty to music recommendation systems for companies like Xiami Music and NetEase Cloud Music. It represents different types of music in the last.fm online music system as different types of nodes in a heterogeneous graph. Using a DPGNN model, it provides personalized recommendations based on users' music preferences, predicting and recommending song types that users might like. This overcomes the limitation of traditional link prediction recommendation methods, which often rely on a relatively fixed range of recommended music types. Attached Figure Description

[0046] Figure 1 This is a flowchart of the method of the present invention.

[0047] Figure 2 This is a preferred embodiment of the DPGNN network architecture of the present invention. Detailed Implementation

[0048] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0049] In this invention, various aspects of the invention are described with reference to the accompanying drawings, in which numerous illustrative embodiments are shown. Embodiments of the invention are not limited to those depicted in the drawings. It should be understood that the invention is implemented through any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed herein are not limited to any particular implementation. Furthermore, some aspects of the invention disclosed may be used alone or in any suitable combination with other aspects of the invention disclosed.

[0050] like Figure 1 As shown, this invention proposes a music recommendation method based on dual-path heterogeneous graph link prediction, comprising the following steps:

[0051] S1. In the first path, message passing is a common mechanism in GNN models. This mechanism can be viewed as the learned classification or regression function. GNN makes predictions by estimating the posterior distribution of the adjacency matrix A and the feature matrix X. To improve model performance, X is used... i As input to the local enhancement model, it is sampled from the generative model to introduce enhanced feature vectors. For node i. Based on the graph neural network, a random label feature matrix is ​​added to the music label feature matrix representing each node to form a locally enhanced feature matrix.

[0052] The local enhancement feature matrix is ​​obtained using the following formula.

[0053]

[0054] Here, X' is a subset of features extracted from the music tag feature matrix X of the node, and the dimension of X'||R is the same as that of the node. The dimensions are consistent, || represents the join operation, and R represents the random label feature.

[0055] S2. Based on the local enhanced feature matrix, combined with the adjacency matrix and degree matrix after adding self-loops, and the learnable weight matrix, a generative model is constructed; based on the graph convolutional network, the generative model is connected after the first graph convolutional layer of the graph convolutional network to construct a locally enhanced graph convolutional neural network; the adjacency matrix is ​​used to reflect whether there is a relationship between different types of music, and the degree matrix is ​​used to reflect the degree of relationship between various types of music and other types of music.

[0056] Construct a local augmentation graph convolutional neural network using the following formula:

[0057]

[0058] Among them, H (1) This is the first graph convolutional layer, where X is the node music tag feature matrix. For local enhancement feature matrix, The model representing GCN, Represents a generative model. Let A and D represent the adjacency matrix A and degree matrix D after adding self-loops, respectively; || denotes the join operation; σ denotes the activation function; W is the learnable weight matrix, with its subscripts and superscripts representing the layer number and parameter index, respectively; and let... dim represents dimension.

[0059] S3. Connect residual connections at the output of the local augmentation graph convolutional neural network, and then perform L2 normalization to obtain the first update node corresponding to each node.

[0060] A residual connection is made at the output of the local augmentation map convolutional neural network, followed by L2 normalization, as shown in the following equation:

[0061]

[0062] Among them, H (0) This is the initial enhanced feature matrix. ||·|| represents the L2 norm.

[0063] This invention makes the model more powerful by adding random features to each node. Furthermore, the random features can help GNNs solve the minimum dominating set and maximum matching problems by using near-optimal approximation ratios.

[0064] The above describes the structure used in the first path of the dual-path GNN model. This path innovatively introduces an improved locally augmented graph neural network, which applies local augmentation to a graph neural network (GCN). Using GCN as the base model, a novel local augmentation method is proposed, capable of generating more features even with only a few neighbors, thus enhancing the expressive power of the GNN. Furthermore, when there are only a few neighbor nodes and edges, the model is prone to overfitting. The introduction of residual connections and L2 normalization can reduce the occurrence of vanishing gradients or prevent overfitting during training.

[0065] In another path, the dual-path GNN model uses a different structure: a graph attention network based on a dynamic attention mechanism. In this path, the introduction of GATv2 enables the use of dynamic attention to acquire node embeddings. This allows for the aggregation of more meaningful neighbor information during weighted summation, and addresses the issue of attention score ranking being unrestricted by the query node, which could lead to overfitting. Simultaneously, residual connections and L2 normalization reduce the occurrence of gradient vanishing.

[0066] S4. Calculate the importance of the music tag features of the neighboring nodes representing each node to the music tag features of that node, and then perform normalization processing to obtain normalized attention coefficients; including the following steps S4.1 to S4.2:

[0067] S4.1 Calculate the importance of the feature of the neighboring node j of node i to the feature of node i using the following formula:

[0068] e(h i ,h j ) = a T LeakyReLU(W·[h i h j ]),

[0069] in, It is the learning coefficient, the concatenation operation, h i and h j LeakyReLU is a non-linear activation function that represents the features of nodes i and j.

[0070] S4.2. Normalize according to the following formula:

[0071]

[0072] Among them, N i Let h represent the set of neighboring nodes of node i. j and h k Let j and k represent the features of node j and node k respectively, and softmax represent the normalization operation.

[0073] S5. Combining the normalized attention coefficients, the music tag features of neighboring nodes are weighted and summed to construct a graph attention network based on a dynamic attention mechanism. A residual connection is then connected to the output of this graph attention network, followed by L2 normalization to obtain the second updated node for each node. This includes the following sub-steps S5.1 to S5.2:

[0074] S5.1. Combining the normalized attention coefficient, the features of neighboring nodes are weighted and summed as follows:

[0075]

[0076] Where σ is the nonlinear activation ReLU function;

[0077] S5.2 Obtain the second update node corresponding to each node according to the following formula:

[0078]

[0079] S6. After concatenating and fusing the first and second update nodes, the final update node is obtained. Using the auxiliary learning model and the correction module, the music recommendation link prediction result between nodes is obtained. The correction module is used to better balance the weights of the auxiliary task. The graph used in this module is an augmented graph containing virtual nodes, and the updated virtual nodes are obtained according to the following formula:

[0080]

[0081] Among them, f θ Represents a linear layer and ReLU, AGG represents aggregating neighbor features in an additive manner, h m It is a virtual node. The music tag features of node c after introducing the virtual node are obtained according to the following formula:

[0082] h c '=f θ (h c +h' m ,AGG({h j |j∈N i})),

[0083] Among them, h c and h c 'represents the music tag features of node c before and after introducing virtual nodes in the augmented graph; h c 'Weighting functions applied to auxiliary tasks to balance the auxiliary tasks.'

[0084] The main function of the correction module is to correct the answer using more information from the graph. Specifically, it operates on the weight function of the auxiliary task selection, helping the model better balance weights and prevent the auxiliary task from dominating. The graph used by the correction module is an augmented graph containing virtual nodes, where virtual nodes are connected to all nodes in the graph. Before updating node information, the information of the virtual nodes is first propagated to every node in the graph. This invention uses a Generative Neural Network (GNN) to update the node information. Afterward, the node's information in the graph is propagated to the virtual nodes to update their representations. This allows nodes in the graph to quickly exchange information.

[0085] like Figure 2 As shown, this invention uses the LastFM dataset for music prediction. The LastFM dataset is music rating data from the Last.fm online music system, mainly containing music listening information from 2000 users. It contains 15,084 nodes, 73,384 edges, and 122 edge types. This dataset lacks node features; to quantitatively evaluate the proposed DPGNN model, this invention uses the area under the ROC curve (AUC) and prediction accuracy (ACC) as metrics.

[0086] This embodiment uses the PyTorch library in Python to experiment with the proposed DPGNN model. Before training the neural network, the LastFM dataset needs to be preprocessed using KGNN-LS. In the experiment, all models are first set to two layers. In the LastFM dataset, the node embedding dimension is 16, requiring 100 epochs of training, while the neighborhood sampling sample size is 8.

[0087] This embodiment selects eight methods for comparison to evaluate the effectiveness of the representations learned by DPHNN in a dual-path structure. These methods can be divided into two categories. One is based on isomorphic graph models, such as GAT, Graph Transformer Network (GTN), Graph Isomorphism Network (GIN), and GATv2; the other is based on heteromorphic graph models, such as HAN, Heterogeneous Graph Neural Network with Co-Contrastive Learning (HeCo), SELAR, and LAGCN.

[0088] AUC and ACC were used as quantitative indicators to evaluate various models, and the quantitative evaluation results are shown in Table 1.

[0089]

[0090] Note: The last line in bold indicates the best performance, and "*" indicates the next best performance.

[0091] The following conclusions can be drawn from Table 1:

[0092] (1) The DPGNN proposed in this invention consistently outperforms all baselines on the dataset. The results demonstrate the effectiveness of DPGNN in link prediction tasks.

[0093] (2) Clearly, when comparing the two different types of methods, the heterogeneous graph-based model generally outperforms the homogeneous graph-based model. This suggests that considering heterogeneity is very useful for link prediction.

[0094] (3) As a heterogeneous graph-based model, DPGNN outperforms LAGCN and SELAR. SELAR uses GAT as its base model, while LAGCN uses local enhancements in GCN. The DPGNN model proposed in this invention uses both GATv2 and an improved LAGCN as its base models. From the above results, it can be seen that the DPGNN model using GATv2, the improved LAGCN, and the correction module achieves the best performance on heterogeneous graphs.

[0095] In summary, it can be determined that the model proposed in this invention outperforms other baseline models in all performance indicators, achieving the best recommendation effect.

[0096] While the present invention has been described above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A music recommendation method based on dual-path heterogeneous graph link prediction, wherein each type of music corresponds to a node containing the tag features of that type of music, characterized in that... Includes the following steps: S1. Based on graph neural networks, a random label feature matrix is ​​added to the music label feature matrix representing each node to form a local enhanced feature matrix; S2. Based on this locally enhanced feature matrix, combined with the adjacency matrix and degree matrix after adding self-loops, and the learnable weight matrix, a generative model is constructed; based on a graph convolutional network, the generative model is connected after the first graph convolutional layer of the graph convolutional network to construct a locally enhanced graph convolutional neural network, as shown in the following equation: ， Among them, H (1) This is the first graph convolutional layer. The node music tag feature matrix, For local enhancement feature matrix, The model representing GCN, Represents a generative model. , array and Let A and D represent the adjacency matrix and degree matrix after adding self-loops, respectively. Indicates a connection operation. This represents the activation function. Let be a learnable weight matrix, where the subscripts and superscripts represent the layer number and parameter index, respectively; and let , Indicates dimension; S3. Connect residual connections at the output of the local augmentation graph convolutional neural network, and then perform L2 normalization to obtain the first update node for each node. S4. Calculate the importance of the music tag features of the neighboring nodes representing each node to the music tag features of that node, and then perform normalization to obtain the normalized attention coefficient. S5. Combining the normalized attention coefficient, the music tag features of the neighboring nodes are weighted and summed to construct a graph attention network based on the dynamic attention mechanism. A residual connection is then connected at the output of the graph attention network based on the dynamic attention mechanism, followed by L2 normalization to obtain the second update node for each corresponding node. S6. After splicing and merging the first update node and the second update node, the final update node is obtained. Using the auxiliary learning model and the correction module, the music recommendation link prediction result between nodes is obtained. The correction module is used to better balance the weights of auxiliary tasks. The graph used in this module is an augmented graph containing virtual nodes, and the updated virtual nodes are obtained according to the following formula: ， in, Represents linear layers and ReLU. This indicates that neighbor features are aggregated in an additive manner. It is a virtual node; The node after introducing the virtual node is obtained using the following formula. Music tag characteristics: ， in, and These represent the nodes before and after the introduction of virtual nodes in the augmented graph. Music tag features; As a weighting function for auxiliary tasks, it is used to balance auxiliary tasks; Represents a node The set of neighboring nodes, and Representing nodes respectively and nodes Music tag features.

2. The music recommendation method based on dual-path heterogeneous graph link prediction according to claim 1, characterized in that, Step S1 is as follows: The local enhancement feature matrix is ​​obtained using the following formula. : ， in, The music tag feature matrix of the node A portion of the features were extracted, and Dimensions and The dimensions are consistent. Indicates a connection operation. This represents the feature of random labels.

3. The music recommendation method based on dual-path heterogeneous graph link prediction according to claim 2, characterized in that, In step S3, a residual connection is made at the output of the local enhancement map convolutional neural network, followed by L2 normalization, as shown in the following equation: ， Among them, H (0) This is the initial enhanced feature matrix. , This represents the L2 norm.

4. The music recommendation method based on dual-path heterogeneous graph link prediction according to claim 3, characterized in that, Step S4 includes the following sub-steps: S4.1 Calculate the nodes according to the following formula neighboring nodes The importance of music tag features to the music tag features of this node: ， in, It is the learning coefficient. It's a splicing operation. and Representing nodes respectively and nodes Music tag features, It is a non-linear activation function; S4.

2. Normalize according to the following formula: ， in, Represents a node The set of neighboring nodes, and Representing nodes respectively and nodes Music tag features, This indicates a normalization operation.

5. The music recommendation method based on dual-path heterogeneous graph link prediction according to claim 4, characterized in that, Step S5 includes the following sub-steps: S5.

1. Combining the normalized attention coefficient, the features of neighboring nodes are weighted and summed as follows: ， in, Nonlinear activation function; S5.2 Obtain the second update node corresponding to each node according to the following formula: 。

Images