A label-based recommendation algorithm for heterogeneous graphs

Abstract

The present invention discloses a label-based heterogeneous graph recommendation algorithm. The algorithm comprises the following steps: (1): constructing a heterogeneous graph: constructing three heterogeneous relationship graphs of user-item, user-label and item-label; (2): initialization: associating an embedding representation with each node in the graph and initializing; dividing the embedding representation into multiple separated blocks, each block corresponding to an attribute; (3): heterogeneous graph decoupling layer; step (4): high-order information propagation: stacking L layers of heterogeneous graph decoupling layers, with the output of the previous layer serving as the input of the next layer; step (5): model training: using the BPR method to train the model; step (6): model testing. The present invention fully exploits label information, reduces sparsity, and combines information propagation and neighborhood routing mechanisms to automatically identify multi-attribute problems, decouple multiple preferences of users and multiple features of items, thereby providing interpretability for the model and improving the performance of the recommendation model.

Description

Technical Field

[0001] The present invention belongs to the field of information processing, and in particular relates to a label-based heterogeneous graph recommendation algorithm. Background Art

[0002] With the advent of the information age, information overload is becoming more and more serious. Recommendation algorithms have been widely studied as an important means to solve this problem. It can find data that meets the personalized needs of users from massive data, and then automatically recommend it to users. It has been widely used in various platforms, such as multimedia, online shopping, etc. Among them, a large number of studies use the interaction between users and items to obtain the representation of users and items. Although good results have been achieved, they are prone to sparsity problems, that is, the proportion of interactive data is very low, and it is difficult to learn accurate user representations through a small amount of information. Therefore, a large number of studies began to introduce various additional data (text, pictures). For example, many studies use labels as a kind of meta-information, obtain the label-based representation of users and items through neural networks, and then combine collaborative signals for recommendation. Or use the popular graph convolution model to add labels to the collaborative graph, and then propagate the information of the expanded nodes to alleviate the sparsity problem. Although the sparsity problem is solved to a certain extent, good performance has been achieved. However, these models cannot provide a reliable explanation for high performance, making them not robust.

[0003] The reason why the performance is not optimal is that it does not take into account the fact that users have multiple preferences and items also have multiple attributes, and entangles messages on different attribute dimensions. In order to decouple the reasons behind the interaction, some studies have introduced decoupling graph models into this field, aiming to decouple the main reasons behind the interaction, and then decouple the multiple attribute information of users and items. However, the inference performance of this decoupling method based only on interaction information cannot reach the optimal level. Summary of the invention

[0004] In order to simultaneously solve the problems of sparsity and multiple attributes of nodes in the graph, the present invention proposes a recommendation algorithm that decouples the multiple attributes of users and items based on labels, and introduces labels to expand the information of users and items, so that the model can accurately obtain the information of users and items on each attribute, automatically identify the main reasons behind the interaction, and achieve better recommendation performance and explainability.

[0005] The technical solution to achieve the purpose of the present invention is: a label-based heterogeneous graph recommendation algorithm, comprising the following steps:

[0006] Step (1): Construct heterogeneous graphs: Use the user’s annotation history to construct three heterogeneous relationship graphs: user-item, user-label, and item-label.

[0007] Step (2): Initialization: Associate an embedded representation with each node in the graph and initialize it; divide the embedded representation into multiple separate blocks, each block corresponding to an attribute;

[0008] Step (3): Heterogeneous graph decoupling layer: 1) The distribution of the edge on each attribute is calculated based on the embedded representation of the nodes at both ends of the edge on each attribute, and the heterogeneous graph is divided into multiple heterogeneous subgraphs related to each attribute; 2) Graph convolution is performed on the heterogeneous subgraphs corresponding to each attribute, and the information of the node on each attribute under the relationship is expanded;

[0009] 3) Calculate the degree of attention of nodes to the attributes based on the updated attribute information on each heterogeneous graph; 4) Fuse the different heterogeneous information on each attribute to obtain the overall representation of the node on the attribute; 5) Repeat the above steps T times to make the model converge and automatically identify the distribution of edges on each attribute and the degree of attention of nodes to each attribute;

[0010] Step (4): High-order information propagation: stack L layers of heterogeneous graph decoupling layers, use the output of the previous layer as the input of the next layer, take the average of the outputs of all layers as the final representation of the node, and use the inner product to represent the matching degree between the user and the item;

[0011] Step (5): Model training: Use the BPR method to train the model, consider items that users have interacted with as positive samples, and items that users have not interacted with as negative samples, and expect the matching score of positive samples to be greater than that of negative samples;

[0012] Step (6): Model testing: After training, use the above model to obtain the embedded representation of users and items. The inner product is used to obtain the matching scores of users to all items. The items are sorted and the top-k items with the highest scores are selected as recommendations to test the accuracy of the recommendations.

[0013] Compared with the prior art, the present invention has the following significant advantages:

[0014] (1) The present invention introduces label information and uses graph convolution to effectively solve the sparsity problem and the multi-word synonymy problem of the label itself;

[0015] (2) The neighborhood routing mechanism is used to decouple the multiple attributes of nodes and obtain the attention scores of nodes on different attributes, which provides interpretability for the model, making the model more robust and the recommendation accuracy higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of constructing heterogeneous graphs and overall framework by annotating history.

[0017] Figure 2 It is the model structure diagram of the interpretable heterogeneous graph decoupling layer. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0019] like Figure 1 , 2 As shown in Figure 1, a label-based heterogeneous graph recommendation algorithm is shown in Figure 2. The specific steps are as follows:

[0020] Step (1) first construct a heterogeneous graph based on the annotation history of all users. Specifically, a annotation record can be represented as α =<u,i,t> , indicating that user u has labeled item i with label t. From all the annotation histories, according to the relationship between entities, three heterogeneous graphs can be obtained, where<u,i> Indicates that there is an interactive relationship between user u and user i,<u,t> Indicates that user u has used tag t,<i,t> Indicates that item i is labeled with label t.

[0021] Step (2), initialization. Associate a d-dimensional vector with each node (user, item, label) in the graph represents the information of the node and is randomly initialized using Xavier; it is then divided into K independent blocks, each representing the information of the node on each attribute, which can be expressed as e = (e1, e2, ..., e K ).in

[0022] Step (3) uses the neighborhood routing mechanism to automatically identify the probability distribution of edges on each attribute and the importance of nodes to each attribute, achieving graph decoupling and thus providing interpretability for the model. The specific steps are as follows:

[0023] Step 3-1: Determine the probability distribution of the edge on each attribute: Based on the information of the central node and the neighboring nodes on different attributes, calculate the closeness of the central node μ and its neighboring node ν on each attribute in the jth iteration Determine which attribute causes this edge, and then obtain the probability distribution of the edge on each attribute Thus, the graph is divided into multiple subgraphs related to the attributes, and the formula is as follows:

[0024]

[0025]

[0026] in represents transpose, They represent the similarity score and probability distribution of the central node μ and its neighboring node ν on the kth attribute in the jth iteration. In the 0th round, the node is represented as the initialized embedding representation, that is,

[0027] Step 3-2: Update the information propagated from each attribute on each heterogeneous graph. After dividing each heterogeneous graph into multiple attribute-related subgraphs, perform graph convolution on each subgraph, receive neighborhood information, and expand the node's information on the attribute. Use the best graph convolution model, lightgcn, with the following formula:

[0028]

[0029] in Respectively expressed in In the relationship, the degree of node μ, ν on the kth attribute in the jth iteration is Indicates that the node μ is Neighborhood in a relationship.

[0030] Step 3-3, update the attention of each node to each attribute in each relationship: after obtaining the information of each attribute of the node in various relationships, for different heterogeneous information, the node pays different attention to its various attributes, and the node's information on the attribute is used to determine its attention to various attributes. The calculation formula is as follows:

[0031]

[0032]

[0033] in Indicates that the node μ is The importance of the k-th attribute under the relationship, is the corresponding probability distribution. for Attention vector under relation.

[0034] Step 3-4: Fusion of information of each attribute on each heterogeneous graph. Fusion of heterogeneous information on various attributes from various relationships to obtain complete information representation of the node on the attribute. The calculation formula is as follows:

[0035]

[0036] where ψ μ is the set of heterogeneous relations contained in node μ. is the complete information of node μ on the kth attribute after the jth iteration.

[0037] Step 3-5: Repeat the above steps to make and Close to the true distribution. After T iterations, As the neighborhood information of node μ on the kth attribute, the graph convolution layer is packaged into a function f, which can be expressed as:

[0038]

[0039] in represents the neighborhood message extended by node μ on the kth attribute, It represents the distribution of historical interactions μ-v on various attributes, from which we can know the main factors that form this interaction. Indicates the importance of each attribute to the node μ. When making recommendations, items related to the attributes that the user values ​​can be recommended.

[0040] Step (4), high-order information propagation: Take the previous-order neighborhood information of the node as the new information of the node, and continue to perform the above graph convolution, then you can receive the information propagated from higher-order neighborhoods. The formula can be expressed as:

[0041]

[0042] Where f represents the graph convolutional layer proposed in step (3). By stacking L layers, we can obtain L-order neighborhood information and integrate the information propagated from each order of neighborhood, that is, Then all attributes are concatenated as the final embedding representation of the node, i.e., e μ =e μ1 ||e μ2 ||...||e μK . Perform the above operations on users, items, and tags.

[0043] Step (5), model training: After the above steps, the embedding representations of all users and items can be obtained. u and e i , and then use the inner product to represent the user's preference for the item, that is In order to improve the performance, the popular BPR is used for model optimization, and the loss function can be expressed as:

[0044]

[0045] in is the set of items that the user has interacted with in the training set, i.e., positive samples. is a set of items that have not been interacted with, i.e., negative samples. At the same time, in order to prevent overfitting, a regularization term is added to the model, and the objective function is:

[0046] loss=loss BPR +||Θ||2 (10)

[0047] Where Θ is the parameter of this model.

[0048] Step (6), model testing: After the model is trained, the embedded representations of all users and items can be obtained according to steps (3) and (4). The inner product can be used to obtain the user's preference scores for all items, sort them, and take the top-k items with the largest scores as recommended items. At the same time, the probability distribution of the edge on each attribute and the attention distribution of each node on each attribute are obtained. The main relationship behind each interaction is revealed to provide interpretability for the model.

Claims

1. A label-based heterogeneous graph recommendation algorithm, characterized in that: The following steps are involved: Step (1): Construct heterogeneous graphs: Use the user’s annotation history to construct three heterogeneous relationship graphs: user-item, user-label, and item-label. Step (2): Initialization: Associate an embedded representation with each node in the graph and initialize it; divide the embedded representation into multiple separate blocks, each block corresponding to an attribute; Step (3): Heterogeneous graph decoupling layer: 1) Calculate the distribution of the edge on each attribute based on the embedded representation of the nodes at both ends of the edge on each attribute, and divide the heterogeneous graph into multiple heterogeneous subgraphs related to each attribute; 2) Perform graph convolution on the heterogeneous subgraphs corresponding to each attribute, and expand the information of the node on each attribute under this relationship; 3) Calculate the degree of attention of the node to the attribute based on the attribute information updated on each heterogeneous graph; 4) Fuse the different heterogeneous information on each attribute to obtain the overall representation of the node on the attribute; 5) Repeat the above steps T times to make the model converge, and automatically identify the distribution of the edge on each attribute and the degree of attention of the node to each attribute; Step (4): High-order information propagation: stack L layers of heterogeneous graph decoupling layers, use the output of the previous layer as the input of the next layer, take the average of the outputs of all layers as the final representation of the node, and use the inner product to represent the matching degree between the user and the item; Step (5): Model training: Use the BPR method to train the model, consider items that users have interacted with as positive samples, and items that users have not interacted with as negative samples, and expect the matching score of positive samples to be greater than that of negative samples; Step (6): Model testing: After training, use the above model to obtain the embedded representation of users and items. The inner product is used to obtain the matching scores of users to all items. The items are sorted and the top-k items with the highest scores are selected as recommendations to test the accuracy of the recommendations.

2. The algorithm according to claim 1, characterized in that The step (1) is specifically implemented as follows: using the user's annotation history, constructing three heterogeneous relationship graphs of user-item, user-label, and item-label. Specifically, a annotation record can be represented as α=<u,i,t> , indicating that user u has labeled item i with label t. From all the annotation histories, according to the relationship between entities, three heterogeneous graphs can be obtained, where<u,i> Indicates that there is an interactive relationship between user u and user i,<u,t> Indicates that user u has used tag t,<i,t> Indicates that item i is labeled with label t.

3. The algorithm according to claim 2, characterized in that The step (2) is specifically implemented as follows: Associate a d-dimensional vector with each node in the heterogeneous graph Represents the information of the node and uses xavier for random initialization; Then it is divided into K independent blocks, which represent the information of the node on each attribute, which can be expressed as e = (e1, e2, ..., e K ),in 4. The algorithm according to claim 3, characterized in that The step (3) is specifically divided into the following steps: Step (3-1): Determine the probability distribution of the edge on each attribute: Based on the information of the central node and the neighboring nodes on different attributes, calculate the closeness between the central node μ and its neighboring node ν on each attribute in the jth iteration Determine which attribute causes this edge, and then obtain the probability distribution of the edge on each attribute Thus, the graph is divided into multiple subgraphs related to the attributes, and the formula is as follows: in represents transpose, They represent the similarity score and probability distribution of the central node μ and its neighboring node ν on the kth attribute in the jth iteration respectively; in the 0th round, the node is represented as the initialized embedding representation, that is, Step (3-2): Update the information propagated by each attribute on each heterogeneous graph; after dividing each heterogeneous graph into multiple subgraphs related to the attribute, perform graph convolution on each subgraph, receive neighborhood information, and expand the information of the node on the attribute; use the graph convolution model lightgcn, the formula is as follows: in Respectively expressed in In the relationship, the degree of node μ, ν on the kth attribute in the jth iteration is Indicates that the node μ is Neighborhood in a relationship; Step (3-3): Update the attention of each node to each attribute under each relationship: After obtaining the information of each attribute of the node in various relationships, for different heterogeneous information, the node pays different attention to its various attributes. The node's information on the attribute is used to determine its attention to various attributes; the calculation formula is as follows: in Indicates that the node μ is The importance of the k-th attribute under the relationship, is the corresponding probability distribution; for Attention vector under relation; Step (3-4): Fuse the information of each attribute on each heterogeneous graph; fuse the heterogeneous information on various attributes from various relationships to obtain the complete information representation of the node on the attribute; The calculation formula is as follows: where ψ μ is the set of heterogeneous relations contained in node μ, is the complete information of node μ on the kth attribute after the jth iteration; Step (3-5): Repeat the above steps to make and Close to the true distribution; After T iterations, As the neighborhood information of node μ on the kth attribute, the graph convolution layer is packaged into a function f, which can be expressed as: in represents the neighborhood message extended by node μ on the kth attribute, represents the distribution of historical interactions μ-v on various attributes, from which we can know the main factors that form this interaction; Indicates the importance that node μ places on each attribute.

5. The algorithm according to claim 4, characterized in that The step (4) is specifically implemented as follows: By taking the previous-order neighborhood information of the node as the new information of the node and continuing to perform the above graph convolution, the information propagated from higher-order neighborhoods can be received. The formula can be expressed as: Where f represents the graph convolutional layer proposed in step (3). By stacking L layers, we can obtain L-order neighborhood information and integrate the information propagated from each order of neighborhood, that is, Then all attributes are concatenated as the final embedding representation of the node, i.e., e μ =e μ1 ||e μ2 ||...||e μK ; Perform the above operations on users, items, and tags.

6. The algorithm according to claim 5, characterized in that The step (5) is specifically implemented as follows: Get the embedding representations of all users and items u and e i Then, the inner product is used to represent the user's preference for the item, that is, Using the popular BPR for model optimization, the loss function can be expressed as: in is the set of items that the user has interacted with in the training set, i.e., positive samples. is a set of items that have not been interacted with, that is, negative samples; at the same time, in order to prevent overfitting, a regular term is added to the model, and the objective function is: loss=loss BPR +||Θ||2 (10) Where Θ is the parameter of this model.

7. The algorithm according to claim 6, characterized in that The step (6) is specifically implemented as follows: After the model is trained, the embedded representations of all users and items can be obtained according to steps (3) and (4). The inner product can be used to obtain the user's preference scores for all items, sort them, and take the top-k items with the largest scores as recommended items. At the same time, the probability distribution of the edges on each attribute and the attention distribution of each node on each attribute are obtained; the main relationship behind each interaction is revealed, providing explainability for the model.

Images