A Recommendation Method Based on Knowledge Graph and Attention Mechanism

Abstract

The present invention relates to personalized recommendation technology, and discloses a recommendation method based on a knowledge graph and an attention mechanism, which solves the problems of data sparsity and poor interpretability existing in traditional recommendation methods, the actual recommendation effect is not good, and it is difficult to meet the actual use requirements. The present invention first aligns the items in the recommendation system with the entities of the knowledge graph to form a training sample set; then constructs a recommendation model including an input layer, an encoding layer, a feature fusion layer and an output layer, uses the training sample set as the input of the recommendation model, and uses the selected loss function as the optimization target to train the recommendation model; finally, calculates the probability that a user adopts a to-be-recommended item based on the trained recommendation model, sorts according to the size of the probability, and generates a recommendation candidate set for the user. It is applicable to the personalized recommendation of various types of content, resources, and items.

Description

Technical Field

[0001] The present invention relates to personalized recommendation technology, and particularly to a recommendation method based on a knowledge graph and an attention mechanism. Background Art

[0002] In recent years, the use of recommendation systems has become increasingly popular on the Internet. From the perspective of users, this type of system helps to alleviate the problems brought by information overload, because users will receive personalized content or resources according to their personal profiles or preferences. On the other hand, product or web content providers are concerned about this type of service because they can attract the interest of customers or users, thereby increasing their sales or the use of the content they provide. For researchers, behind the recommendation system, corresponding technical challenges need to be solved to improve the recommendation results and user experience.

[0003] Existing traditional recommendation methods can be divided into three categories: The first category is the collaborative filtering method, which makes recommendations based on common user preferences and historical interaction data; the second category is the content-based filtering method, which uses the information of items according to the user's profile for further recommendation; the third category is the demographic filtering method, which makes recommendations based on the situation that users with certain common personal attributes (such as gender, age, country, region, etc.) also have common preferences.

[0004] The above three recommendation methods are limited by insufficient interaction between users, do not effectively utilize semantic information, keyword information and hierarchical structure knowledge, have problems of data sparsity and poor interpretability, and the actual recommendation effect is not good, making it difficult to meet the actual use requirements. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: to propose a recommendation method based on a knowledge graph and an attention mechanism to solve the problems of data sparsity, poor interpretability, poor actual recommendation effect and difficulty in meeting the actual use requirements existing in traditional recommendation methods.

[0006] The technical solution adopted by the present invention to solve the above technical problems is:

[0007] A recommendation method based on a knowledge graph and an attention mechanism, comprising the following steps:

[0008] A. Training a recommendation model:

[0009] A1. Constructing a training sample set: Collecting user historical interaction information for preprocessing, converting it into a user-item interaction matrix, the data of the interaction matrix including user numbers, item numbers and interaction information, and constructing a corresponding matrix according to the items of the interaction matrix and the entities of the knowledge graph;

[0010] A2. Build a recommendation model. Use the training sample set as the input of the recommendation model, and adopt the BRP loss function as the optimization objective to train the recommendation model. The built recommendation model includes an input layer, an encoding layer, a feature fusion layer, and an output layer;

[0011] Input layer: Used to convert the input data into a low-dimensional embedding representation;

[0012] Encoding layer: Based on the correlation between the relationship and the user preference, use the attention mechanism to update the user preference representation; Based on the updated user preference, construct a neighbor node set of each node in the knowledge graph, and use the relationship type information of the neighbor nodes of each node to update the knowledge representation of each entity in the knowledge graph;

[0013] Feature fusion layer: Fuse the updated user preference representation and the knowledge representation to update the user embedding representation;

[0014] Output layer: Combine the updated user embedding representation and the item number embedding representation to generate the interaction probability between the user and the item;

[0015] B. Make recommendations according to the trained item recommendation model:

[0016] Calculate the probability that the user adopts the item to be recommended based on the trained recommendation model, sort according to the size of the probability, and generate the user's recommendation candidate set.

[0017] Further, step A1 specifically includes:

[0018] A11. Obtain the user's historical interaction records and convert them into a user-item interaction matrix;

[0019] A12. Align the items in the interaction matrix with the entities in the knowledge graph to obtain the corresponding matrix;

[0020] A13. Perform initial encoding on the user number, item number, and generate the initial encoding of the user preference;

[0021] A14. Encode the entities and relationships in the knowledge graph using the knowledge embedding method.

[0022] Further, the input layer converts the input data into a low-dimensional embedding representation, specifically including:

[0023] For the user number, item number, user preference, knowledge graph entity, and relationship, obtain the vector corresponding to the index by looking up the table in the randomly initialized embedding table, so as to convert it into a low-dimensional embedding value.

[0024] Further, the encoding layer updates the user preference representation using the attention mechanism based on the correlation between the relationship and the user preference, and the update formula is:

[0025]

[0026] Among them, is the attention score, indicating the relationship r and the user preference p correlation; e p represents the embedding representation of the user preference p e r represents the embedding representation of the relationship r in the knowledge graph, R represents the set of relationship types in the knowledge graph.

[0027] Furthermore, the calculation formula of the attention score is as follows:

[0028]

[0029] Among them, is the trainable weight representing the relationship and the user preference ;

[0030] Furthermore, based on the correlation between the relationship and the user preference, the encoding layer uses the attention mechanism to update the user preference representation, and also includes: constructing a distance formula to measure the association between the user preference and other user preferences p' , and iterating the attention for updating the user preference representation in the encoding layer to minimize the dependence of the user preference on other user preferences p' , and the calculation formula is as follows:

[0031]

[0032] Among them, d Cor(·) is the distance correlation between the user preference and other user preferences p' , and its formula is as follows:

[0033]

[0034] Among them, d Cov(·) is the distance covariance between the user preference and other user preferences p' , d Var(·) is the distance variance of the user preference.

[0035] Further, the encoding layer constructs a set of neighbor nodes for each node in the knowledge graph based on the updated user preferences, and updates the knowledge representation of each entity in the knowledge graph using the relationship type information of the neighbor nodes of each node. The update formula is as follows:

[0036] ;

[0037] Among them, represents the aggregation of node v and relationship r . respectively represent the embedding representations of node v and relationship r . 0 indicates that it is the input embedding representation; represents the embedding representation of node i . 1 indicates that it is the updated embedding representation; is the relevant parameter of the relationship type, f() function represents the Relu activation function, represents the set of neighbor nodes constructed by node i based on the updated user preferences.

[0038] Further, the feature fusion layer fuses the updated user preference representation and the knowledge representation to update the user embedding. The update formula is as follows:

[0039]

[0040] Among them, represents the embedding representation of user u . 1 indicates that it is the updated embedding representation; represents the attention score between user u and preference p , e p represents the embedding representation of user preference , represents the embedding representation of node i . 0 indicates that it is the input embedding representation, represents the product; represents the set of items taken by user u , represents quantity.

[0041] Further, the calculation formula of the attention score is as follows:

[0042]

[0043] Among them, represents useru The embedded representation, where 0 indicates that it is the embedded representation of the input; e p Indicates user preferences The embedded representation, is the matrix transpose.

[0044] Furthermore, the output layer combines the updated user embedded representation and the item number embedded representation to generate the interaction probability between the user and the item. The calculation formula is:

[0045]

[0046] Represents the user u and the item i The interaction probability between them, Represents the user u The final embedded representation, is the matrix transpose, Represents the item i The final embedded representation.

[0047] Furthermore, in step A2, the BRP loss function is used as the optimization objective to train the recommendation model. The loss function is as follows:

[0048]

[0049] Among them, is the observed interaction record between user u and the item and the unobserved interaction record respectively form the training data set, Represents the sigmoid function;

[0050] And the processing steps of the input layer, encoding layer, feature fusion layer and output layer are iterated in a stochastic gradient descent manner until the training cycle ends, and the model with the minimum loss is used as the trained recommendation model.

[0051] The beneficial effects of the present invention are:

[0052] The attention mechanism is used to combine the user's preferences with the relationships in the knowledge graph, providing better expression ability for the preferences, making the final recommendation more interpretable;

[0053] The graph neural network is used for knowledge aggregation, better aggregating the structural information of neighbor nodes, and then using the attention mechanism to combine user preferences to transfer knowledge to the recommendation system, thereby enriching the representation of users and items and achieving better recommendation effects. It is applicable to personalized recommendations of various types of content, resources, and items. Description of the Drawings

[0054] Figure 1 Flowchart of the recommendation method based on knowledge graph and attention mechanism in the embodiment;

[0055] Figure 2 Schematic diagram of the recommendation model structure in the embodiment. Detailed implementation manners

[0056] The present invention aims to propose a recommendation method based on knowledge graph and attention mechanism to solve the problems of data sparsity, poor interpretability, unsatisfactory actual recommendation effect and difficulty in meeting actual usage requirements existing in traditional recommendation methods. In the present invention, first, the items in the recommendation system are aligned with the entities of the knowledge graph to form a training sample set; then, a recommendation model including an input layer, an encoding layer, a feature fusion layer and an output layer is constructed, the training sample set is used as the input of the recommendation model, and the selected loss function is used as the optimization target to train the recommendation model; finally, the probability that the user adopts the item to be recommended is calculated based on the trained recommendation model, and sorting is performed according to the size of the probability to generate a recommendation candidate set for the user.

[0057] Embodiment:

[0058] As Figure 1 shown, the implementation steps of the recommendation method based on knowledge graph and attention mechanism in this embodiment are as follows:

[0059] A. Training the recommendation model:

[0060] A1. Collecting and preprocessing the user's historical interaction information and aligning the items in the recommendation system with the entities of the knowledge graph to form a training sample set, specifically including steps A11 - A14:

[0061] A11. Obtaining and preprocessing the user's historical interaction records:

[0062] That is, converting the user's historical interaction information into a user - item interaction matrix; where the data of the interaction matrix includes user ID, item ID and interaction records;

[0063] A12. Obtaining the corresponding matrix of the alignment between the user items and the knowledge graph entities:

[0064] That is, performing alignment processing on the user items and the entities in the knowledge graph so that the items correspond to the entities in the knowledge graph one by one, thereby obtaining the corresponding matrix, where the data items of the corresponding matrix represent the item ID and the corresponding knowledge graph entity ID, and the data of the knowledge graph entities comes from the publicly available knowledge graph dataset;

[0065] A13. Initialize the user ID and project ID, generate the initial encoding of user preferences, and perform numerical compression on the encoded result. The processed numerical information is a series of consecutive integers starting from 0.

[0066] A14. Encode the entities and relationships in the knowledge graph according to the knowledge embedding method. The common knowledge embedding method can use the TransE knowledge embedding model.

[0067] A2. Design a recommendation model. Use the training sample set as the input of the recommendation model, and adopt the BPR loss function as the optimization objective to train the recommendation model. It specifically includes steps A21 - A22:

[0068] A21. Model construction:

[0069] In this step, we design a recommendation model for commodity recommendation based on the knowledge graph and attention mechanism. This model consists of an input layer, an encoding layer, a feature fusion layer, and an output layer.

[0070] The input layer is responsible for converting various data into low - dimensional embedding representations.

[0071] The encoding layer is responsible for updating the user preference representation using the attention mechanism based on the relevance between the relationship and user preferences. Based on the updated user preferences, construct a set of neighbor nodes of each node in the knowledge graph, and use the relationship type information of the neighbor nodes of each node to update the knowledge representation of each entity in the knowledge graph.

[0072] The feature fusion layer is responsible for fusing the updated user preference representation and knowledge representation to update the user embedding representation.

[0073] The output layer is responsible for generating the interaction probability between the user and the project by combining the updated user embedding representation and the project ID embedding representation.

[0074] The following specifically describes the above four layers of the model as follows:

[0075] (1) Input layer:

[0076] In the input layer, the encoded representations of the user ID, project ID, user preferences, entities, and relationships in the knowledge graph are converted into low - dimensional embedding values by looking up the vectors corresponding to the indices in the randomly initialized embedding table:

[0077] The embedding representation of the user encoding is e u , the embedding representation of the project encoding is e i , the embedding representation of the user preference is e p , and the embedding representation of the relationship in the knowledge graph is e r。

[0078] (2)Coding layer:

[0079] User preferences are a complex combination of relationships but do not include all relationships. Different preferences abstract different user behavior patterns. This can enhance the widely used collaborative filtering effect through finer-grained assumptions - users driven by similar preferences will have similar preferences for items. Therefore, the user-item relationship can be modeled at the granularity of preferences.

[0080] The distribution of knowledge graph relationships can be utilized to assign each preference p k ∈ P. Specifically, an attention mechanism can be used to create preference embeddings. k is the predefined number of preferences, usually optimal at half of the total number of relationships.

[0081] Its specific update method is:

[0082] Among them, is the attention score, indicating the correlation between the relationship r and the user preference ; e p represents the embedding representation of the user preference p e r represents the embedding representation of the relationship r in the knowledge graph, R represents the set of relationship types in the knowledge graph.

[0083] And The calculation formula of is:

[0084]

[0085] Among them, is the trainable weight representing the relationship and the user preference .

[0086] To obtain better learning of user preferences, a distance function can be introduced to guide the representation learning of user preferences, that is, through the distance function, the distance between two vectors is made as far as possible, so that their differences are greater. Specifically, distance-related formulas can be used, which measure the association between the linear and non-linear of any two variables. When and only when these variables are independent, its coefficient is zero. Minimizing the distance correlation of the user's implicit purpose can reduce the dependence of different implicit purposes. Its formula is as follows:

[0087]

[0088] Among them, d Cor(·) is the user preference And other user preferences p' The distance correlation between them is as follows:

[0089]

[0090] Wherein, d Cov(·) is the user preference and other user preferences p' of the distance covariance, d Var(·) is the distance variance of the user preference.

[0091] To achieve the independence between preferences, the mutual information between any two different preference representations is minimized here to quantify their independence. Emphasizing this distance loss can make the differences between different preferences greater and give these preferences obvious boundaries, thus endowing the user preferences with better interpretability.

[0092] Secondly, the knowledge is embedded by means of a graph neural network:

[0093] First, refine the collaborative information from the user-item interaction matrix. The collaborative filtering effect assumes that users with similar behaviors have similar preferences for items to represent the user behavior pattern. Therefore, the personal history (i.e., the items previously adopted by the user) can be regarded as the pre-existing features of the individual user.

[0094] In addition, in the user-item interaction graph, finer-grained information can be captured at the granularity level of user preferences by assuming that users with similar preferences will show similar preferences for items.

[0095] Therefore, based on the updated user preferences, the neighbor node sets of each node in the knowledge graph are constructed, and the knowledge representations of each entity in the knowledge graph are updated by using the relationship type information of the neighbor nodes of each node.

[0096] The embedding update method of the knowledge is: ;

[0097] Wherein, represents the aggregation of node v and relationship r , respectively represent the embedding representations of node v and relationship r , 0 indicates that it is the input embedding representation; represents the embedding representation of node i , 1 indicates that it is the updated embedding representation; is the relevant parameter of the relationship type, f() the function represents the Relu activation function, represents node iA set of neighbor nodes constructed based on updated user preferences.

[0098] (3) Feature fusion layer:

[0099] The role of this layer is to transfer the obtained knowledge embedding to the recommendation system to obtain a denser embedding representation. Specifically, it can integrate preference-aware information from historical items to create a representation of user u. The specific method is as follows:

[0100]

[0101] Among them, represents the embedding representation of user u , and 1 indicates that it is the updated embedding representation; represents the attention score between user u and preference p , e p represents the embedding representation of user preference , represents the embedding representation of node i , and 0 indicates that it is the input embedding representation, represents the product; represents the set of items taken by user u , represents the quantity of.

[0102]

[0103] Among them, represents the embedding representation of user u , and 0 indicates that it is the input embedding representation; e p represents the embedding representation of user preference , is the matrix transpose.

[0104] (4) Output layer:

[0105] Based on the embedding representation of user u output by the above fusion layer and the embedding representation of item i , calculate the interaction probability:

[0106]

[0107] represents the interaction probability between user u and item i , represents the final embedding representation of user u , is the matrix transpose, The final embedded representation of the representative project i is as follows.

[0108] The recommended model structure designed as above is as shown in Figure 2 the figure. The "final representation" here is the final result after multiple rounds of updates of the graph neural network, and the number of layers of the graph neural network needs to be specified according to the actual situation.

[0109] A22. Model training and saving:

[0110] In this step, the BPR loss function is used as the optimization objective function of the model, and iterative training is carried out within a specified period to obtain the optimal trained model for saving.

[0111] Specifically, since the interaction probability between a user and a project is either 1 or 0, this model believes that for a given user, its historical projects should be assigned a higher prediction score than unobserved projects. Therefore, the BPR loss function is selected as the optimization objective function of this model.

[0112] The loss function can be expressed as:

[0113] where is the observed interaction record between user u and the project and the unobserved interaction record which respectively form the training data sets, represents the sigmoid function;

[0114] The training method is stochastic gradient descent, the optimizer uses Adam, the learning rate is set to 0.001, the regularization parameter Dropout is set to 0.2, the training epoch is initialized to 100 times, the number of layers of the graph neural network is 3, and the model with the lowest loss within the set period is saved as the model for subsequent deployment.

[0115] B. Making recommendations based on the trained recommendation model:

[0116] This step is to use the trained recommendation model for actual application recommendations. In specific applications, the historical records of the user u' to be recommended are obtained, and the interaction probability values between the user u' to be recommended and each project are calculated. The projects are sorted in descending order of the probability values, and the top certain number of items (such as 10 items) are selected as the recommendation set and pushed to the user.

[0117] Although the present invention has been described herein with reference to embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited by the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope of the principles and spirit disclosed in this application.

Claims

1. A recommendation method based on knowledge graph and attention mechanism, characterized in that, it includes the following steps: A. Train the recommendation model: A1. Construct the training sample set: Collect the user's historical interaction information for preprocessing, convert it into a user-item interaction matrix, the data of the interaction matrix includes user ID, item ID and interaction information, and construct a corresponding matrix according to the items of the interaction matrix and the entities of the knowledge graph; A2. Construct the recommendation model, use the training sample set as the input of the recommendation model, adopt the BRP loss function as the optimization target, and train the recommendation model; the constructed recommendation model includes an input layer, an encoding layer, a feature fusion layer and an output layer; Input layer: Used to convert the input data into a low-dimensional embedding representation; Encoding layer: Based on the relevance between the relationship and the user preference, use the attention mechanism to update the user preference representation; based on the updated user preference, construct the neighbor node set of each node in the knowledge graph, and use the relationship type information of the neighbor nodes of each node to update the knowledge representation of each entity in the knowledge graph; Feature fusion layer: Fuse the updated user preference representation and knowledge representation to update the user embedding representation; Output layer: Combine the updated user embedding representation and item ID embedding representation to generate the interaction probability between the user and the item; B. Make recommendations according to the trained item recommendation model: Calculate the probability that the user adopts the item to be recommended based on the trained recommendation model, sort according to the size of the probability, and generate the user's recommendation candidate set; The encoding layer constructs the neighbor node set of each node in the knowledge graph based on the updated user preference, and uses the relationship type information of the neighbor nodes of each node to update the knowledge representation of each entity in the knowledge graph. The update formula is: ； Among them, represents the aggregation of nodes v and relationship r ; respectively represent the embedding representations of node v and relationship r ; 0 indicates that it is the input embedding representation; represents the embedding representation of node i ; 1 indicates that it is the updated embedding representation; is the relevant parameter of the relationship type, f() The function represents the Relu activation function, represents the set of neighbor nodes constructed by node i based on the updated user preferences; The feature fusion layer fuses the updated user preference representation and knowledge representation to update the user embedding. The update formula is: Among them, represents the embedding representation of the user u , where 1 indicates that it is the updated embedding representation; represents the user u and the attention score p between the preferences e p represents the embedding representation of the user preference , represents the embedding representation of the node i , where 0 indicates that it is the input embedding representation, represents the product; represents the set of items u adopted by the user represents the quantity.

2. A recommendation method based on knowledge graph and attention mechanism according to claim 1, characterized in that, Step A1 specifically includes: A11. Obtain the user's historical interaction records and convert them into a user-item interaction matrix; A12. Align the items of the interaction matrix with the entities of the knowledge graph to obtain a corresponding matrix; A13. Perform initial encoding on the user ID and item ID, and generate the initial encoding of the user preference; A14. Encode the entities and relationships in the knowledge graph by using the method of knowledge embedding.

3. A recommendation method based on knowledge graph and attention mechanism according to claim 2, characterized in that: The input layer converts the input data into a low-dimensional embedding representation, specifically including: Obtain the vectors corresponding to the indexes by looking up the table in the randomly initialized embedding table for the user ID, item ID, user preference, knowledge graph entity and relationship, so as to convert them into low-dimensional embedding values.

4. A recommendation method based on knowledge graph and attention mechanism according to any one of claims 1, 2 or 3, characterized in that, The encoding layer updates the user preference representation by using the attention mechanism based on the relevance between the relationship and the user preference. The update formula is: Among them, is the attention score, indicating the relationship r with the user preference p correlation; e p represents the user preference embedding representation, e r represents the relationship r embedding representation in the knowledge graph, represents the set of relationship types in the knowledge graph.

5. A recommendation method based on a knowledge graph and an attention mechanism as described in claim 4, characterized in that, The attention score is calculated as follows: Among them, represents the relationship and user preferences as trainable weights.

6. A recommendation method based on a knowledge graph and an attention mechanism as described in claim 4, characterized in that, The encoding layer updates the user preference representation using an attention mechanism based on the correlation between relationships and user preferences, and further includes: constructing a distance formula to measure the user preference and other user preferences p' to update the attention of the user preference representation in the encoding layer iteratively to minimize the dependence of the user preference on other user preferences p' The calculation formula is as follows: Among them, d Cor(·) is the distance correlation between user preferences p' and other user preferences, and its formula is as follows: Among them, d Cov(·) is the user preference and the distance covariance of other user preferences p' is the distance covariance, d Var(·) and d Var(·) is the distance variance of the user preference.

7. A recommendation method based on a knowledge graph and an attention mechanism as described in claim 1, characterized in that, The attention score is calculated as follows: Among them, represents the u embedded representation of the user, and 0 indicates that it is the embedded representation of the input; e p represents the embedded representation of the user preference, which is the matrix transpose.

8. A recommendation method based on a knowledge graph and an attention mechanism as described in any one of claims 1, 2 or 3, characterized in that, The output layer combines the updated user embedding representation and the item number embedding representation to generate the interaction probability between the user and the item, and its calculation formula is: Represents the user u and the project i The interaction probability between Represents the user u The final embedding representation of Is the matrix transpose Represents the project i The final embedding representation of 9. A recommendation method based on a knowledge graph and an attention mechanism as described in claim 1, characterized in that, In step A2, the BRP loss function is used as the optimization objective to train the recommendation model, and the loss function is as follows: Among them, is the interaction record observed between user u and the project and the unobserved interaction record respectively form the training data set, represents the sigmoid function; And the processing steps of the input layer, the encoding layer, the feature fusion layer and the output layer are iterated in a stochastic gradient descent manner until the training period ends, and the model with the minimum loss is used as the trained recommendation model.

Images