A recommendation method and system based on a knowledge graph and high-order collaborative information

Abstract

The application provides a recommendation method and system based on a knowledge graph and high-order collaborative information, comprising: determining interaction information of a user and an item and a knowledge graph; determining an item having interaction with the user according to an interaction matrix of the user and the item, taking the item having interaction as a first-order neighbor, determining other users having interaction with the first-order neighbor as second-order neighbors, determining items having interaction with the second-order neighbors as third-order neighbors, and taking the user's high-order embedding representation by connecting each order neighbor of the user through high-order connectivity; randomly sampling the first-order neighbors of the item in the knowledge graph, determining the relationship between entities, and determining the weight of each entity relationship based on an attention mechanism; aggregating the embedding representation of the item itself and the embedding representation of the first-order neighborhood to obtain the aggregated embedding representation of the item; and recommending the item of interest of the user according to the high-order embedding representation of the user and the aggregated embedding representation of the item. The recommendation method provided by the application has high accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence, and more particularly, to a recommendation method and system based on a knowledge graph and high-order collaborative information. BACKGROUND

[0002] With the advent of the Internet era, information is growing rapidly. For users, available information is like finding a needle in a haystack. In order to alleviate the problem of information overload, improve the utilization rate of information, filter out effective information for the majority of users, and realize accurate recommendation of information, a recommendation model emerges as the times require. The recommendation model can filter and recommend potential interesting items for users, thereby improving the decision-making efficiency of users and enabling operators to achieve accurate product marketing, bringing huge commercial benefits.

[0003] At present, there are numerous recommendation methods based on knowledge graphs, such as the KCAN model, which distills local knowledge from a large amount of knowledge graph to mine more accurate target preferences in a local subgraph, and then recursively aggregates the features of neighbors through a graph convolutional neural network to update the node representation. The MANN model is a multi-context-aware recommendation algorithm based on a knowledge graph, which combines path-based and propagation-based methods to learn item representations. The KGCN model samples part of the neighbor nodes of each entity in the knowledge graph, iteratively aggregates neighbor features through multi-layer KGCN, and then obtains the representation of the corresponding item. Overall, the current use of graph neural networks to capture information in the knowledge graph and further apply it to recommendation is only modeling from the item end, which cannot be examined from the user's perspective. The user-item interaction history is not incorporated into the user representation, resulting in insufficient user embedding representation.

[0004] In recent years, researchers have also been exploring recommendation models based on user-item two-end modeling, such as the DEKGCN model, which aggregates auxiliary information of items using heterogeneous information of the knowledge graph on the item end to obtain item representations; on the user end, it uses auxiliary information such as user age and occupation to construct a user attribute graph for modeling. This makes both the user end and the item end contain rich semantic representations. The DGAR model propagates and expands user's potential interest through the connection between entities in the knowledge graph on the user end; on the item end, it captures high-order structure and semantic information in the knowledge graph, samples entity neighbors, and models high-order neighborhood information through a graph attention mechanism.

[0005] Overall, the current two-end recommendation method has deficiencies in exploring user-item high-order interaction information for user representation, and cannot learn deeper interaction information between users and items, still having the problem of insufficient user embedding representation. SUMMARY

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a recommendation method and system based on knowledge graphs and high-order collaborative information. This aims to solve the problem that current two-end recommendation methods are insufficient in exploring the representation of users by high-order user-item interaction information, are unable to learn deeper user-item interaction information, and still suffer from insufficient user embedding representation.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a recommendation method based on knowledge graphs and high-order collaborative information, comprising the following steps:

[0008] Determine the user-project interaction information and knowledge graph; the projects include entities in the scenarios to be recommended to the user.

[0009] Based on the user-project interaction information, an interaction matrix is ​​determined to reflect whether there is interaction between the user and each project. Based on the user-project interaction matrix, projects that have interacted with the user are identified and designated as the user's first-order neighbors. Other users that have interacted with the first-order neighbors are identified as the user's second-order neighbors. Then, projects that have interacted with the second-order neighbors are identified as the user's third-order neighbors, and so on. By connecting the user's neighbors at each order, a higher-order interaction graph between the user and projects is obtained. Based on the higher-order interaction graph, a layer-by-layer embedding iteration is performed to determine the user's higher-order embedding representation.

[0010] Randomly sample the first-order neighbors of a project to form the first-order neighborhood of the project; determine the relationship between the project and each neighbor in its first-order neighborhood in the knowledge graph, and determine the weight of each neighbor relationship based on the attention mechanism to reflect the importance of each neighbor relationship to the user; combine the representation of each neighbor in the first-order neighborhood and its corresponding relationship weight to determine the embedding representation of the project's first-order neighborhood; aggregate the project's own embedding representation with the embedding representation of its first-order neighborhood to obtain the aggregated embedding representation of the project.

[0011] The user's preference score for items is predicted based on the user's higher-order relational embedding representation and the item's aggregated embedding representation, and items that the user is interested in are recommended based on the preference score.

[0012] In an optional example, the user-project interaction matrix is ​​determined based on the user's interaction information with the project, specifically as follows:

[0013] Let the set of users be represented as U = {u1, u2, ..., u...} m}; m represents the number of users, u i This represents the i-th user; 1 ≦ i ≦ m;

[0014] Let the set of items be represented as V = {v1, v2, ..., v...} n}; n represents the number of items, v jdenotes the j-th item; 1 < j < n;

[0015] Define the interaction matrix Y e R m×n , where y uv is 1 if there is an interaction between user u and item v, and 0 otherwise; y uv is an element in the interaction matrix Y.

[0016] In an optional example, the high-order embedding representation of the user is determined by the following steps:

[0017] Let the contribution of the first-order neighbor v of the user u to the feature of the user u be u,v :

[0018]

[0019] where N(v) denotes the set of first-order neighbors of item v, |(v)| denotes the number of first-order neighbors of item v, N(u) denotes the set of first-order neighbors of user u, |(u)| denotes the number of first-order neighbors of user u, W1 denotes a weight matrix, is the initial vector representation of item v;

[0020] The first-order representation of the user u is :

[0021]

[0022] The high-order representation of the user u is :

[0023]

[0024] where LeakyReLU is an activation function, integrates the information of the first-order neighbor to the l-th order neighbor, and is the representation of the user at the l-th layer of the high-order interaction graph, integrates the information of the first-order neighbor to the (l-1)-th order neighbor, and is the representation of the user at the (l-1)-th layer of the high-order interaction graph;

[0025] The high-order embedding representation of the user u is obtained by adding the embedding representation of the user at each layer :

[0026]

[0027] In an optional example, the aggregated embedding representation of the item is determined by the following steps:

[0028] Let e i denote the entity i, and e jdenotes the entity j, r denotes the relationship between them; the attention mechanism is introduced to determine the importance of the relationship r to the user u as the relationship weight, and the relationship weight of the relationship r to the user u is:

[0029]

[0030] wherein sum denotes the sum operation, u e ,r e are the vector representations of the user u and the relationship r respectively;

[0031] The vector representation of the first-order neighbor of the item v is multiplied by the corresponding normalized relationship weight and summed to obtain the first-order neighborhood representation of the item v as:

[0032]

[0033] wherein π(u,r v,i ) denotes the normalized relationship weight, r v,i denotes the relationship between the item v and the entity i, e i is the embedding representation of the entity i;

[0034] The aggregated embedding representation of the item v is :

[0035]

[0036] wherein σ denotes the activation function ReLU, v0 is the vector representation of the item itself, W2 is a linear transformation matrix, and b denotes a bias term.

[0037] In an optional example, the root point multiplies the high-order embedding representation of the user and the aggregated embedding representation of the item, sums them up, and combines an activation function to predict the preference score of the user to the item:

[0038] In a second aspect, the present application provides a recommendation system based on a knowledge graph and high-order collaborative information, comprising:

[0039] A determination unit is configured to determine the interaction information of the user and the item and the knowledge graph; the item includes an entity in a scene to be recommended to the user;

[0040] The user high-order embedding representation unit is used to determine the user-item interaction matrix based on the user-item interaction information to reflect whether there is interaction between the user and each item; based on the user-item interaction matrix, the items that have interacted with the user are identified as the user's first-order neighbors, and other users that have interacted with the first-order neighbors are identified as the user's second-order neighbors, then the items that have interacted with the second-order neighbors are identified as the user's third-order neighbors, and so on, connecting the user's neighbors of each order to obtain the user-item high-order interaction graph, and performing layer-by-layer embedding iterations based on the high-order interaction graph to determine the user's high-order embedding representation;

[0041] The project aggregation embedding representation unit is used to randomly sample the first-order neighbors of a project to form the first-order neighborhood of the project; determine the relationship between the project and each neighbor in its first-order neighborhood in the knowledge graph, and determine the weight of each neighbor relationship based on the attention mechanism to reflect the importance of each neighbor relationship to the user; combine the representation of each neighbor in the first-order neighborhood and its corresponding relationship weight to determine the embedding representation of the project's first-order neighborhood; aggregate the project's own embedding representation with the embedding representation of its first-order neighborhood to obtain the project's aggregate embedding representation.

[0042] The project recommendation unit is used to predict the user's preference score for projects based on the user's higher-order relational embedding representation and the project's aggregated embedding representation, and recommend projects that the user is interested in based on the preference score.

[0043] In an optional example, the user higher-order embedding representation unit determines the user-item interaction matrix based on the user-item interaction information, specifically as follows:

[0044] Let the set of users be represented as U = {u1, u2, ..., u...} m}; m represents the number of users, u i Let i represent the i-th user; 1 ≤ i ≤ m; represent the set of items as V = {v1, v2, ..., v...} n}; n represents the number of items, v j Let Y represent the j-th item; 1 ≦ j ≦ n; define the interaction matrix Y ∈ R. m×n If user u interacts with project v, then y is defined. uv If y is 1, then 0 is defined otherwise; uv (Elements in the interaction matrix Y).

[0045] In an optional example, the user higher-order embedding representation unit determines the user's higher-order embedding representation through the following steps:

[0046] Let I be the contribution of user u's first-order neighbor v to the formation of user u's features. u,v :

[0047]

[0048] where N(v) denotes the set of first-order neighbors of item v, |N(v)| denotes the number of first-order neighbors of item v, N(u) denotes the set of first-order neighbors of user u, |N(u)| denotes the number of first-order neighbors of user u, W1 denotes the weight matrix, is the initial vector representation of item v;

[0049] The first-order representation of user u is

[0050]

[0051] The high-order representation of user u is

[0052]

[0053] where LeakyReLU is an activation function, is the representation of user u at the l-th layer of the high-order interaction graph, which integrates the information of the first-order neighbors to the (l-1)-th order neighbors, is the representation of user u at the (l-1)-th layer of the high-order interaction graph, which integrates the information of the first-order neighbors to the (l-1)-th order neighbors;

[0054] The high-order embedding representation of user u is obtained by adding the embedding representation of user u at each layer

[0055]

[0056] In an optional example, the item aggregation embedding representation unit determines the aggregation embedding representation of the item by the following steps:

[0057] Let e i denote entity i, e j denote entity j, and r denote the relationship between them; a attention mechanism is introduced to determine the importance of relationship r to user u as a relationship weight, and the relationship weight of relationship r to user u is:

[0058]

[0059] where sum denotes summation operation, u e ,r e are the vector representations of user u and relationship r, respectively;

[0060] The vector representation of the first-order neighbor of item v is multiplied by the corresponding normalized relationship weight and summed to obtain the first-order neighborhood representation of item v as:

[0061] ​​​

[0062] wherein π (u, r v,i ) denotes the normalized relation weight, r v,i denotes the relation connecting item v and entity i, e i is the embedding representation of entity i;

[0063] The aggregated embedding representation of item v is denoted as:

[0064]

[0065] wherein σ denotes the activation function ReLU, v0 is the vector representation of the item itself, W2 is a linear transformation matrix, and b denotes a bias term.

[0066] In an optional example, the item recommendation unit predicts the preference score of the user for the item by point multiplication and summation of the high-order embedding representation of the user and the aggregated embedding representation of the item combined with an activation function:

[0067] Overall, compared with the prior art, the above technical solutions conceived by the present application have the following beneficial effects:

[0068] The present application provides a recommendation method and system based on a knowledge graph and high-order collaborative information, further explores user-item double-end modeling, makes up for the deficiency of single-end modeling, alleviates the information overload problem to a certain extent, improves the utilization rate of information, filters out effective information for the majority of users, and improves the recommendation performance.

[0069] The present application provides a recommendation method and system based on a knowledge graph and high-order collaborative information, fully utilizes user click item information, proposes a new embedding propagation method to embed high-order user-item collaborative interaction information into user representation, and makes the user-end embedding representation more rich and accurate.

[0070] The present application provides a recommendation method and system based on a knowledge graph and high-order collaborative information, and experiments are conducted in a book recommendation scenario to evaluate effectiveness. Compared with other recent methods, the recommendation performance of the present application is superior to other methods. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1 is a recommendation method flowchart based on a knowledge graph and high-order collaborative information provided by the embodiment of the present application.

[0072] Figure 2 is a schematic diagram of a high-order user-item interaction graph provided by the embodiment of the present application.

[0073] Figure 3 ​is a schematic diagram of a general process of forming entity embedding representation by using rich first-order and second-order neighbor auxiliary information in a knowledge graph provided by an embodiment of the present application.

[0074] Figure 4 is a schematic diagram of a recommendation model structure provided by an embodiment of the present application.

[0075] Figure 5 is a schematic diagram of third-order embedding propagation of a user u1 provided by an embodiment of the present application.

[0076] Figure 6 is a schematic diagram of embedding representation of an item v provided by an embodiment of the present application.

[0077] Figure 7 is a schematic diagram of a recommendation system architecture based on a knowledge graph and high-order collaborative information provided by an embodiment of the present application. DETAILED DESCRIPTION

[0078] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0079] The present application proposes a double-end modeling recommendation method based on a knowledge graph and high-order collaborative information. The method models from the user-item double end, makes up for the shortcomings of single-end modeling, and fully utilizes high-order user-item interaction information to enrich the user-end representation, so that the recommendation performance is stronger.

[0080] Figure 1 is a flowchart of a recommendation method based on a knowledge graph and high-order collaborative information provided by an embodiment of the present application, as shown in Figure 1 includes the following steps:

[0081] S101, determine the interaction information of the user and the item and the knowledge graph; the item includes an entity in a scene to be recommended for the user;

[0082] S104, determine the interaction matrix of the user and the item according to the interaction information of the user and the item, to reflect whether there is interaction between the user and each item; according to the interaction matrix of the user and the item, determine the items that have interaction with the user, take the items that have interaction as the first-order neighbors of the user, and determine the other users that have interaction with the first-order neighbors as the second-order neighbors of the user, then determine the items that have interaction with the second-order neighbors as the third-order neighbors of the user, and so on, connect the neighbors of each order of the user to obtain a high-order interaction graph of the user and the item, and perform embedding iteration layer by layer based on the high-order interaction graph to determine the high-order embedding representation of the user;

[0083] S103, randomly sampling the first-order neighbors of the item to form the first-order neighborhood of the item; determining the relationships between the item and each neighbor in the first-order neighborhood of the item in the knowledge graph, and determining the weights of each neighbor relationship based on an attention mechanism to reflect the importance of each neighbor relationship to the user; determining the embedding representation of the first-order neighborhood of the item in combination with the representation of each neighbor in the first-order neighborhood and the corresponding relationship weight of each neighbor; and aggregating the embedding representation of the item itself and the embedding representation of the first-order neighborhood of the item to obtain the aggregated embedding representation of the item;

[0084] S104, predicting the preference score of the user for the item according to the high-order embedding representation of the user and the aggregated embedding representation of the item, and recommending the item of interest of the user based on the preference score.

[0085] In one specific embodiment, the technical solution provided by the present application includes the following four key layers:

[0086] Input layer: taking the user interaction information and the knowledge graph as input.

[0087] Embedding layer: modeling the high-order connectivity in the high-order user-item interaction graph to capture deeper features of the user end to enrich the embedding representation of the user end, and embedding the neighbor features in the knowledge graph by neighbor sampling and combining the attention mechanism to enrich the embedding representation of the item end.

[0088] Aggregation layer: aggregating the embedding representation of the neighbor calculated by the embedding layer and the self-representation to obtain the final user and item representation.

[0089] Prediction layer: combining the embedding representation of the user and the item through a prediction function, and finally outputting the preference degree of the given user for the target item to express the size of the user interest.

[0090] Preferably, the present application simultaneously incorporates high-order collaborative information and a knowledge graph into the recommendation system. The high-order user-item interaction information represents the high-order interaction history of the user, which can deeply mine the preferences of the user. The knowledge graph represents the current trend of recording information in the form of entity relationships, which can provide additional (lateral) information about products and users. These information can make the recommendation more robust. Therefore, the present application proposes a double-end modeling recommendation method based on a knowledge graph and high-order interaction information. Due to the construction of the user-item perspective of the model, it overcomes the shortcomings of single-end modeling and fully utilizes the high-order user-item interaction information to supplement the user end representation, thereby producing more effective recommendation performance.

[0091] Given the interaction matrix Y and the knowledge graph G, our goal is to learn the embedding vectors of the user and the item and and the objective function to predict whether a user u has potential interest in an item v. The objective function f is defined as follows:

[0092]

[0093] where y uv represents the potential interest of a user in an item v, is the embedding vector of a user, is the embedding vector of an item, and Θ represents the model parameters of the function f.

[0094] When modeling the user side, the user interaction history is taken as input at the input layer, forming a user-item interaction matrix Y∈R mXn .

[0095] The user-item interaction information is modeled at the embedding layer. First, the user representation is embedded and propagated layer by layer through a high-order user-item interaction graph. First, the user embedding representation is initialized, and then high-order connection relationships are injected through multiple embedding propagation layers to optimize the embedding. A new embedding propagation method is proposed at the embedding layer. When generating the vector representation of a user u, the contribution of the first-order neighbor v of the user u to the characteristics of the user u is u,v :

[0096]

[0097] In the formula, |N(v)| represents the first-order neighbor number of the item v, that is, the number of users directly interacting with the item v, |N(u)| represents the first-order neighbor number of the user u, that is, the number of items directly interacting with the user u, W1 is a trainable weight matrix, is the initial vector representation of the item v. According to the traditional GCN idea, the more first-order neighbors a node has, the smaller the influence of each neighbor node on it, so represents the contribution of node v to the characteristics of user u, and then is multiplied by a trainable weight matrix to jointly represent the contribution of node v.

[0098] All first-order neighbor features are integrated into the vector representation of the user u to form the first-order representation of the user u At the same time, according to the traditional graph neural network rule, an activation function LeakyReLU is added, and the calculation method is as follows:

[0099]

[0100] According to the propagation rule of first-order information, multiple embedding propagation layers are stacked using a high-order user-item interaction graph to explore higher-order connection information. Through the stacking of l embedding propagation layers, the user can receive l neighbor propagation messages. Through recursive embedding of layer-by-layer neighbor nodes, the representation of the user is finally formed, and the recursive formula is as follows:

[0101]

[0102] where, The information of the first 1~l neighbors is integrated, which is the representation of the user at the l-th layer, The information of the first 1~l-1 neighbors is integrated, which is the representation of the user at the l-1-th layer, which will continue to act on the l-th layer to obtain the embedding representation of the user at the l-th layer In this way, through the embedding iteration of high-order connectivity layer by layer, deeper information is expanded to enrich the embedding representation of the user.

[0103] At the aggregation layer, after propagation through the l-th layer, we obtain the embedding representation of the user u at the l-th layer Since the vector representation of the user obtained at different layers has different degrees of contribution to the formation of the final user vector, it is necessary to aggregate the embedding representation of the user at each layer at the aggregation layer, and in this way, the embedding expression ability of the user is enhanced. The aggregation method is to sum aggregation, and the embedding representation of the user u is obtained

[0104]

[0105] When modeling at the project end, at the input layer, the model uses the attributes of the project as auxiliary information, and uses the numerous project attribute triples {(h, r, t) | h E, r E, t E} as input, that is, h is the head entity, r is the relationship, and t is the tail entity. E and R are entity set and relationship set respectively.

[0106] At the embedding layer, the rich auxiliary information and high-order graph structure of the knowledge graph are used to mine deep semantic features to expand the project representation, and finally the project v representation is obtained through sampling, weight calculation and weighted aggregation of the first-order neighbor.

[0107] When forming the project representation at the embedding layer, first, the relationship weight is calculated, and in the knowledge graph G, we define e i represents entity i, e j represents entity j, and r represents the relationship between them. Since the importance of entities connected by different relationships to the user is different in the knowledge graph, for example, when recommending books, A user pays more attention to the author of the book, and B user pays more attention to the theme of the book. Therefore, in order to distinguish the importance of entities connected by different relationships, the model introduces an attention mechanism in the knowledge graph to determine the importance of relationship r to user u as the relationship weight,. The calculation method is:

[0108]

[0109] where u e and r e are the vector representation of user u and relation r respectively, and sum denotes summation.

[0110] Then the first-order neighbors of item v are sampled, and the data sparsity problem can be alleviated to some extent by aggregating the neighbor information in the knowledge graph. The vector representation of the first-order neighbor of item v is multiplied by its corresponding normalized relation weight and summed to obtain the first-order neighborhood representation of item v:

[0111]

[0112] where π(u, r v,e ) denotes the normalized relation weight, r v,i denotes the relation between item v and neighbor i, and e i is the vector representation of neighbor i.

[0113] At the aggregation layer, after obtaining the vector representation V N(v) of the neighbors of item v, the vector representation v0 of the item itself is aggregated with V N(v) to form the final item representation. The aggregation method is summation aggregation, and the final vector representation of item v that integrates neighbor features is obtained:

[0114]

[0115] where W2 denotes the linear transformation matrix, b is the bias term, and σ denotes the activation function.

[0116] At the prediction layer, after obtaining the embedding representation of the item end and the embedding representation of the user end , the two are dot-multiplied and summed, and then combined with the activation function σ to predict the preference score of the user for the item:

[0117]

[0118] where σ denotes the activation function ReLU, and sum denotes summation.

[0119] The recommendation model mentioned in this invention can be applied to various recommendation scenarios, including but not limited to e-commerce product recommendations, music recommendations, movie recommendations, and book recommendations. This example uses book recommendations. Experiments were conducted on the PyCharm platform using the PyTorch framework to implement this invention. During the recommendation process, the group of people to be recommended is referred to as "users," and the recommended books are referred to as "items." This application provides a recommendation method that incorporates both a "high-level user-item" interaction graph and a knowledge graph into the recommendation model. This method models from both the user and item ends, compensating for the shortcomings of single-end modeling, and fully utilizes high-level user-item interaction information to enrich the user-end representation, thus improving recommendation performance.

[0120] This invention represents the sets of users and projects as U = {u1, u2, ..., u} m} and V = {v1, v2, ..., v n}, where m represents the number of users and n represents the number of items. We define the interaction matrix Y∈R. m×n If there is interaction between user projects, then define y. uv If it is 1, then it is defined as 0; otherwise, it is defined as 0.

[0121] To address the issues of data sparsity and cold start, we introduce a knowledge graph G, leveraging its rich auxiliary information to enrich the embedding representation at the project level. The embedding representation of entity v in the knowledge graph is formed by aggregating its neighbors and its own features; N(v) is defined as the set of first-order neighbor entities of entity v.

[0122] Given an interaction matrix Y and a knowledge graph G, our goal is to learn the embedding vectors of users and items. and And through the prediction function To predict whether user u has a potential interest in item v. The prediction function f is defined as follows:

[0123]

[0124] Where y uv This indicates the user's potential interest in project v. For the user's embedding vector, Θ represents the embedding vector of the project, and Θ represents the model parameters of function f.

[0125] See appendix Figure 2 On the user side of our model, high-level user-project collaborative interaction information is fully mined, and user characteristics are propagated within the high-level user-project interaction network. As the order increases, deeper-level characteristics can be examined. For example, Figure 2The target user in the figure is u1, we can mine the user u2 similar to the user u1 from the path {u1->i1, i1->u2} because they are connected by the common interaction item i1, therefore, the modeling of the user u1 can take into account u2. The longer path {u1->i1, i1->u2, u2->i5} indicates that the user may choose the item i5. Because the user u2 similar to the user u1 has interacted with the user i5, through high-order connection, the information of u2 and even higher-order i5 can be considered when forming the representation of u1. Since the high-order relationship can be modeled to capture high-order collaboration signals, therefore, a more rich and accurate user vector representation can be generated.

[0126] Referring to the accompanying drawings Figure 3 From the perspective of the item, the knowledge graph is used to expand the representation of the item with accurate and diverse high-level structured semantic information. Figure 3 The generation of the embedding representation of the entity "A Million Dollars" is shown. The representation of the first-order neighbor "satire" and "Mark Twain" can be used as the first-order neighbor representation of the entity "A Million Dollars". At the same time, to explore higher-order neighbor information, the representation of the second-order neighbor "The Prince and the Pauper" can also be included in the representation of the entity. The first-order and second-order neighbors of the knowledge graph can form a set of entities of different distances, and the use of this information can effectively expand the representation of the item, to a certain extent, solve the problem of data sparsity, make the embedding representation of the item more rich and accurate, and finally make the performance of the recommendation more accurate.

[0127] Referring to the accompanying drawings Figure 4 The overall model framework diagram of the present application is shown in the figure Figure 4 : the model is composed of 4 key layers, (1) the input layer, taking the user interaction information and the knowledge graph as the input. (2) the embedding layer, modeling the high-order connectivity in the high-order user-item interaction graph, capturing deeper features of the user end to enrich the embedding representation of the user end. In the knowledge graph, the neighbor feature embedding is obtained by neighbor sampling combined with the attention mechanism to enrich the embedding representation of the item end. (3) the aggregation layer, aggregating the embedding representation of the neighbor calculated by the embedding layer and the own representation to obtain the final user and item representation. (3) the prediction layer, combining the embedding representation of the user and the item through the prediction function, and finally outputting the preference degree of the given user to the target item, expressing the size of the user interest.

[0128] Referring to the accompanying drawings Figure 5 , the specific formation of the user embedding representation is described, when modeling u1, the information from the direct interaction item v2 can be considered, with the expansion of the node v2, the information from the node u3 and even the high-order v4 can be jointly included in the representation of the node u1. Therefore, the embedding representations of v4, u3 and v2 and can be incorporated into the embedding representation of u1

[0129] When generating the vector representation of user u, the contribution of the first-order neighbor v of user u to the feature of user u is:

[0130]

[0131] where |N(v)| represents the number of first-order neighbors of item v, that is, the number of users directly interacting with item v, |N(u)| represents the number of first-order neighbors of user u, that is, the number of items directly interacting with user u, W1 is a trainable weight matrix, is the vector representation of item v itself.

[0132] The contribution of the first-order neighbor v of user u to the feature of user u can be represented by I u,v , which integrates all first-order neighbor features into the vector representation of user u to form the first-order representation of user u, and simultaneously adds an activation function LeakyReLU in accordance with the conventional graph neural network rule, and the calculation method is:

[0133]

[0134] According to the propagation rule of the first-order information, a plurality of embedding propagation layers are stacked by using high-order user-item interaction graphs to explore higher-order connection information. Through stacking of l embedding propagation layers, a user can receive messages propagated by l neighbors, and through recursive embedding of neighbor nodes layer by layer, a representation of the user is finally formed, and the recursive formula is:

[0135]

[0136] where integrates the information of the previous 1-l layers of neighbors, is the representation of the user at the lth layer, integrates the information of the previous 1-l-1 layers of neighbors, is the representation of the user at the l-1th layer, which will continue to act on the lth layer to obtain the embedding representation of the user at the lth layer Through the embedding iteration layer by layer through high-order connectivity, deeper information is extended to enrich the embedding representation of the user.

[0137] After l layers of propagation, the embedding representation of user u1-l layers is obtained Since the user vector representations obtained at different layers contribute to the final user vector formation to varying degrees, the aggregation layer needs to aggregate the user's embedding representations from each layer to enhance the user's embedding representation capabilities. There are three aggregation methods: summation aggregation, concatenation aggregation, and neighbor aggregation. Taking summation aggregation as an example, it adds the user's embedding vector obtained at each layer to the user's own embedding vector to obtain the final user embedding representation.

[0138]

[0139] See appendix Figure 6 This paper provides an illustrative example of the receptive domain of a given entity. To enrich the embedded representation of item v, a neighbor entity set is formed by sampling {e1, e4} from the first-order neighbors. The relation weights of the entities in the entity set are calculated, and the neighbor representations are combined with the entity's own representation to obtain the embedded representation of item v.

[0140] To differentiate the varying importance of entities connected by different relationships, this model introduces an attention mechanism into the knowledge graph. The importance of relation r to user u is determined by relation weight, which is calculated as follows:

[0141]

[0142] Where u e ,r e These are the vector representations of user u and relation r, respectively, and sum represents the summation.

[0143] The first-order neighborhood representation of item v is obtained by multiplying the vector representations of the first-order neighbors connected by relations with their corresponding normalized relation weights and summing them:

[0144]

[0145] In the formula, π(u,r) v,e ) represents the normalized relation weight, r v,i This represents the relationship between item v and its neighbor i, e i Let i be the vector representation of neighbor i.

[0146] The vector representations of the neighbors of item v are obtained at the embedding layer. N(v) Then, the project's own vector representation v0 and v N(v) The aggregation process forms the final item representation. The aggregation method uses addition aggregation, resulting in a vector representation of item v incorporating neighbor features:

[0147]

[0148] Wherein, W2 is a nonlinear transformation matrix, b is a bias term, and sigma is a nonlinear activation function.

[0149] Obtaining the embedding representation of the item end and the embedding representation of the user end Then, the two are point-multiplied and summed, and an activation function is combined to predict the user's preference score for the item. If the result is > 0.5, it is set to 1, and the item is recommended to the user. If the result is < 0.5, it is set to 0, and the item is not recommended to the user. The specific formula is as follows:

[0150]

[0151] In the formula, sigma represents the activation function ReLU, and sum represents the summation operation.

[0152] For the calculation of the loss function, the model uses a binary cross-entropy loss function to perform binary cross-entropy on the data in a sample set and sum it up. The loss function of the model is:

[0153]

[0154] Where l is the cross-entropy loss, P is the sample set, y uv is the true value, is the predicted value, and Theta is a set of model parameters, is a regularization term, and lambda is the regularization term coefficient.

[0155] Embodiment:

[0156] The experiment uses a book rating data set containing 69874 ratings from 17860 users on 14967 books. In the recommendation process, the population to be recommended is referred to as "user", and the recommended book is referred to as "item". The function of the recommendation model is implemented based on the pytorch framework. By modeling from both the user end and the item end, the knowledge graph and high-order collaborative information are incorporated into the recommendation model, and the AUC and ACC indicators are calculated. First, the user interaction information and the knowledge graph are input into the input layer. Then, in the high-order user-item interaction graph, high-order connectivity expression modeling is performed to capture deeper features of the user end to enrich the user end embedding representation. In the knowledge graph, neighbor feature embedding is obtained through neighbor sampling combined with an attention mechanism to enrich the item end embedding representation. Then, in the aggregation layer, the neighbor embedding representation calculated by the embedding layer is aggregated with the self-representation to obtain the final user and item representation. Finally, the user and item embedding representations are combined through a prediction function to output the preference degree of a given user for a target item, expressing the user's interest size. The results are analyzed from the aspects of the recommendation indicators ACC and AUC.

[0157] (1)ACC result analysis: the data used in the comparative experiment is the same as the data used in this experiment, and the prediction function used is consistent. The ACC experimental results of the present application and the MANN model are 0.681 and 0.712 respectively, and the present application is improved by 3.1% compared with the comparative experiment. As can be seen, from the recommendation accuracy, the recommendation method of the present application is better.

[0158] (2) AUC result analysis: the data used in the comparative experiment is the same as the data used in this experiment, and the prediction function used is consistent. The AUC experimental results of the present application and the MANN model are 0.744 and 0.777 respectively, and the present application is improved by 3.3% compared with the comparative experiment. As can be seen, from the recommendation performance, the recommendation method of the present application is better.

[0159] Figure 7 is the recommendation system architecture based on knowledge graph and high-order collaborative information provided by the embodiment of the present application, as shown in Figure 7 , comprising:

[0160] A determination unit 710 is configured to determine the interaction information of the user and the project and the knowledge graph; the project includes an entity in a scene to be recommended to the user;

[0161] A user high-order embedding representation unit 720 is configured to determine a user-project interaction matrix according to the interaction information of the user and the project, to reflect whether there is interaction between the user and each project; determine the projects that have interaction with the user according to the user-project interaction matrix, take the projects that have interaction as the first-order neighbors of the user, and determine the other users that have interaction with the first-order neighbors as the second-order neighbors of the user, then determine the projects that have interaction with the second-order neighbors as the third-order neighbors of the user, and so on, connect the neighbors of each order of the user to obtain a high-order interaction graph of the user and the project, and perform layer-by-layer embedding iteration based on the high-order interaction graph to determine the high-order embedding representation of the user;

[0162] A project aggregation embedding representation unit 730 is configured to randomly sample the first-order neighbors of the project to form a first-order neighborhood of the project, determine the relationship between the project and the neighborhood in the knowledge graph, and determine the weight of each neighbor relationship based on an attention mechanism to reflect the importance of each neighbor relationship to the user, combine the representation of the neighborhood itself with the corresponding relationship weight to determine the embedding representation of the first-order neighborhood of the project; aggregate the embedding representation of the project itself and the embedding representation of the first-order neighborhood of the project to obtain the aggregated embedding representation of the project;

[0163] A project recommendation unit 740 is configured to predict the preference score of the user to the project according to the high-order embedding representation of the user and the aggregated embedding representation of the project, and recommend the project interested by the user based on the preference score.

[0164] It can be understood that,Figure 7 The detailed function implementation of each unit can refer to the description in the foregoing method embodiments, which will not be described herein.

[0165] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A recommendation method based on a knowledge graph and high-order collaborative information, characterized in that, The method comprises the following steps: determine the user's interaction information with the project and the knowledge graph; the project includes entities in the scene to be recommended to the user; determine the user's interaction matrix with the project according to the user's interaction information with the project, to reflect whether there is interaction between the user and each project; determine the projects that have interaction with the user according to the user's interaction matrix with the project, take the projects that have interaction as the first-order neighbors of the user, and determine the other users that have interaction with the first-order neighbors as the second-order neighbors of the user, then determine the projects that have interaction with the second-order neighbors as the third-order neighbors of the user, and so on, connect the neighbors of each order of the user to obtain a high-order interaction graph of the user and the project, and perform embedding iteration layer by layer based on the high-order interaction graph to determine the high-order embedding representation of the user; randomly sample the first-order neighbors of the project to form the first-order neighborhood of the project; determine the relationship between the project and each neighbor in the first-order neighborhood of the project in the knowledge graph, and determine the weight of each neighbor relationship based on an attention mechanism to reflect the importance of each neighbor relationship to the user; determine the embedding representation of the first-order neighborhood of the project by combining the representation of each neighbor in the first-order neighborhood and the corresponding relationship weight; aggregate the embedding representation of the first-order neighborhood of the project and the embedding representation of the project itself to obtain the aggregated embedding representation of the project; predict the preference score of the user for the project according to the high-order embedding representation of the user and the aggregated embedding representation of the project, and recommend the project interested by the user based on the preference score; the high-order embedding representation of the user is determined by the following steps: Set of users First order neighbors To user Contribution to feature formation is : = wherein, representing items a set of first order neighbors, representing items a number of first order neighbors, representing users a set of first order neighbors, representing users a number of first order neighbors, representing a weight matrix, items initial vector representation; User first order representation is: User high-order representation is: wherein is an activation function, integrates information of the 1st order neighbors up to the -1st order neighbors, is a representation of the user at the -1st layer of the high-order interaction graph, integrates information of the 1st order neighbors up to the -1st order neighbors, is a representation of the user at the -1st layer of the high-order interaction graph. The user's high-order embedding representation is obtained by adding the user's embedding representation at each layer u is:​ = + + +…+ 。 2. The method of claim 1, wherein, determine the user's interaction matrix with the project according to the user's interaction information with the project, specifically: A set of users is represented as U = {u , ,..., }; m denotes the number of users, denotes the i i th user; 1 ≦ i i ≦ m ; Let the set of items be denoted by V = {i1, i2,..., in}, where n is the number of items, and where i1, i2,..., in are the items, respectively. , ,..., }; n denotes the number of items, denotes the j th item; 1 ≦ i ≦ n; j ≦ n ; Define interaction matrix Y , define u as 1 if there is an interaction between user and item , otherwise define as 0; is an element in interaction matrix Y.

3. The method of claim 1, wherein, the aggregated embedding representation of the project is determined by the following steps: Set representing entities , representing entities , representing the relationship between them; introduce attention mechanism to determine the relationship the importance of the user as a relationship weight, the relationship to the user relationship weight: ； wherein, sum denotes a sum operation, are vector representations of users and relationships respectively; The item The vector representation of the sampled first order neighbors is multiplied by their corresponding normalized relational weights and summed to obtain the item The first order neighborhood representation of the item is given by: wherein denotes the normalized relation weight, denotes the item v and entity i connected relation, is the embedding representation for the entity i . Item v of the polymeric insert is: = ( *( + )+b) wherein, denotes the activation function ReLU, is a vector representation of the item itself, is a linear transformation matrix and b denotes a bias term.

4. The method of claim 3, wherein, dot product and summing the user's high-order embedding representation and the aggregated embedding representation of the item in combination with an activation function to predict a preference score of the user for the item: = .

5. A recommendation system based on knowledge graph and high-order collaborative information, characterized in that, comprise: a determination unit configured to determine the user's interaction information with the project and the knowledge graph; the project includes entities in the scene to be recommended to the user; a user high-order embedding representation unit configured to determine the user's interaction matrix with the project according to the user's interaction information with the project, to reflect whether there is interaction between the user and each project; determine the projects that have interaction with the user according to the user's interaction matrix with the project, take the projects that have interaction as the first-order neighbors of the user, and determine the other users that have interaction with the first-order neighbors as the second-order neighbors of the user, then determine the projects that have interaction with the second-order neighbors as the third-order neighbors of the user, and so on, connect the neighbors of each order of the user to obtain a high-order interaction graph of the user and the project, and perform embedding iteration layer by layer based on the high-order interaction graph to determine the high-order embedding representation of the user; a project aggregated embedding representation unit configured to randomly sample the first-order neighbors of the project to form the first-order neighborhood of the project; determine the relationship between the project and each neighbor in the first-order neighborhood of the project in the knowledge graph, and determine the weight of each neighbor relationship based on an attention mechanism to reflect the importance of each neighbor relationship to the user; determine the embedding representation of the first-order neighborhood of the project by combining the representation of each neighbor in the first-order neighborhood and the corresponding relationship weight; aggregate the embedding representation of the first-order neighborhood of the project and the embedding representation of the project itself to obtain the aggregated embedding representation of the project; The item recommendation unit is configured to predict a preference score of a user for an item according to the high-order embedding representation of the user and the aggregated embedding representation of the item, and recommend the item of interest of the user based on the preference score. The user high-order embedding representation unit determines the high-order embedding representation of the user by the following steps: Set of users First order neighbors To user Contribution to feature formation is : = wherein, representing items a set of first order neighbors, representing items a number of first order neighbors, representing users a set of first order neighbors, representing users a number of first order neighbors, representing a weight matrix, are items initial vector representations; User first order representation is: User high-order representation is: wherein is an activation function, integrates information of the 1st order neighbors up to the -1st order neighbors, is a representation of the user at the -1st level of the high-order interaction graph, integrates information of the 1st order neighbors up to the -1st order neighbors, is a representation of the user at the -1st level of the high-order interaction graph; The user's high-order embedding representation is obtained by adding the user's embedding representation at each layer u is:​ = + + +…+ 。 6. The system of claim 5, wherein, The user high-order embedding representation unit determines the interaction matrix of the user and the item according to the interaction information of the user and the item, specifically: The set of users is represented as U={ , ,..., }; m Indicates the number of users. Indicates the first i One user; 1≦ i ≦ m The set of projects is represented as V={ , ,..., }; n Indicates the number of items. Indicates the first j One project; 1≦ j ≦ n Define the interaction matrix Y If the user u With the project If there is interaction between them, then define It is defined as 1 if it is not 1, otherwise it is defined as 0; These are elements in the interaction matrix Y.

7. The system of claim 5, wherein, The item aggregated embedding representation unit determines the aggregated embedding representation of the item by the following steps: Set Representing entities , Representing entities , Representing the relationship between the two; introduce attention mechanism to determine the relationship The importance of the user as a relationship weight, the relationship The relationship weight for the user is: ； wherein, sum denotes a sum operation, are vector representations of users and relationships respectively; The item The vector representation of the sampled first order neighbors is multiplied by their corresponding normalized relational weights and summed to obtain the item The first order neighborhood representation of the item is given by: wherein denotes the normalized relation weight, denotes the item v and entity i connected relation, is the embedding representation for the entity i . Item v of the polymeric insert representation is: = ( *( + )+b) where, denotes the activation function ReLU, is the vector representation of the item itself, is a linear transformation matrix and b denotes the bias term.

8. The system of claim 7, wherein, The item recommendation unit point-multiplies and sums the high-order embedding representation of the user and the aggregated embedding representation of the item and combines an activation function to predict a preference score of the user for the item: = .

Images