Deep information perception graph neural network recommendation model method

Abstract

The invention discloses a deep information perception graph neural network recommendation model method comprising the following steps: S10, generating a cooperation information graph based on user article interaction data and an article knowledge graph; s20, obtaining a high-order cooperation information graph and a high-order interaction graph based on a high-order neighborhood graph generation module; s30, learning node representations in the high-order cooperation information graph and the high-order interaction graph through a deep information aggregation module; and S40, carrying out model training, calculating preference scores of the users on each article based on a training result, and generating a recommendation list of the users according to the scores. According to the method, the article knowledge graph and the user article interaction graph are combined to enrich node connection; in addition, a high-order neighborhood graph generation module is provided to support nodes to establish connection with high-order neighborhoods in advance.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural networks and relates to a graph neural network recommendation model method for deep information perception. Background Art

[0002] The graph neural network recommendation system can effectively learn the complex relationships of multi-source data such as users, items, attributes, and interaction behaviors by constructing a unified graph structure, making up for the deficiency of traditional recommendation systems in utilizing multi-dimensional information. It captures high-order neighborhood features based on the message passing mechanism and integrates them into the target node embedding, explicitly using the collaborative filtering effect, and significantly improving the quality of user and item representations and the recommendation effect.

[0003] However, due to the existence of the over-smoothing problem, the increase in the number of graph convolutional layers will cause the embedding representations of different nodes to become similar, making it difficult for the model to distinguish each node. This limits the depth of the graph neural network model, ignores the high-order neighborhood information of the target node, resulting in the model being unable to fully learn the information in the graph structure, and making it difficult to accurately recommend potential preferred items in their high-order neighborhoods to the target users during recommendation, thus affecting the accuracy of the recommendation. Summary of the Invention

[0004] Aiming at the problem that it is difficult for graph neural networks to learn high-order neighborhood information, the present invention proposes a deep information aware-knowledge graph neural network recommendation model (Deep Information Aware-Knowledge Graph Neural Network, DIA-KGNN). DIA-KGNN combines the item knowledge graph with the user-item interaction graph to enrich node connections. In addition, a high-order neighborhood graph generation module is proposed to support nodes to establish connections with high-order neighborhoods in advance. In the deep information aggregation module, the information aggregation based on the triple attention mechanism enables the model to more accurately understand the semantic information in the graph, improve the quality of node representations, and introduce an initial residual connection to alleviate the over-smoothing problem.

[0005] The present invention provides a graph neural network recommendation model method for deep information perception, including the following steps:

[0006] S10, generating a collaborative information graph based on user-item interaction data and the item knowledge graph;

[0007] S20, obtaining a high-order collaborative information graph and a high-order interaction graph based on the high-order neighborhood graph generation module;

[0008] S30, learning the node representations in the high-order collaborative information graph and the high-order interaction graph through the deep information aggregation module;

[0009] S40, Train the model and calculate the preference scores of users for each item based on the training results, and generate a recommendation list for users based on the scores.

[0010] Preferably, the S10 includes the following steps:

[0011] S11, Represent the user interaction data in the form of a graph structure;

[0012] S12, Represent the item knowledge graph in the form of a graph structure;

[0013] S13, Integrate the user-item interaction graph and the item knowledge graph to obtain a collaborative information graph.

[0014] Preferably, the S11 is specifically as follows. For the given user interaction data, use the user-item interaction graph G UI to represent the interaction data of users and items; the definition of the user-item graph is G UI = {(u, i, ε ui ) | u ∈ U, i ∈ I, ε ui ∈ Ε}, where u is the user node, i is the item node, and if there is an interaction relationship between user u and item i, there is an edge ε ui between node u and node i;

[0015] The S12 is specifically as follows. The item knowledge graph G IK is defined as G IK = (E k , R k , T k ), where E k is the set of all entities in the knowledge graph. In the item knowledge graph, the entities are composed of item entities and item auxiliary information entities, that is, E k = I ∪ S Attri , S Attri is the set of item auxiliary information entities; R k is the set of all relationships in the knowledge graph R k = {r1, r2, r3...... r |R|}, T k is the set of triples in the knowledge graph T k = {(h, r, t) | h, t ∈ E k , r ∈ R k}, and the triples in the set are represented as (h, r, t), where h and t represent the head entity and the tail entity respectively, and r is the relationship between these two entities;

[0016] The S13 is specifically as follows. Merge the user-item interaction graph G UI and the item knowledge graph G IK to obtain the collaborative information graph G CITo enrich the connectivity of nodes in the graph; in the interaction graph G UI Potential preferred items that have no connection with the user originally in CI G form indirect connections with potential preferred items through semantic associations of the knowledge graph; the obtained collaborative information graph G CI The mathematical representation is as follows:

[0017] G CI =(E ci , R ci , T ci ) Equation (1)

[0018] Among them, E ci is the entity set, R ci is the relationship set, and T ci is the triple set.

[0019] Preferably, the S20 includes the following steps:

[0020] S21, calculate the triple attention score in the collaborative information graph;

[0021] S22, obtain the first-order neighborhood sampling result of the high-order collaborative information graph based on the triple attention score;

[0022] S23, obtain the high-order neighborhood sampling result of the high-order neighborhood information graph in a convolutional manner;

[0023] S24, calculate and obtain the high-order adjacency matrix of the user-item interaction graph;

[0024] S25, filter out the noise connections in the high-order adjacency matrix based on the user-item similarity;

[0025] S26, use a 1x1 convolutional layer to learn the sampling results of each order neighborhood of the collaborative information graph and the interaction graph to generate a high-order collaborative information graph and a high-order interaction graph.

[0026] Preferably, the S21 is specifically as follows. For the input triple (h, r, t), the calculation formula of its triple attention score is as follows:

[0027]

[0028] Among them, sim(·,·) is the calculation formula of cosine similarity. The input head entity embedding e h and the tail entity embedding e t First, map to the semantic space of the relationship through two trainable matrices W h and W t respectively, and then combine with the relationship embedding e rPerform a dot product to capture the connection between entities and relationships; finally, calculate the cosine similarity between the two to measure the degree of semantic alignment; α(h,r,t) is the triple attention score of the triple (h,r,t). After softmax normalization of the cosine similarity sim(e h W h ·e r ,e t W t ·e r ), the triple attention score α(h,r,t) is obtained;

[0029] Specifically, in S22, first, based on the triple attention score, obtain the top K CI first-order neighborhood triples with the highest scores for the user in graph G c . The formula is as follows:

[0030]

[0031] Among them, is the set of triples in the first-order neighborhood of user u in graph G IC . K c is a hyperparameter for the number of selected elements. The function TopK selects the top K triples with the highest triple attention scores α(u,r,t) in the set to form the first-order neighborhood sampling graph c

[0032] Specifically, in S23, generate a high-order neighborhood sampling graph in a convolutional manner. The specific method is as follows:

[0033]

[0034] Among them, represents the (i + 1)-th order neighborhood sampling graph of the user. The (i + 1)-th order sampling graph shown in formula (5) performs second-order neighborhood sampling of the user on the i-th order neighborhood sampling graph to obtain the candidate triple set S i , and select the top K i triples with the highest triple scores in the set S c to form the (i + 1)-th order neighborhood sampling graph Since the graph contains the sampling results of the user's i-th order neighborhood, performing second-order neighborhood sampling on the graph can capture the user's (i + 1)-th order neighborhood connection;

[0035] Specifically, in S24, given the user-item interaction graph G UI, which can be represented as an adjacency matrix A. The matrix uses user IDs as rows and item IDs as columns. The adjacency matrix A ∈ {0, 1} |U|×|I| , if there is an interaction between user u and item i, then the matrix element A ui = 1; otherwise, it is equal to 0. By calculating the k-th power of the adjacency matrix A k to obtain the k-hop neighborhood of the user; the high-order adjacency matrix is used to describe the topological relationship of multi-hop connections between nodes. The high-order adjacency matrix A k is the k-th power of the adjacency matrix A, and its calculation formula (7), where the element represents the number of paths from node u to node i after k steps,

[0036] A k = A × A k-1 Equation (7)

[0037] It can be seen from the definition of the high-order adjacency matrix that when the matrix element indicates that there is more than one path between node u and node i. Therefore, node i is the k-hop neighborhood of node u. Unify the elements greater than 1 in the high-order neighborhood matrix to 1 to determine the existence of a connection between two nodes. The specific formula is:

[0038]

[0039] Specifically, S25 is to calculate the cosine similarity of user items, and then sample neighborhood nodes according to the noise filtering threshold μ. The calculation formula is:

[0040]

[0041] Among them, is the adjacency matrix of the interactive neighborhood sampling graph of In the graph, connections with similarities less than μ are regarded as noise connections and the connections between them are disconnected to avoid interfering with the learning of node representations during the subsequent information aggregation process. Among them, the odd-hop neighborhood of the user is its preferred item. Therefore, only the odd-hop sampling graph k % 2 = 1;

[0042] Specifically, S26 is to use a 1×1 convolutional layer to aggregate the sampling results of graph G CI and graph G UI respectively to obtain the high-order collaborative information graph G HC and the high-order interaction graph G HI , allowing the model to learn the importance of each-hop neighborhood by itself. The specific aggregation method is as follows:

[0043]

[0044] Among them, 1×1Conv represents the 1x1 convolution process, ωk Denote the training weights corresponding to the k-th layer. The softmax function is used for normalization to stabilize the values in the high-order graph, ensuring the stable operation of the information aggregation module; For the graph the set of triples in corresponding to the sampled graph the adjacency matrix; the obtained high-order graph G HC and G HI contain the high-order neighborhood connections of the user and include the connection weights of each connection and

[0045] Preferably, the S30 includes the following steps:

[0046] S31, introducing a triple attention mechanism for information aggregation of the high-order collaborative information graph;

[0047] S32, introducing an initial residual connection during the learning process of the high-order interaction graph node representation;

[0048] S33, concatenating the collaborative information embedding and the high-order interaction embedding to obtain the final node representation.

[0049] Preferably, the S31 is specifically as follows. The information aggregation calculation formula is:

[0050]

[0051] where N(h) represents the first-order neighborhood of the head entity h in the graph G HC α(h,r,t) is the same as the triple attention score in Equation (3), is the connection weight in the graph G HC ; is the node embedding of the (l + 1)-th layer; each model layer will generate its embedding representation for the nodes in the graph. As the number of layers increases, the information perception range of the model becomes larger. The collaborative information embedding e ci of the nodes is obtained by combining the embeddings of each layer through layer combination, and its calculation formula is:

[0052]

[0053] where L is the number of model layers, and λ l is the aggregation weight of the embeddings of each layer;

[0054] The S32 is specifically as follows. In the graph neural network, the initial residual connection can effectively alleviate the over-smoothing phenomenon. The calculation formula for the node embedding is:

[0055]

[0056] Equations (17) and (18) are the calculation processes of the embedding of the user at the l+1 layer and the embedding of the item respectively, where represents the connection weight in the high-order interaction graph G HI . η is a hyperparameter used to control the strength of the initial residual. After L layers of information aggregation, the node embeddings of each layer are obtained. Subsequently, the embeddings of each layer are fused through layer combination to obtain the high-order interaction embedding e hi :

[0057]

[0058] Equations (19) and (20) are the user-item interaction embeddings of user u and item i respectively. λ l is a hyperparameter used to control the aggregation weight of each layer;

[0059] Specifically, S33 is to obtain the final node representations of the user and the item by concatenating the two:

[0060]

[0061] e u and e i are the node representations of user u and item i respectively, and || is the concatenation operation.

[0062] Preferably, S40 includes the following steps:

[0063] S41, calculating the preference score of the user for the item;

[0064] S42, selecting a loss function and training the model.

[0065] Preferably, specifically in S41, for the obtained user representation e u and item representation e i , their inner product is used as the predicted score:

[0066]

[0067] is the predicted score of user u for item i.

[0068] Preferably, specifically in S42 for training the model, first, all the parameters in the model are initialized. For the input user-item interaction graph G UI and item knowledge graph G IK , the two are fused to obtain a collaborative information graph G CI ; in the high-order neighborhood graph construction module, calculate the triple attention score for neighborhood sampling and construct a high-order collaborative information graph G HC, construct a high-order interaction graph \(G\) by filtering out noisy connections through cosine similarity HI ; for \(G\) HC Obtain the triple attention scoring weights and perform neighborhood aggregation to obtain the embedding \(e\) hc , and at the same time for \(G\) HI Introduce an initial residual connection in the node embedding learning process to calculate the high-order interaction embedding \(e\) hi , concatenate the two embeddings to obtain the final node embedding \(e\); finally calculate the predicted scores of each item, use the BPR loss to adjust the model parameters, and when the loss function converges, obtain the model parameters.

[0069] The beneficial effects of the present invention at least include:

[0070] (1) High-order neighborhood recommendation ability. The present invention performs high-order neighborhood sampling through the high-order neighborhood sampling method proposed in the high-order neighborhood graph generation module, and further uses 1x1 convolution to aggregate the sampling results to construct a high-order collaborative information graph and a high-order interaction graph. Direct connections between users and preferred entities, preferred items, and similar users are established in the above graphs, supporting the model to capture complex user behavior patterns. In addition, the construction of the high-order collaborative information graph and the high-order interaction graph advances the high-order neighborhood connections that originally required multiple layers of convolution to obtain to the high-order neighborhood graph. This enables the model to stack the same number of convolutional layers during graph convolution, and DIA-KGNN can learn higher-order neighborhood information, thereby improving the high-order neighborhood recommendation ability of the model.

[0071] (2) Alleviate the data sparsity problem. DIA-KGNN introduces an item knowledge graph based on user-item interaction data through the high-order neighborhood graph generation module, enabling potential preferred items that have no connection with the user to form indirect connections with potential preferred items through the semantic association of the knowledge graph. Further, by constructing a high-order collaborative information graph, connections between users and their preferred entities and other users are established, enriching the semantic relationships, supporting the model to capture complex user behavior patterns, and supplementing the connections between users and their high-order neighborhood nodes and the missing information in user interaction behavior data.

[0072] (3) Improve the generalization ability of the model. The 1x1 convolutional layer can dynamically adjust the weights of each order of neighborhood, assign more reasonable weights to each order of neighborhood according to the importance of each order of neighborhood in different datasets, so that even if the preferred items of users in some datasets are concentrated in the low-order neighborhoods of users, the model can still have good performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is the step flow chart of the deep information-aware graph neural network recommendation model method of the present invention;

[0074] Figure 2Schematic diagram of the model structure of the graph neural network recommendation model for deep information perception of the present invention;

[0075] Figure 3 Process diagram of high-order neighborhood graph generation module in the graph neural network recommendation model for deep information perception of the present invention in specific embodiments;

[0076] Figure 4 Schematic diagram of the relationship between the user-item interaction adjacency matrix and each order of neighborhoods in the graph in the graph neural network recommendation model for deep information perception of the present invention in specific embodiments;

[0077] Figure 5 Schematic diagram of the information aggregation process of the high-order collaborative information graph in the deep information aggregation module of the graph neural network recommendation model for deep information perception of the present invention in specific embodiments;

[0078] Figure 6 Graph showing the comparison results of the high-order recommendation performance between the present invention and other recommendation models in the Last.fm dataset;

[0079] Figure 7 Graph showing the comparison results of the high-order recommendation performance between the present invention and other recommendation models in the Book-Crossing dataset. Detailed implementation manners

[0080] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

[0081] On the contrary, the present invention covers any alternatives, modifications, equivalent methods and solutions made within the spirit and scope of the present invention defined by the claims. Further, in order to enable the public to better understand the present invention, some specific details are described in detail in the following detailed description of the present invention. Those skilled in the art can fully understand the present invention without the description of these detail parts.

[0082] See Figure 1 And Figure 2 , the modules and implementation steps included in the graph neural network recommendation model for deep information perception are as follows:

[0083] S10. Generate a collaborative information graph based on user-item interaction data and an item knowledge graph;

[0084] S11. Represent the user interaction data in the form of a graph structure;

[0085] For the given user interaction data, use the user-item interaction graph G UIRepresents the interaction data of user items. The definition of the user-item graph is as follows: G UI ={(u, i, ε ui )|u ∈ U, i ∈ I, ε ui ∈ Ε}, where u is the user node, i is the item node, and if there is an interaction relationship between user u and item i, then there is an edge ε ui .

[0086] S12, represent the item knowledge graph in the form of a graph structure;

[0087] The item knowledge graph G IK is defined as G IK =(E k , R k , T k ), where E k is the set of all entities in the knowledge graph. In the item knowledge graph, entities are composed of item entities and item auxiliary information entities, that is, E k =I ∪ S Attri , S Attri is the set of item auxiliary information entity sets. R k is the set of all relationships in the knowledge graph R k ={r1, r2, r3......r |R|}, T k is the set of triples in the knowledge graph T k ={(h, r, t)|h, t ∈ E k , r ∈ R k}, and the triples in the set are represented as (h, r, t), where h and t represent the head entity and the tail entity respectively, and r is the relationship between these two entities.

[0088] S13, fuse the user-item interaction graph and the item knowledge graph to obtain the collaborative information graph;

[0089] The present invention combines the user-item interaction graph G UI with the item knowledge graph G IK to obtain the collaborative information graph G CI to enrich the connectivity of the nodes in the graph. Potential preference items that are not connected to the user in the interaction graph G UI can form an indirect connection with the potential preference items through the semantic association of the knowledge graph in G CI . For example, the user has an interaction with item i and has no connection with the potential preference item j, and there is the same item auxiliary information entity a between item i and item j in the knowledge graph, then the user u and item j can form an indirect connection through the path u → i → a → j. The obtained collaborative information graph G CI is mathematically represented as follows:

[0090] G CI = (E ci , R ci , T ci ) Equation (1)

[0091] S20. Obtain a high - order collaboration information graph and a high - order interaction graph based on the high - order neighborhood graph generation module;

[0092] In a recommendation system based on a graph neural network, using high - order neighborhood information can enable the recommendation system to recommend items in the high - order neighborhoods of target users to them. By learning high - order neighborhood information, the system can break through the shallow preferences of users and capture the deep interest preferences of users through high - order paths (user a, item b, user c, item c,...). In addition, the connection between users and entities can refine user preferences and is conducive to the system capturing more complex user behavior patterns, which is crucial for revealing user preferences. Establishing the connection between users can discover the social relationships and behavioral similarities between users, enhancing the model's understanding of user preferences. To establish a connection with the high - order neighborhood, the present invention proposes a high - order neighborhood graph generation module, which takes the collaboration information graph and the user - item interaction graph as inputs and outputs the high - order collaboration information graph G HC and the high - order interaction graph G HI . The specific steps of S20 are as follows:

[0093] S21. Calculate the triple attention scores in the collaboration information graph;

[0094] To make full use of the information in the high - order information graph and capture the connections between users, relationships, and entities, the present invention proposes a triple attention mechanism for calculating triple scores and performing high - order neighborhood sampling in the collaboration information graph G CI . The triple attention scores can quantify the semantic association strength between the head entity, relationship, and tail entity. The present invention uses the attention mechanism to dynamically learn the importance of triples. For the input triple (h, r, t), the calculation formula of its triple attention score is as follows:

[0095]

[0096] where sim(·,·) is the calculation formula of cosine similarity. The input head entity embedding e h and the tail entity embedding e t are first mapped to the semantic space of the relationship through two trainable matrices W h and W t respectively, and then are combined with the relationship embedding e rPerform dot product to capture the connection between entities and relations. Finally, calculate the cosine similarity between the two to measure the degree of semantic alignment. α(h,r,t) is the triple attention score of the triple (h,r,t). In formula (2), the cosine similarity sim(e h W h ·e r ,e t W t ·e r ) is normalized with softmax to obtain the triplet attention score α(h, r, t).

[0097] S22, obtains the first-order neighborhood sampling results of the high-order collaborative information graph based on the triple attention score;

[0098] First, based on the triple attention score, the user's CI The highest rated K c A first-order neighborhood triplet, whose formula is as follows:

[0099]

[0100] in For graph G IC The triplet set of the first-order neighborhood of user u in K c It is a hyperparameter for selecting the number of elements, and the function TopK selects the set K triples with the highest attention score α(u,r,t) c Triples form a first-order neighborhood sampling graph

[0101] S23, obtaining a high-order neighborhood sampling result of the high-order neighborhood information graph in a convolution manner;

[0102] In order to obtain a sampling map of a user's higher-order neighborhood, the present invention generates a higher-order neighborhood sampling map in a convolution manner, specifically in the following manner:

[0103]

[0104] in Represents the user's i+1th order neighborhood sampling graph, as shown in formula (5): Sampling graph in the i-th order neighborhood Perform second-order neighborhood sampling on the user to obtain the candidate triple set S i , and select the set S i The K with the highest triple score c The triples form the i+1th order neighborhood sampling graph As shown in formula (6). contains the sampling results of the user's i-th order neighborhood, Performing second-order neighborhood sampling can capture the (i + 1)-order neighborhood connection of the user. Therefore, by means of convolution, using the sampling result of the previous layer as the input for the next layer of sampling can capture deeper user preferences. See Figure 3 , Figure 3 The left side shows the construction process of the high-order neighborhood sampling graph of Figure 3 The right side shows the corresponding relationship between the high-order neighborhood sampling graphs of user u1 at each order and its neighborhood.

[0105] S24, calculate the high-order adjacency matrix of the user-item interaction graph;

[0106] In the graph neural network recommendation system, the interaction between the user and the preferred items can intuitively and accurately reflect the user's preferences. Therefore, the present invention samples the high-order neighborhood preferred items of the user to obtain the high-order neighborhood sampling graph of the user-item interaction.

[0107] Given the user-item interaction graph G UI , which can be represented as an adjacency matrix A, with the user ID as the row and the item ID as the column. The adjacency matrix A ∈ {0, 1} |U|×|I| , if there is an interaction between user u and item i, then the matrix element A ui = 1, otherwise it is equal to 0. The present invention calculates the k-th power A k of the adjacency matrix to obtain the k-th order neighborhood of the user. The high-order adjacency matrix is used to describe the topological relationship of multi-hop connections between nodes. The high-order adjacency matrix A k is the k-th power of the adjacency matrix A, and its calculation formula is shown in Equation (7). The element represents the number of paths from node u to node i after k steps.

[0108] A k = A × A k-1 Equation (7)

[0109] According to the definition of the high-order adjacency matrix, when the matrix element indicates that there is more than one path between node u and node i, so node i is the k-th order neighborhood of node u. The relationship between the adjacency matrix and the neighborhood is as Figure 4 shown. Since the present invention only uses the high-order adjacency matrix to determine whether there is a connection between two nodes, the present invention unifies the elements greater than 1 in the high-order neighborhood matrix to 1 to determine that there is a connection between two nodes. The specific formula is as follows:

[0110]

[0111] S25, filter out the noise connections in the high-order adjacency matrix based on the user-item similarity;

[0112] The higher-order neighborhood nodes of the user are connected by multiple paths. If there is noise information in the path, the entire path will be contaminated, resulting in a decrease in the quality of node representation during information aggregation. At the same time, in the higher-order neighborhood graph neural network, due to the increase in the model's information aggregation range, the possibility of learning noise information further increases. Therefore, the present invention designs a filtering method to sample the user's higher-order preferred items to weaken the possibility of the appearance of noise connections. Specifically, the cosine similarity of user items is calculated, and then the neighborhood nodes are sampled according to the noise filtering threshold μ. The specific calculation formula is as follows:

[0113]

[0114] Among them, is the adjacency matrix of the interactive neighborhood sampling graph In the graph the connections with similarity less than μ are regarded as noise connections and the connections between them are disconnected to avoid interfering with the learning of node representation during the subsequent information aggregation process. Among them, the odd-order neighborhood of the user is its preferred item, so only the odd-order sampling graph k % 2 = 1.

[0115] S26. Use a 1x1 convolutional layer to learn the sampling results of each order neighborhood of the collaborative information graph and the interactive graph to generate a higher-order collaborative information graph and a higher-order interactive graph;

[0116] After completing the higher-order neighborhood sampling, for the collaborative information graph G CI obtain the sampling result The sampling result of the interactive graph G UI is where β is the hyperparameter of the sampling layer. In the graph neural network recommendation system, the neighborhood information of different orders has different effects on nodes. In addition, the importance of neighborhood information of each order in different datasets is also different. The present invention uses a 1×1 convolutional layer to aggregate the sampling results of the graph G CI and the graph G UI respectively to obtain the higher-order collaborative information graph G HC and the higher-order interactive graph G HI , allowing the model to learn the importance of each order neighborhood by itself. The specific aggregation method is as follows:

[0117]

[0118] Among them, 1×1Conv represents the 1x1 convolutional process, ω k represents the training weight corresponding to the k-th layer, and the softmax function is used for normalization to make the values in the higher-order graph stable, so as to ensure that the information aggregation module can operate stably. is the set of triples in the graph , corresponds to the sampling graph The adjacency matrix of. The obtained high-order graph G HC and G HI contains the high-order neighborhood connections of users and includes the connection weights of each connection and The model establishes connections between users and high-order neighborhood preference entities and related users in G through learning the high-order neighborhood HC and establishes connections with high-order preference items in G HI The construction of the high-order neighborhood graph can obtain more comprehensive and complex semantic information, expand the information perception range of the graph neural network, and make full use of the information contained in the graph.

[0119] S30. Based on the deep information aggregation module, learn the node representations in the graph;

[0120] After generating the high-order collaborative information graph G HC and the high-order interaction graph G HI , information aggregation is required to learn the embedding representations of the nodes in the graph. Traditional graph convolution uses weighted averaging in neighborhood aggregation. As the number of convolution layers increases, the node embeddings become similar, and the increase in the node information perception range will lead to the over-smoothing problem. To solve the over-smoothing problem, the present invention proposes a deep information aggregation module to learn the node embedding representations in graph G HC and graph G HI . The specific steps are as follows:

[0121] S31. Introduce the triple attention mechanism for information aggregation of the high-order collaborative information graph;

[0122] When aggregating the information of the high-order collaborative information graph, the present invention introduces the triple attention mechanism to learn the contribution of different triple relationships to the target node and realizes dynamic weight allocation. The information aggregation process is as Figure 5 shown, and the information aggregation calculation formula is as follows:

[0123]

[0124] where N(h) represents the first-order neighborhood of the head entity h in graph G HC , α(h, r, t) is the same as the triple attention score in formula (3), is the connection weight in graph G HC , is the node embedding of the (l + 1)-th layer. Each model layer will generate its embedding representation for the nodes in the graph. As the number of layers increases, the model information perception range becomes larger. The collaborative information embedding e ci of the nodes is obtained by combining the embeddings of each layer through layer combination. The calculation formula is as follows:

[0125]

[0126] where L is the number of model layers, and λ l is the aggregation weight of each layer's embedding, which is set to 1 / (L + 1) in the present invention, i.e., mean aggregation.

[0127] S32. Introduce an initial residual connection during the learning process of the high-order interaction graph node representation;

[0128] Since there is only one relationship between users and items in the high-order interaction graph, it is difficult for the attention mechanism to learn significant weight differences. Therefore, using the attention mechanism to alleviate the over-smoothing problem has limited effect. The present invention uses the initial residual connection technology (Initial Residual Connection) during the graph convolution process. In each layer of graph convolution, the initial residual connection performs weighted fusion of the features of the current layer and the initial features of the nodes, so that even if a large amount of neighborhood node information is aggregated, the nodes can still retain their own key information and maintain the distinctiveness of the features. In the graph neural network, the initial residual connection can effectively alleviate the over-smoothing phenomenon. The calculation formula for node embedding is as follows:

[0129]

[0130] Equations (17) and (18) are the calculation processes of the user embedding and item embedding of the (l + 1)-th layer respectively. Where represents the connection weight in the high-order interaction graph G HI , and η is a hyperparameter used to control the strength of the initial residual. After L layers of information aggregation, the node embeddings of each layer are obtained, and then the embeddings of each layer are fused through layer combination to obtain the high-order interaction embedding e hi :

[0131]

[0132] Equations (19) and (20) are the user-item interaction embeddings of user u and item i respectively, and λ l is a hyperparameter used to control the aggregation weight of each layer. In the present invention, the weights of each layer are set to 1 / (L + 1) uniformly, i.e., mean aggregation.

[0133] S33. Concatenate the collaborative information embedding and the high-order interaction embedding to obtain the final node representation;

[0134] The present invention obtains the final node representations of users and items by concatenating the two:

[0135]

[0136] e u and e i are the node representations of user u and item i respectively, and || is the concatenation operation. The present invention is based on the representation eu With e i Perform a prediction task.

[0137] S40. Train the deep information-aware graph neural network recommendation model. The specific steps are as follows:

[0138] S41. Calculate the preference score of the user for the item;

[0139] For the obtained user representation e u and the item representation e i , the present invention uses the inner product of the two as the prediction score:

[0140]

[0141] Is the prediction score of user u for item i.

[0142] S42. Loss function selection;

[0143] For model training, the present invention uses the Bayesian Personalized Ranking (BPR) loss to optimize the model. The BPR loss uses the items that the user has interacted with as positive samples, and randomly selects non-interacted items as negative samples. For a given user and positive and negative samples, calculate the prediction scores of the user for the samples, and optimize the model to maximize the difference in prediction scores between the positive and negative samples So that the model can learn the personalized preferences of the user, and rank the items that the user is more likely to be interested in before the items that the user is not interested in. The Bayesian personalized ranking loss is as follows:

[0144]

[0145] Where i is the positive sample, j is the randomly sampled negative sample, S d Is the training data set, σ(·) is the sigmoid function, θ is the training parameter of the model, and λ||θ|| 2 Is the regularization term.

[0146] The training process of the DIA-KGNN model is as follows: First, initialize all the parameters in the model. For the input user-item interaction graph G UI and the item knowledge graph G IK Fuse the two to obtain the collaborative information graph G CI . In the high-order neighborhood graph construction module, calculate the triple attention score for neighborhood sampling and construct the high-order collaborative information graph G HC , and construct the high-order interaction graph G HI by filtering out the noise connections through cosine similarity. For G HCObtain the triple attention scoring weights and perform neighborhood aggregation to obtain the embedding e hc , and at the same time for G HI Introduce an initial residual connection in the node embedding learning process to calculate the high-order interaction embedding e hi . After concatenating the two embeddings, the final node embedding e is obtained. Finally, the predicted scores of each item are calculated, and the BPR loss is used to adjust the model parameters. When the loss function converges, the model parameters are obtained.

[0147] The experiment is constructed based on the PyTorch framework. The model parameters are initialized using Xavier, the embedding dimension is set to 64, the regularization coefficient λ is set to 1e-5, the batch size is set to 1024 during the training process, the noise filtering threshold μ is set to 0.2, the model depth L is set to 2, the learning rate is adjusted in the range [1e-3, 1e-5], and the Adam optimization algorithm is used for iterative update of the parameters. This embodiment conducts experiments on the Last.fm, Book-Crossing, and MovieLens-20M datasets. Specifically, the Last.fm dataset is related to the music recommendation field and is provided by the music recommendation service Last.fm, and is commonly used in music recommendation and recommendation fields related to knowledge graphs; the Book-Crossing dataset is an online book rating dataset for recommendation system research, created by Cai-Nicolas Ziegler et al., and the data comes from the Book-Crossing book sharing and communication platform where users can rate books; the MovieLens-20M dataset is a large-scale movie rating dataset provided by the GroupLens research project, which contains approximately 20 million user movie ratings, providing rich resources for researchers to develop and test recommendation system algorithms. Since the original data is explicit feedback data, the present invention converts it into an implicit feedback form. Specifically, for the MovieLens-20M dataset, the interactions with a rating greater than or equal to the threshold of 4 are selected as positive samples. To control the computational cost, the first 200,000 positive sample records in the MovieLens20M dataset are selected. For the other two datasets, no rating threshold is set due to their high sparsity. In addition, corresponding item knowledge graphs are supplemented for all datasets. Finally, the users in the three datasets are screened, and users with less than 5 interaction records are removed to improve the quality of the training data. Recall and NDCG are selected as evaluation metrics for the experiment, and the method of the present invention is compared with the NGCF, LightGCN, CKE, KGAT, CKAN, KGRec, and KDGCN models.

[0148] Table 1 Performance comparison between DIA-KGNN and other recommendation models

[0149]

[0150] The experimental results are shown in Table 1. The table is divided into three parts. The upper part is the model without using the knowledge graph, the middle part is the model using the knowledge graph, and the lower part is the high-order neighborhood recommendation model. The bold data in the table are the optimal results. Observing the experimental results, the DIA-KGNN of the present invention achieves the best recommendation performance on all three datasets. On the Last.fm dataset, DIA-KGNN improves by 1.23% in the Recall@20 metric and 0.92% in the NDCG@20 metric compared to the second-best model; on the Book-Crossing dataset, DIA-KGNN improves by 3.18% in the Recall@20 metric and 1.04% in the NDCG@20 metric; on the MovieLens-20M dataset, DIA-KGNN improves by 0.76% in the Recall@20 metric and 0.94% in the NDCG@20 metric.

[0151] The present invention attributes the performance improvement of DIA-KGNN to the following reasons: (1) DIA-KGNN supplements the connection between the user and its high-order neighborhood nodes through the high-order neighborhood graph generation module, providing the basis for the model to perceive the user's deep preferences and alleviating the problem of data sparsity; (2) By constructing a high-order collaborative information graph, the connection between the user and its preferred entities and other users is established, enriching the semantic relationship and supporting the model to capture complex user behavior patterns; (3) In the deep information aggregation module, triple attention information aggregation and initial residual fusion methods are introduced to enhance the node representation quality and alleviate the over-smoothing problem caused by aggregating a large amount of node information.

[0152] In addition, the improvement of DIA-KGNN on the MovieLens-20M dataset is lower than that of the other two datasets because the preferred items of users in the MovieLens-20M test set are concentrated in the 3rd-order neighborhood of the users, which also shows that the method of the present invention also has good performance in low-order neighborhood recommendation.

[0153] Compared with other methods, the high-order neighborhood recommendation model has better recommendation performance than other recommendation models in most cases, especially on datasets with sparse data (Book-Crossing and Last.fm). This shows the importance of learning high-order neighborhood information, and capturing high-order neighborhood information through appropriate methods can improve the recommendation performance of the model.

[0154] To verify the high-order neighborhood recommendation performance of DIA-KGNN, the present invention conducts experiments on two datasets, Last.fm and Book-Crossing. Specifically, the present invention divides the test set according to the path length from the user to the item. The item order with a path length of 3 is the 3rd-order neighborhood of the user, and so on. An order of -1 indicates that there is no path to reach the node, representing the highest-order neighborhood. And the present invention selects two low-order recommendation models, LightGCN and KGRec, and a high-order neighborhood recommendation model, KDGCN, as comparison methods. See Figures 6 to 7 , the recommendation performance of DIA-KGNN is on par with KGRec in the 3rd-order neighborhood, indicating that DIA-KGNN ensures excellent recommendation results for low-order neighborhoods (3rd-order neighborhoods) while recommending high-order neighborhood items. This benefits from the 1x1 convolution in the high-order neighborhood graph generation module, which enables the model to learn the importance of each order of neighborhood by itself. Comparing the high-order recommendation performance above the 3rd order, the high-order neighborhood recommendation performance of DIA-KGNN is slightly better than that of KDGCN, and the number of stacked model layers of DIA-KGNN is much less than the 8-layer graph convolution of KDGCN. This shows that the high-order neighborhood graph can accurately capture the high-order neighborhoods of nodes and improve the high-order neighborhood recommendation ability of the model.

[0155] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for a graph neural network recommendation model with deep information perception, characterized in that, It includes the following steps: S10. Generate a collaborative information graph based on user-item interaction data and an item knowledge graph; S20. Obtain a high-order collaborative information graph and a high-order interaction graph based on a high-order neighborhood graph generation module; S30. Learn node representations in the high-order collaborative information graph and the high-order interaction graph through a deep information aggregation module; S40. Train the model and calculate the preference scores of the user for each item based on the training results, and generate a recommendation list for the user relying on the scores.

2. The method of a graph neural network recommendation model for deep information perception according to claim 1, wherein The S10 includes the following steps: S11. Represent user interaction data in the form of a graph structure; S12. Represent the item knowledge graph in the form of a graph structure; S13. Integrate the user-item interaction graph and the item knowledge graph to obtain a collaborative information graph.

3. The method of a graph neural network recommendation model for deep information perception according to claim 2, wherein, Specifically, S11 is to use the user-item interaction graph G for given user interaction data UI to represent the interaction data of users and items; the definition of the user-item graph is G UI ={(u, i, ε ui ) | u ∈ U, i ∈ I, ε ui ∈ Ε}, where u is a user node, i is an item node, and if there is an interaction relationship between user u and item i, there is an edge ε ui ; The specific content of S12 is that the item knowledge graph G IK is defined as G IK =(E k , R k , T k ), where E k is the set of all entities in the knowledge graph. In the item knowledge graph, entities are composed of item entities and item auxiliary information entities, that is, E k =I∪S Attri , S Attri is the set of item auxiliary information entities; R k is the set of all relationships in the knowledge graph R k ={r1, r2, r3......r |R|}, T k is the set of triples in the knowledge graph T k ={(h, r, t)|h, t∈E k , r∈R k}, and the triples in the set are represented as (h, r, t), where h and t represent the head entity and the tail entity respectively, and r is the relationship between these two entities; Specifically, S13 is to combine the user-item interaction graph G UI with the item knowledge graph G IK to obtain a collaborative information graph G CI to enrich the connectivity of the nodes in the graph; for potential preference items that have no connection with the user in the interaction graph G UI , indirect connections are formed with the potential preference items through the semantic association of the knowledge graph in G CI ; the obtained collaborative information graph G CI is mathematically represented as follows: G CI = (E ci , R ci , T ci ) Equation (1) Among them, E ci , R ci , T ci are respectively the entity set, the relationship set, and the triple set in graph G CI .

4. A method for a graph neural network recommendation model with deep information perception according to claim 1, characterized in that The S20 includes the following steps: S21. Calculate the triple attention scores in the collaborative information graph; S22. Obtain the first-order neighborhood sampling results of the high-order collaborative information graph based on the triple attention scores; S23. Obtain the high-order neighborhood sampling results of the high-order neighborhood information graph in a convolutional manner; S24. Calculate the high-order adjacency matrix of the user-item interaction graph; S25. Filter out the noise connections in the high-order adjacency matrix based on the user-item similarity; S26. Use a 1x1 convolutional layer to learn the neighborhood sampling results of each order of the collaborative information graph and the interaction graph to generate a high-order collaborative information graph and a high-order interaction graph.

5. The method of a graph neural network recommendation model for deep information perception according to claim 4, characterized in that Specifically, for the input triple (h, r, t), the calculation formula for its triple attention score is as follows: where sim(,) is the calculation formula of cosine similarity, and the input head entity embedding e h and the tail entity embedding e t First, they are respectively mapped to the semantic space of the relationship through two trainable matrices W h and W t Then, a dot product is performed with the relationship embedding e r of the two to capture the connection between the entity and the relationship; finally, the cosine similarity between the two is calculated to measure the degree of semantic alignment; α(h, r, t) is the triple attention score of the triple (h, r, t). In formula (2), after softmax normalization of the cosine similarity sim(e h W h ·e r , e t W t ·e r ), the triple attention score α(h, r, t) is obtained; Specifically, S22 is as follows. First, based on the triple attention score, obtain the K CI first-order neighborhood triples with the highest scores of the user in the graph G c , and the formula is as follows: Among them is the graph G IC in the set of triples of the first-order neighborhood of user u, and K c is a hyperparameter for selecting the number of elements. The function TopK selects the set with the highest triple attention score α(u, r, t) among the K c triples to form the first-order neighborhood sampling graph Specifically, S23 is to generate a high-order neighborhood sampling graph in a convolutional manner, and the specific method is as follows: Among them, represents the (i + 1)-th order neighborhood sampling graph of the user, and the (i + 1)-th order sampling graph shown in formula (5) is based on the i-th order neighborhood sampling graph to perform second-order neighborhood sampling of the user to obtain the candidate triple set S i , and select the set S i with the top K c triples with the highest scores to form the (i + 1)-th order neighborhood sampling graph Since the graph contains the sampling results of the user's i-th order neighborhood, performing second-order neighborhood sampling on the graph can capture the user's (i + 1)-th order neighborhood connections; Specifically, S24 is to give the user-item interaction graph G UI , which can be represented as an adjacency matrix A. The matrix takes the user ID as the row and the item ID as the column. The adjacency matrix A ∈ {0, 1} |U|×|I| . If there is an interaction between user u and item i, then the matrix element A ui = 1; otherwise, it is equal to 0. By calculating the k-th power of the adjacency matrix A k , the k-th order neighborhood of the user can be obtained. The high-order adjacency matrix is used to describe the topological relationship of multi-hop connections between nodes. The high-order adjacency matrix A k is the k-th power of the adjacency matrix A, and its calculation formula (7). The element represents the number of paths from node u to node i after k steps A k = A × A k-1 Equation (7) According to the definition of the high-order adjacency matrix, when the matrix element indicates that there is more than one path between node u and node i, so node i is the k-order neighborhood of node u; unify the elements greater than 1 in the high-order neighborhood matrix to 1 to determine the existence of a connection between two nodes. The specific formula is: Specifically, S25 is to calculate the cosine similarity between the user and the item, and then sample the neighborhood nodes according to the noise filtering threshold μ. The calculation formula is: Among them, is the adjacency matrix of the interactive neighborhood sampling graph . In the graph , connections with a similarity less than μ are regarded as noise connections and the connections between them are disconnected to avoid interfering with the learning of node representations during the subsequent information aggregation process. Since the odd-order neighborhood of the user is their preferred item, only the odd-order sampling graph Specifically, S26 uses a 1×1 convolutional layer to process graph G CI and graph G UI to respectively aggregate the sampling results to obtain a high-order collaboration information graph G HC and a high-order interaction graph G HI , enabling the model to automatically learn the importance of each-order neighborhood. The specific aggregation method is as follows: Among them, 1×1Conv represents the 1x1 convolution process, and ω k represents the training weight corresponding to the k-th layer. The softmax function is used for normalization to stabilize the values in the high-order graph, ensuring the stable operation of the information aggregation module; is the set of triples in the graph , corresponding to the adjacency matrix of the sampled graph . The obtained high-order graph G HC and G HI contain the high-order neighborhood connections of users and include the connection weights of each connection and 6. A method for a graph neural network recommendation model for deep information perception according to claim 5, characterized in that The S30 includes the following steps: S31. Introduce a triple attention mechanism for information aggregation in the high-order collaborative information graph; S32. Introduce an initial residual connection in the learning process of the high-order interaction graph node representation; S33. Concatenate the collaborative information embedding and the high-order interaction embedding to obtain the final node representation.

7. The method of a graph neural network recommendation model for deep information perception according to claim 6, wherein Specifically, S31 is that the information aggregation calculation formula is: where N(h) represents the first-order neighborhood of the head entity h in the graph G HC The triple attention score α(h, r, t) is the same as in Equation (3), and is the connection weight in the graph G HC ; is the node embedding of the (l + 1)-th layer; each model layer generates an embedding representation for the nodes in the graph. As the number of layers increases, the information perception range of the model becomes larger. The collaborative information embedding e of the nodes is obtained by combining the embeddings of each layer through layer combination ci , and its calculation formula is: Among them, L is the number of model layers, and λ l is the aggregation weight of the embeddings of each layer; Specifically, S32 is that in a graph neural network, the initial residual connection can effectively alleviate the over-smoothing phenomenon, and the calculation formula for the node embedding is: Equations (17) and (18) are the calculation processes of the user embedding and item embedding in the (l + 1)-th layer, respectively. Among them, represents the connection weights in the high-order interaction graph G , η is a hyperparameter used to control the strength of the initial residual. After L layers of information aggregation, the node embeddings of each layer are obtained. Subsequently, the embeddings of each layer are fused through layer combination to obtain the high-order interaction embedding e HI : hi ​ Equations (19) and (20) are the user-item interaction embeddings of user u and item i respectively, and λ l is a hyperparameter used to control the aggregation weights of each layer; Specifically, S33 is to obtain the final node representation of the user and the item by concatenating the two: e u and e i are the node representations of user u and item i respectively, and || is the concatenation operation.

8. A method for a graph neural network recommendation model with deep information perception according to claim 7, characterized in that, The S40 includes the following steps: S41. Calculate the preference scores of the user for the items; S42. Select a loss function and train the model.

9. The method of a graph neural network recommendation model for deep information perception according to claim 8, wherein, Specifically, S41 is to use the inner product of the obtained user representation e u and the item representation e i as the predicted score: The predicted score for user u on item i.

10. The method of a graph neural network recommendation model for deep information perception according to claim 9, wherein, The training model in S42 is specifically as follows. First, initialize all parameters in the model. For the input user-item interaction graph G UI and the item knowledge graph G IK Fuse the two to obtain the collaborative information graph G CI ; In the high-order neighborhood graph construction module, calculate the triple attention scores for neighborhood sampling and construct the high-order collaborative information graph G HC , and construct the high-order interaction graph G by filtering out noise connections through cosine similarity HI ; For G HC Obtain the triple attention score weights and perform neighborhood aggregation to obtain the embedding e hc , and at the same time for G HI Introduce the initial residual connection in the node embedding learning process to calculate the high-order interaction embedding e hi , concatenate the two embeddings to obtain the final node embedding e; Finally, calculate the predicted scores of each item, use the BPR loss to adjust the model parameters, and when the loss function converges, obtain the model parameters.

Images