Recommendation method and device based on graph neural network

Abstract

A method for graph neural network-based recommendation is disclosed. The method comprises the following steps: receiving a user-project bipartite graph, wherein the user-project bipartite graph comprises a user node set representing a user set, a project node set representing a project set, and an edge set representing interaction between users and projects; obtaining user embedding of each user in the user set and project embedding of each project in the project set; performing an intra-layer aggregation process to generate an aggregated user embedding of each user in the user set and an aggregated item embedding of each item in the item set based on different weight coefficients of each user and each item; executing an interlayer propagation process to generate a propagation user embedding of each user in the user set and a propagation item embedding of each item in the item set based on different weight coefficients of each propagation layer; and making recommendations for the users in the user set based on the propagation user embedding and the propagation item embedding.

Description

Technical Field

[0001] This disclosure generally relates to artificial intelligence technology, and more specifically, to recommendation systems based on graph neural networks. Background Technology

[0002] In the real world, some data structures can be represented by graphs. Graphs can consist of nodes and edges. Nodes can be people, places, things, etc., and edges can define the relationships between nodes. Graph Neural Networks (GNNs) are a special type of neural network that can work with graph data structures. GNNs can be used to predict nodes, edges, and graph-based tasks. An input graph can be passed through a series of neural networks and transformed into graph embeddings, maintaining valid information about the nodes, edges, and global context of the input graph.

[0003] In recent years, Generative Neural Networks (GNNs) have made significant progress in recommendation tasks. The core mechanism of GNN-based recommendation systems is to iteratively aggregate neighboring information on the user-item interaction graph. However, existing GNNs treat users and items equally and cannot distinguish the various local patterns of each node, making them suboptimal in recommendation scenarios. Therefore, there is a challenge to improve GNN-based methods for recommendation. Summary of the Invention

[0004] The following is a simplified overview of one or more aspects of this disclosure in order to provide a basic understanding of these aspects. This overview is not a comprehensive overview of all anticipated aspects, and is neither intended to identify key or essential elements of all aspects, nor to depict the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that follows.

[0005] In one aspect of this disclosure, a method for recommendation based on a graph neural network is disclosed. The method includes: receiving a user-item bipartite graph, the user-item bipartite graph including a set of user nodes representing a set of users, a set of item nodes representing a set of items, and a set of edges representing interactions between the users and the items; obtaining a user embedding for each user in the user set and an item embedding for each item in the item set; performing an intra-layer aggregation process to generate aggregated user embeddings for each user in the user set and aggregated item embeddings for each item in the item set based on different weight coefficients for each user and each item; performing an inter-layer propagation process to generate propagated user embeddings for each user in the user set and propagated item embeddings for each item in the item set based on different weight coefficients for each propagation layer; and making recommendations for users in the user set based on the propagated user embeddings and the propagated item embeddings.

[0006] In another aspect of this disclosure, an apparatus for recommendation based on a graph neural network is disclosed. The apparatus includes: an input module for receiving a user-item bipartite graph, the user-item bipartite graph including a set of user nodes representing a set of users, a set of item nodes representing a set of items, and a set of edges representing interactions between the users and the items; an embedding module for obtaining a user embedding for each user in the user set and an item embedding for each item in the item set; an aggregation module for performing an intra-layer aggregation process to generate an aggregated user embedding for each user in the user set and an aggregated item embedding for each item in the item set based on different weight coefficients for each user and each item; a propagation module for performing an inter-layer propagation process to generate a propagated user embedding for each user in the user set and a propagated item embedding for each item in the item set based on different weight coefficients for each propagation layer; and a recommendation module for making recommendations for users in the user set based on the propagated user embeddings and the propagated item embeddings.

[0007] In another aspect of this disclosure, an apparatus for recommendation based on a graph neural network is disclosed. The apparatus may include a memory and at least one processor coupled to the memory. The at least one processor may be configured to: receive a user-item bipartite graph, the user-item bipartite graph including a set of user nodes representing a set of users, a set of item nodes representing a set of items, and a set of edges representing interactions between the users and the items; obtain a user embedding for each user in the user set and an item embedding for each item in the item set; perform an intra-layer aggregation process to generate aggregated user embeddings for each user in the user set and aggregated item embeddings for each item in the item set based on different weight coefficients for each user and each item; perform an inter-layer propagation process to generate propagated user embeddings for each user in the user set and propagated item embeddings for each item in the item set based on different weight coefficients for each propagation layer; and make recommendations for users in the user set based on the propagated user embeddings and the propagated item embeddings.

[0008] In another aspect of this disclosure, a computer-readable medium is disclosed for storing computer code for recommendation based on a graph neural network. When executed by a processor, the computer code enables the processor to: receive a user-item bipartite graph, the user-item bipartite graph including a set of user nodes representing a set of users, a set of item nodes representing a set of items, and a set of edges representing interactions between the users and the items; obtain a user embedding for each user in the user set and an item embedding for each item in the item set; perform an intra-layer aggregation process to generate aggregated user embeddings for each user in the user set and aggregated item embeddings for each item in the item set based on different weight coefficients for each user and each item; perform an inter-layer propagation process to generate propagated user embeddings for each user in the user set and propagated item embeddings for each item in the item set based on different weight coefficients for each propagation layer; and make recommendations for users in the user set based on the propagated user embeddings and the propagated item embeddings.

[0009] In another aspect of this disclosure, a computer program product for recommendation based on graph neural networks is disclosed. The computer program product may include processor-executable computer code configured to: receive a user-item bipartite graph, the user-item bipartite graph including a set of user nodes representing a set of users, a set of item nodes representing a set of items, and a set of edges representing interactions between the users and the items; obtain a user embedding for each user in the user set and an item embedding for each item in the item set; perform an intra-layer aggregation process to generate aggregated user embeddings for each user in the user set and aggregated item embeddings for each item in the item set based on different weight coefficients for each user and each item; perform an inter-layer propagation process to generate propagated user embeddings for each user in the user set and propagated item embeddings for each item in the item set based on different weight coefficients for each propagation layer; and make recommendations for users in the user set based on the propagated user embeddings and the propagated item embeddings.

[0010] Other aspects or variations of this disclosure will become apparent from the following detailed description and accompanying drawings. Attached Figure Description

[0011] The following figures depict various embodiments of the present disclosure for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the methods and structures disclosed herein can be implemented without departing from the spirit and principles of the present disclosure described herein.

[0012] Figure 1A and Figure 1BAn example of a user-item bipartite graph for a recommendation system and its higher-order connectivity with a tree-structured graph is shown according to one aspect of this disclosure.

[0013] Figure 2 The diversity of the user's local structure in a recommendation scenario according to one aspect of this disclosure is illustrated.

[0014] Figure 3 An architecture for an adaptive graph neural network for recommendation is shown according to one aspect of this disclosure.

[0015] Figure 4 A flowchart of a recommendation method based on a graph neural network according to one aspect of this disclosure is shown.

[0016] Figure 5 A block diagram of an apparatus for recommendation based on a graph neural network, according to one aspect of this disclosure, is shown.

[0017] Figure 6 A block diagram of an apparatus for recommendation based on a graph neural network, according to one aspect of this disclosure, is shown. Detailed Implementation

[0018] Before explaining any embodiment of this disclosure in detail, it should be understood that the application of this disclosure is not limited to the construction details and feature arrangements set forth in the following description. This disclosure can have other embodiments and can be practiced or implemented in various ways.

[0019] Figure 1A An example of a user-item bipartite graph 110 of a recommendation system according to one aspect of this disclosure is shown. Figure 1A As shown, nodes u1, u2, u3, u4, and u5 can represent a set of users in the recommender system, such as customers on a shopping platform, and nodes v1, v2, v3, and v4 can represent a set of items in the recommender system, such as clothes, shoes, mobile phones, and computers on a shopping platform. The edges between user nodes and item nodes can represent interactions between customers and items on a shopping platform, such as purchasing, rating, clicking, or adding to favorites. Although a shopping platform is used here as an example application of a recommender system, recommender systems can also be used in various other applications, such as smart manufacturing analytics and scalable industrial analytics, and the terms "user" and "item" in this disclosure can refer to their general meaning in different applications of recommender systems.

[0020] Figure 1B One aspect of this disclosure is shown. Figure 1A An example of the higher-order connectivity of the user-item bipartite graph 110 with a tree structure graph 120. (See also...) Figure 1BAs shown, project nodes v1 and v2 are 1-hop (h=1) neighbors of user node u1. User nodes u2, u3, and u4 are 2-hop (h=2) neighbors of user node u1. Project nodes v3 and v4 are 3-hop (h=3) neighbors of user node u1.

[0021] Graph Neural Networks (GNNs) have achieved significant performance on recommendation tasks. Specifically, GNNs perform message passing on the graph structure to aggregate local neighborhood information and stack multiple representation layers with high-order propagation. GNN-based recommendation systems apply information propagation operations on the user-item bipartite graph, such as... Figure 1A and 1B As shown. However, there are still remaining problems in existing GNN-based models used for recommendation.

[0022] First, existing GNN-based recommendation systems do not distinguish between node types. User-item interaction networks are a special type of graph where edges can only exist between users and items. In other words, there is no direct communication between two users or items. So far, existing GNN-based recommendation methods are no different from general GNNs that handle other common graphs (such as reference graphs, social networks, etc.), i.e., they employ the same modeling strategy for all users and items. Typically, in high-order neighborhood aggregation settings, a user only aggregates information from 1-hop item neighbors with layer 0 embeddings, 2-hop users with layer 1 representations, and again 3-hop items with layer 2 embeddings. By making all layers identical, the aggregation step in existing GNN-based recommendation systems ignores the semantic differences between users and items at each subgraph structure—and by expanding their embeddings.

[0023] Secondly, the local structure of different users / projects leads to different recommendations, such as... Figure 2 As shown in Figures 210 and 220, which are based on the Ali and AMiner datasets, respectively, the horizontal axis represents node degree and the vertical axis represents the recall metric @20. The Ali dataset comes from the Alibaba e-commerce platform, and its users' interaction records exceed 10 in the user-item bipartite graph. The AMiner dataset collects scientific resource reading behavior from its users' AMiner.org. Figure 2 This demonstrates the node-level necessity of GNN-based recommender systems. Specifically, 1- / 2- / 3-hop aggregation and propagation are performed using LightGCN on the Ali and AMiner datasets, respectively. Figure 2The diagram illustrates the relationship between recall @20 results and user nodes with different degree intervals. It can be observed that the average result distribution of nodes with different degrees on each hop propagation is highly unbalanced, and the optimal propagation number for each node is uncertain, implying that each user / item has its own local structure. This suggests that different nodes with different structures should be assigned different importance in recommendation, and that node-by-node adaptive aggregation in the GNN used for recommendation is a necessary step in learning node-level differences between layers. Therefore, this disclosure investigates whether each node should be treated differently in different layers during the aggregation process of GNN-based recommendation methods.

[0024] Previous GNN-based recommender systems treated all nodes as identical in their aggregation steps, ignoring the importance of their variations, leading to suboptimal performance. Previous attention-based GNN models (e.g., GAT) attempted to model the distinct influence between user and item nodes without considering local diversity. Furthermore, degree-based GNN processes (e.g., LightGCN) improved embedding learning and achieved better performance in bipartite graphs for recommendation, but still neglected the local diversity problem, in order to differentiate the importance of historical items in interactions. In recent diffusion-based GNN research (e.g., GDC), appropriate neighborhoods are flexibly selected to enhance the expressiveness of GNNs during propagation. For example, graph diffusion convolution models incorporate hot kernels and node centrality information into the message passing process of GNNs to enhance structural smoothness. However, GDC is based on the homogeneity assumption and performs poorly on link prediction tasks. Adaptive graph convolution (ADC) work proposes to utilize hot kernel theory to learn the optimal neighborhood for enhancing low-frequency filters. Although hot kernels were originally used to implement neighborhood selection in GNNs, they also provide a natural solution to the aforementioned recommendation problems.

[0025] This disclosure presents a novel AdaPtivE GNN method (ApeGNN) that performs node-level adaptive aggregation in a GNN for recommendation. Instead of treating each user and item equally at every layer during high-order aggregation and propagation, the ApeGNN method adaptively assigns unique weights (inner layer weights) to each hop of neighbors using the graph diffusion process and distinguishes information from different GNN layers, thereby improving aggregation methods from fixed aggregation to node-level aggregation. The concept of ApeGNN for GNN-based recommendation is that each user... And each project Each has a unique aggregation weight ( t ( u i ), l )and θ ( t ( vj ), l ), which has a coefficient t and layers l These diffusion weights represent the weights captured by ApeGNN in each layer. and Its unique contribution to embedding.

[0026] ApeGNN can be used as a plugin and naturally integrated into any existing GNN-based model for recommendation without modifying the model architecture. Extensive experiments with the ApeGNN method have been conducted on six widely used public datasets, comparing it with representative GNN-based and attention-based models to demonstrate its effectiveness. The experimental procedure—data splitting, optimization, and evaluation—is followed as in existing GNN-based recommendation research. Results show that the proposed ApeGNN consistently outperforms state-of-the-art GNN baselines on all datasets in terms of both recall and NDCG (up to 48.94% relative improvement on Ali, 24.09% on Amazon, and 7.67% on AMiner), demonstrating the benefits of node-based adaptive aggregation. Case studies show that adaptive node weights contribute to both overall and layer-by-layer performance.

[0027] Graph Neural Networks for Recommendation This section describes a GNN-based recommender system and discusses the limitations of existing GNNs used for recommendations.

[0028] Generally speaking, such as Figure 1A As shown, the input to a GNN-based recommendation model can be a user-item bipartite graph. It has users ,project The collection of data and interactions between users and items (such as purchases, ratings, and clicks). ,in It is the number of users, and Indicates the total number of items. For each interaction, if... and Interaction, It is set to 1 if true, otherwise 0. The purpose of a recommender system is to utilize... Predicting items that users will further interact with. Generative Neural Networks (GNNs) are already widely used in recommender systems. Similar to common GNN models, GNN-based recommender systems perform message passing on the input graph structure to obtain contextual representations. Typically, the message passing process includes aggregation and pooling.

[0029] Regarding aggregation, for nodes During propagation in GCN-based models, at layers The general representation of can be expressed as: (1) in and These represent the aggregation and update functions, respectively. Taking GCN (Graph Convolutional Network) as an example, it aggregates the neighboring nodes of each node and performs message passing. Specifically, the propagation rule equation (1) of GCN can be defined by the following equation: (2) in express l After GCN The embedded representation. In particular, It is a 0-layer embedding; It is a non-linear activation used for feature propagation. It is a weight matrix; It is a node The neighborhood of T. T can be accessed through... Calculation, where It has The degree matrix of the diagonal nodes. Note that... and ,in It is an adjacency matrix with self-circulation, and It is the identity matrix that implies a self-loop connection on the node.

[0030] Inspired by the design and ideas of GCN models, emerging research (e.g., GCMC and NGCF) attempts to utilize GCN architectures for recommendation tasks. However, recent research reports that some of the most commonly used GCN designs complicate the learning process without significantly contributing to recommendation performance. These designs are intended to simplify GCNs for recommendation and achieve better performance. LightGCN, for example, removes... and Therefore, its aggregation and propagation process can be represented as: (3) Among them, the normalization term Depend on calculate.

[0031] Regarding pooling, in GCNs, the representation vectors learned by the final layer are used to perform node classification. However, the final representation of a node can be varied by changing the pooling operation at each GCN layer (also known as the layer combination operation). In most GNN-based recommendation models, pooling functions / modules can be combined into the embeddings propagated at each layer. And generate the final embedding, i.e., the representation vector. Common pooling operations include weighted sum-based pooling, convolution-based pooling, attention-based pooling, and other methods. Typically, users... Pooling function This can be expressed by the formula: (4) in, It is the total number of propagation layers, and the project The pooling functions are similar to user-defined pooling functions. In particular, weighted pooling functions are the most commonly used. For example, It is the first one used for final embedding l The weights of the layer embeddings. In particular, they assign constant values ​​to the embeddings of each layer (e.g., The weights are fixed to perform layer composition. In other words, the importance of the embeddings represented by each subgraph structure in different GCN layers is the same.

[0032] Therefore, existing GNN-based recommendation models have limitations. By default, current GNN-based recommendation models have the following settings during the aggregation process: first, these methods treat all nodes in the bipartite graph as identical; second, they consider the embeddings of multiple layers that have equal importance in a local perspective.

[0033] By design, GNN aggregators (or aggregation modules) mainly fall into two categories: degree-based aggregators represented by LightGCN and attention-based aggregators represented by GAT. In practice, they do not explicitly distinguish between nodes with a global perspective during the message passing process, even though all nodes in a recommender system are inherently different.

[0034] For the degree-based aggregator represented by LightGCN, the extended equation (3) is used for user embedding. The graph convolutions on the first and second layers are formulated as equation (5), and the item embeddings are similarly formulated.

[0035] (5) in, and These are asymmetric and symmetric normalization, respectively. It can be observed that the aggregation and propagation processes of user and item embeddings are exactly the same, that is, information updated from nodes of the same type (i.e., user or item) propagates in odd-numbered GCN layers (1, 3, ...), and messages from another type propagate in even-numbered GCN layers.

[0036] Degree-based aggregators are applied to each node to control the decay factor during aggregation and propagation, potentially making these processes different. However, the normalization coefficient is a constant value calculated from the degree of the node, independent of node type and subgraph structure. In fact, users and items are two different types of nodes with unique semantics in the user-item bipartite graph, and their own embeddings in different subgraphs should naturally be considered dissimilar. This disclosure first attempts to explicitly distinguish between node-aware users and items in a simple GNN architecture and investigates whether different treatments of them can benefit GNN-based recommender systems.

[0037] For an attention-based aggregator represented by GAT, the contributions of each neighbor should be different when obtaining the representation of a node through all its neighbors. Unlike degree-based aggregators, attention-based aggregators assign different weights to the neighbors of each node during aggregation in a local graph structure. The aggregation process is described as follows: (6) in, The attention score is typically calculated using the Softmax function. Attention weights are calculated in the local graph and implicitly propagated to the next layer via a non-linear activation function, ignoring the decay factor of the centrality distribution of each node from a node-by-node perspective. However, embeddings at different layers of a (graph) neural network should inherently capture different levels of features. Furthermore, as mentioned above, information updated from the same type and other types of nodes in a GNN-based recommendation model propagates in odd and even GCN layers, respectively. This disclosure investigates whether different weights on each node should be assigned from one layer to another. Therefore, this disclosure attempts to address the extent to which such node-by-node and layer-by-layer distinction can improve the recommendation performance of GNNs.

[0038] Adaptive Graph Neural Network (ApeGNN) This section describes an improved GNN-based recommender system, ApeGNN. The main idea of ​​ApeGNN is to distinguish each user and item from the nodes during aggregation and compute the influence in a node-by-node manner within the GNN. To achieve this, the idea of ​​adaptive operations based on graph diffusion is incorporated into the aggregation. The following will combine... Figure 3 This disclosure describes the details of the ApeGNN model.

[0039] Figure 3An architecture for ApeGNN for recommendation according to one aspect of this disclosure is shown. The input 310 of ApeGNN is a user-item bipartite graph consisting of a set of user nodes, a set of item nodes, and a set of edges representing interactions between users and items. The main parts of the ApeGNN model include node-level inner-layer aggregation and inter-layer propagation, as shown in box 320, with user... u 1 is used as an example. In ApeGNN, during aggregation and propagation, adaptive weights are utilized via hot kernels (as shown in boxes 322, 324, and 326) or personalized page ranking. Different models are used for users and projects at each level. Project nodes (e.g., projects shown in box 330) are modeled differently. v The aggregation and propagation processes in 1) are similar to those in box 320 for user nodes, and therefore, for the sake of simplicity, Figure 3 It is not shown in detail in the text.

[0040] As shown in box 320, a node-by-node adaptive aggregation mechanism was designed to incorporate node importance into the existing aggregation process in GNN-based recommendation models. For users... and its neighboring nodes , with users Weighting coefficient function aggregate functions It can be represented as: (7) in, User The parameters of the adaptive operation. Similarly, with items Weighting coefficient function The aggregation function 𝐴𝐺𝐺 can be described in equation (8). Then, it will be used during the aggregation process described below. and .

[0041] (8) Regarding weighting coefficients As mentioned earlier, the semantics contained in embeddings at different layers often differ in graph neural networks. For a GNN on a bipartite graph, each layer consists of users or items, which naturally makes the semantics of each layer different. In other words, embeddings at different layers should be processed discriminatively by setting different weights to capture the unique semantics of each layer during aggregation. Specifically, two methods, hot kernel (HT) and personalized page ranking (PPR), are proposed in this disclosure to simulate the graph diffusion process and provide better support for importance selection. The formulation of these two methods into coefficients is described in detail below. .

[0042] First, regarding the hot kernel, feature propagation between nodes in a GNN-based model can be viewed as a practical application of Newton's law of cooling (also known as the hot kernel), where heat is transferred from a region with a higher temperature to a region with a lower temperature. That is, the embedding propagation between two nodes is naturally proportional to their representation. Therefore, the derivation of this prior knowledge is calculated as follows: (9) in, and Indicates user and projects In time The following is a representation. This derivation indicates... Once related to its neighbors. Based on the diffusion design of the hot kernel, the hot kernel can be incorporated into a GNN-based model for personalized recommendations. Given the time of the hot kernel in the graph... Represented as The initial definition, where, It is a picture The Laplace matrix. According to this definition, equation (3) can be reformulated as: (11) in, Indicates diffusion time The hidden representation of subsequent nodes. For With a hot core The convolution kernel is expressed by the formula: in, Indicates the current layer, parameters With the The size of the neighborhood of a node at a given layer is related to the size of that neighborhood. Therefore, each node is related to... function and with function Used for automatic learning and updating during training and Its formula can be described as: (13) Regarding Personalized Page Rank (PPR), the ApeGNN method aims to identify local patterns from the graph structure of each node. To achieve this, the PPR method can be used to construct graph structure information and assign unique weights to each node. Users have transient probabilities. coefficient It can be represented as: (14) By utilizing a hot kernel and PPR, the ApeGNN method can flexibly assign appropriate weights to each node to enhance the low-frequency filter and implement smoothness on the graph.

[0043] On the importance of centrality ,like Figure 2 As shown, The optimal propagation is uncertain. Therefore, when modeling node importance, neighbor weights at each layer should be considered, and the different influences between user nodes and item nodes should be modeled when node representations are obtained during aggregation. Previous research on node importance estimation has shown that node importance is positively correlated with its centrality in the graph. Typically, user nodes... in-degree This represents its centrality and popularity. Therefore, we use in-degree at the initial layer. As weight and This is to model the differences between users and projects. Here, we provide users with... definition and obtain The centrality importance of the project is similar to the centrality importance of the user in equation (15).

[0044] (15) in, It is a small, positive constant, and is chosen as And the probability of teleportation .

[0045] Weights can exist during aggregation. Three different values, including Fixed initial values ​​for user type and project type (fixed) Train a decentralized system for each user and project. (train For each user and project, a unique, centrally-oriented training method is used. t (ApeGNN- t ). In combining ApeGNN with its three variants (fixed) t ,train t and ApeGNN- tAfter comparison, it is shown that optimal performance can be obtained when node centrality is considered and a unique node is trained on all datasets. It also shows that the degree of a node is important for representing its centrality and popularity in user-item interaction graphs, further validating the need to consider importance during aggregation and propagation. Furthermore, a unique node is trained separately for all nodes and all items. t The results are better than fixing all nodes on six datasets. t It demonstrates that users and items are different types of nodes and should be differentiated to learn better representations and improve recommendation performance. In summary, ApeGNN can distinguish between users and items and, during the aggregation process, leverage ApeGNN- t Assigning weights to nodes based on both the center and the layer can improve recommendation performance.

[0046] In ApeGNN, by taking a user-item interaction bipartite graph as input, the embeddings of each user and each item are distinguished by adaptive aggregation, and the embeddings are parameterized node by node to form the final representation.

[0047] By using enhanced representations during the aggregation process, each embedding layer can be added to the propagation layer to mine higher-order connectivity information. (User) and projects In the The propagation embedding of layers is formulated as follows: (16) in, and yes as well as In the The unique weight of the layer, and This is the symmetric normalization recommended by GCN. Note that the effect of self-loop connections in GCN layers can be captured by the weighted summation specified by LightGCN. By following the same settings, self-loop connections can be removed to reduce information redundancy.

[0048] By expanding the embedding aggregation and propagation functions with convolutional kernels, the first... Users and items in the layered ApeGNN have weight matrices Adaptive graph convolution matrix and This can be expressed as equation (17): (17) By converting equation (17) into the actual model training process, the graph convolution matrix associated with the layer can be transformed. The pooling operation in box 340.

[0049] Given the input user and item embedding at level 0 and As in box 322, they first propagate through higher layers via equation (16), as in boxes 324 and 326. User and item integrations can be updated using equation (1) by embedding the latest and current layers. The embeddings of each layer can then be combined to formulate a final user recommendation. (e.g., in box 340) and items The final embedding. In ApeGNN, equation (4) can be formulated as a model of the hop-by-hop semantic differences represented by each layer of embeddings. Therefore, and The final embedding is pooled as follows: (18) in, and Used for users and projects The The weights and importance of layer embeddings are modeled.

[0050] The impact of layer number φ on propagation was analyzed from 1 to 4, and compared with LightGCN. The results show that, overall, ApeGNN outperforms LightGCN across all datasets and all layer settings, with performance initially increasing and then decreasing as the number of layers increases. Specifically, performance increases to a peak as the number of layers increases from 1 to 2 (or 3), and then decreases. This suggests that it is helpful for nodes to consider their lower-order neighbors, which can smooth their embeddings and yield strong representations.

[0051] Finally, in box 350, and The inner product of the embeddings can be calculated as It can be used to calculate users For the project Preferences.

[0052] The neural network structure of ApeGNN can be optimized using Bayesian Personalized Ranking (BPR) loss. BPR is a pairwise loss, which is formulated as follows: (19) in, It is paired training data; It is the set of pairs of interactions between users and projects that have not been actually observed, and It is a collection of interaction records. Influence The strength of regularization; the sigmoid function in this formula is determined by Indicate; and utilize Regularization is used to avoid overfitting.

[0053] In ApeGNN, L2 These are additional hyperparameters. ApeGNN is relatively insensitive to these hyperparameters. Different coefficients were analyzed. Impact on representative datasets Amazon and MovieLens. Optimal values ​​for the Amazon and MovieLens datasets. Both are 10 -2 .when Less than 10 -3 At that point, performance began to decline, indicating that regularization can prevent ApeGNN from overfitting to some extent.

[0054] Adaptive operations can preserve the GNN properties and flexibility of ApeGNN. In fact, ApeGNN can be used as a plug-in for any GNN-based model used for personalized recommendations. Therefore, other advanced training techniques (e.g., skip-mix negative sampling strategies) can be directly applied to ApeGNN.

[0055] In some embodiments, ApeGNN can be implemented in PyTorch. For all models, the batch size can be set to 2048. ApeGNN can be optimized using a learning rate of 0.001 and the Adam optimizer. The propagation layers can be set from 1 to 4, the embedding dimension can be in the range {64, 128, 256, 512}, and the L2 normalized coefficients... It can be found in {10 -2 10 -3 10 -4 10 -5 10 -6 Within the range of}. Regarding hyperparameters, a grid search can be applied to find the optimal settings. For the Ali, Amazon, AMiner, Gowalla, MovieLens, and Yelp2018 datasets, the coefficients of L2 are... They can be 10 respectively -3 10 -2 10 -3 10 -3 10 -2 and 10 -3Furthermore, the Xavier initializer can be used to initialize model parameters. Additionally, during the training phase, if recall @20 does not increase for 10 consecutive epochs on the validation dataset, an early stopping strategy can be applied to halt training.

[0056] The following pseudocode illustrates the overall training process of ApeGNN.

[0057] Figure 4 A flowchart of a recommendation method 400 based on a graph neural network according to one aspect of this disclosure is shown. Method 400 can be the ApeGNN method as described above. Method 400 can be implemented on a cloud server or a client computing device, or a combination thereof. Method 400 can be implemented at least partially in the training or prediction phase.

[0058] In box 410, method 400 may include receiving a user-item bipartite graph that includes a set of user nodes representing a set of users, a set of item nodes representing a set of items, and a set of edges representing interactions between users and items. The terms "user" and "item" as used herein refer to their general meaning in recommender systems used in various application scenarios. In the example of a shopping platform, users may include customers, items may include goods for sale, and interactions may include one or more of the following between a customer and a product: purchasing, rating, clicking, adding to favorites, etc.

[0059] In box 420, method 400 may include obtaining the user embedding for each user in the user set and the item embedding for each item in the item set. During the training phase, this obtaining step may include initializing the embeddings for each user and each item using methods employed in existing GNN training approaches. During the prediction phase, this obtaining step may include obtaining the initial embeddings for each user and each item in layer 0 using the trained GNN model.

[0060] In box 430, method 400 may include an in-layer aggregation process to generate aggregated user embeddings for each user in the user set and aggregated item embeddings for each item in the item set, based on different weight coefficients for each user and each item (e.g., using equations (7) and (8) respectively). In one aspect, the weight coefficients are calculated using a hot kernel method according to equation (13). In another aspect, the weight coefficients are calculated using a personalized page ranking method according to equation (14). The weight coefficients may be a function of parameters of adaptive operations for the user or item, such as the centrality importance of each node. The weight coefficients may be calculated based on the in-degree of each user node or item node at the initial layer.

[0061] In box 440, method 400 may include performing an inter-layer propagation process to generate propagated user embeddings for each user in the user set and propagated item embeddings for each item in the item set based on different weight coefficients for each propagation layer (e.g., using equation (16)). Method 400 may also include combining the propagated user embeddings and propagated item embeddings for each propagation layer with various pooling operations to generate final embeddings for each user and each item.

[0062] In box 450, method 400 may include making recommendations for users in the user set based on propagated user embeddings and propagated item embeddings, such as by computing the inner product of the final embeddings between each user and each item. During the training phase, method 400 may include optimizing the graph neural network based on the final embeddings of each user and each item by using a Bayesian Personalized Ranking (BPR) loss, for example using equation (19).

[0063] Figure 5 A block diagram of a recommended apparatus 500 for a graph neural network based on one aspect of this disclosure is shown. Apparatus 500 may include components for performing the above-described reference. Figure 4 The described method 400 comprises multiple modules.

[0064] like Figure 5 As shown, device 500 may include an input module 510 for receiving a user-item bipartite graph including a set of user nodes representing a user set, a set of item nodes representing an item set, and a set of edges representing interactions between users and items. Device 500 may include an embedding module 520 for obtaining the user embedding for each user in the user set and the item embedding for each item in the item set. Device 500 may include an aggregation module 530 for performing an intra-layer aggregation process to generate aggregated user embeddings for each user in the user set and aggregated item embeddings for each item in the item set based on different weight coefficients for each user and each item. Device 500 may include a propagation module 540 for performing an inter-layer propagation process to generate propagated user embeddings for each user in the user set and propagated item embeddings for each item in the item set based on different weight coefficients for each propagation layer. Device 500 may include a recommendation module 550 for making recommendations for users in the user set based on the propagated user embeddings and propagated item embeddings. Although Figure 5 Not shown, but device 500 may also include a pooling module for combining the propagated user embeddings and propagated item embeddings of each propagation layer to generate a final embedding for each user and each item; and an optimization module for optimizing the graph neural network based on the final embeddings for each user and each item by using Bayesian Personalized Ranking (BPR) loss.

[0065] Figure 6 A block diagram of a recommendation apparatus 600 for a graph neural network-based recommendation according to one aspect of the present disclosure is shown. The recommendation apparatus 600 for a GNN-based recommendation may include a memory 610 and at least one processor 620. The processor 620 may be coupled to the memory 610 and configured to perform the functions described above. Figure 4 The method 400 is described. Processor 620 may be a general-purpose processor, or it may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Memory 610 may store input data, output data, data generated by processor 620, and / or instructions executed by processor 620.

[0066] The various operations, modules, models, and networks described in conjunction with this disclosure can be implemented using hardware, processor-executed software, firmware, or any combination thereof. According to embodiments of this disclosure, a computer program product for GNN-based recommendations may include tools for performing the above-described references. Figure 4 The described method 400 comprises processor-executable computer code. According to another embodiment of this disclosure, a computer-readable medium may store recommended computer code for GNN-based systems, which, when executed by a processor, enables the processor to perform the above-described references. Figure 4 Method 400 is described. A computer-readable medium includes both non-transitory computer media and communication media, wherein a communication medium includes any medium that facilitates the transfer of a computer program from one place to another. Any connection may be suitably referred to as a computer-readable medium. Other embodiments and implementations are within the scope of this disclosure.

[0067] In summary, the ApeGNN method presented in this disclosure introduces a graph diffusion process into GNN-based recommendation. ApeGNN addresses the problems of inappropriate neighborhood type recognition and the lack of emphasis on each node by extending the propagation of neighborhoods. ApeGNN can assign different weights to entities with different types and importance across different nodes, and assign different importance across multi-level neighborhoods. After experiments on public recommendation, ApeGNN demonstrates superiority over existing baseline models such as LightGCN, GAT, and ADC.

[0068] LightGCN is a representative graph convolutional network with a degree-normalized graph structure, but it ignores the unique semantics of each node. ApeGNN can consider explicitly distinguishing each node based on the importance of the center.

[0069] While attention-based models (such as GAT) perform local aggregation at each layer and consider node importance, they assign weights to local neighbors from a local perspective based on node features. In contrast, ApeGNN can adaptively aggregate neighbors from a global perspective using centrality, node-wise. Furthermore, by performing higher-order propagation in attention-based models, information from the previous layer is implicitly propagated to the next layer via non-linear activations. ApeGNN's adaptive aggregation can consider the influence of subgraph structure and explicitly propagate messages via layer coefficients. ApeGNN can combine node- and layer-specific weights to achieve adaptive aggregation, resulting in better performance compared to attention-based methods.

[0070] The ADC's diffusion model learns the size of its neighbors during propagation from each layer of the network for node classification. Specifically, the ADC learns the initial values ​​for each feature channel and each layer. t However, it overlooks the importance of nodes. In contrast, ApeGNN's adaptive aggregator can learn center awareness for each node. t It takes into account node importance. Experiments also demonstrate the effectiveness of center-based weights in ApeGNN.

[0071] A prior description of the disclosed embodiments is provided to enable those skilled in the art to make or use various embodiments. Various modifications to these embodiments will readily be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the scope of the various embodiments. Therefore, the claims are not intended to limit themselves to the embodiments shown herein, but are consistent with the widest scope consistent with the appended claims and the principles and novel features disclosed herein.

Claims

1. A method for recommendation based on graph neural networks, comprising: Receive a user-item bipartite graph, wherein the user-item bipartite graph includes a set of user nodes representing a set of users, a set of project nodes representing a set of projects, and a set of edges representing the interactions between the users and the projects; Obtain the user embedding for each user in the user set and the project embedding for each project in the project set; The execution layer aggregation process generates aggregated user embeddings for each user in the user set and aggregated project embeddings for each project in the project set, based on different weight coefficients for each user and each project. Perform an inter-layer propagation process to generate a propagated user embedding for each user in the user set and a propagated item embedding for each item in the item set, based on different weight coefficients for each propagation layer. as well as Recommendations are made for users in the user set based on the propagation user embedding and the propagation item embedding.

2. The method according to claim 1, wherein, The interactions include one or more of the following: purchase, rating, click, and favorite.

3. The method according to claim 1, wherein, The weighting coefficients are calculated using either the hot kernel method or a personalized page ranking method.

4. The method according to claim 1, wherein, The weighting coefficients are calculated based on the in-degree of each user node or project node at the initial layer.

5. The method according to claim 1, further comprising: The propagation user embedding and the propagation item embedding of each propagation layer are combined to generate the final embedding for each user and each item.

6. The method according to claim 5, further comprising: The graph neural network is optimized based on the final embedding for each user and each item by using Bayesian Personalized Ranking (BPR) loss.

7. An apparatus for recommendation based on graph neural networks, comprising: The input module is used to receive a user-item bipartite graph, which includes a set of user nodes representing a set of users, a set of project nodes representing a set of projects, and a set of edges representing the interaction between the user and the project. An embedding module is used to obtain the user embedding of each user in the user set and the project embedding of each project in the project set; The aggregation module is used to perform the intra-layer aggregation process to generate aggregated user embeddings for each user in the user set and aggregated project embeddings for each project in the project set, based on different weight coefficients for each user and each project. The propagation module is used to perform the inter-layer propagation process to generate the propagation user embedding for each user in the user set and the propagation item embedding for each item in the item set based on the different weight coefficients of each propagation layer. as well as The recommendation module is used to make recommendations for users in the user set based on the propagation user embedding and the propagation item embedding.

8. The apparatus according to claim 7, wherein, The interactions include one or more of the following: purchase, rating, click, and favorite.

9. The apparatus according to claim 7, wherein, The weighting coefficients are calculated using either the hot kernel method or a personalized page ranking method.

10. The apparatus according to claim 7, wherein, The weighting coefficients are calculated based on the in-degree of each user node or project node at the initial layer.

11. The apparatus according to claim 7, further comprising: A pooling module is used to combine the propagation user embedding and the propagation item embedding of each propagation layer to generate the final embedding for each user and each item.

12. The apparatus of claim 11, further comprising: An optimization module is used to optimize the graph neural network based on the final embedding for each user and each item by using Bayesian Personalized Ranking (BPR) loss.

13. An apparatus for recommendation based on a graph neural network, comprising: Memory; as well as At least one processor coupled to the memory and configured to perform the method according to any one of claims 1-6.

14. A computer-readable medium storing computer code for recommending based on graph neural networks, said computer code, when executed by a processor, causing the processor to perform the method according to any one of claims 1-6.

15. A computer program product for recommendation based on graph neural networks, comprising: A processor for performing the method of any one of claims 1-6 may execute computer code.

Images