Graph representation enhanced model training method, graph representation method and device

Abstract

Embodiments of the present specification provide a graph representation enhanced model training method, a graph representation method and device. In model training, a sparse graph is constructed by performing neighborhood pruning on true head nodes in an original long-tail graph. A graph representation enhanced model is used to perform graph representation enhancement on each node in the sparse graph to obtain an enhanced node representation of each node. The enhanced node representation of each node is learned based on an enhanced sparse graph. The enhanced sparse graph is obtained by enhancing neighborhood missing information for each node in the sparse graph. The neighborhood missing information for each node is predicted based on the center node representation and neighbor node representation of the node by using the graph representation enhanced model. Then, a neighborhood information prediction loss function is determined according to the enhanced node representation of the pseudo tail node and the node representation of the corresponding true head node in the original long-tail graph. And the model parameters of the graph representation enhanced model are adjusted based on the neighborhood information prediction loss function.

Description

Technical Field

[0001] The embodiments in this specification generally relate to the field of artificial intelligence, and in particular to graph representation enhancement model training methods, graph representation methods and apparatus. Background Technology

[0002] Graph Convolution Networks (GNNs) are widely used in various graph representation applications, such as recommender systems, because they effectively utilize high-order neighborhood relationships to learn accurate graph representations. However, these GNN-based graph representation methods do not consider the significant differences between head and tail nodes in long-tailed graphs, resulting in uneven distribution of graph node representations in long-tailed graph applications. Summary of the Invention

[0003] This specification provides a graph representation enhancement model training method, graph representation method, and apparatus. Using this graph representation enhancement model training method and apparatus, pseudo-tail nodes are generated by pruning the neighborhood of the true head nodes in the original long-tail graph. The pseudo-tail nodes and their corresponding true head nodes in the original long-tail graph are then used to train a graph representation enhancement model for predicting missing neighborhood information of nodes. This allows the trained graph representation enhancement model to predict missing neighborhood information of tail nodes based on the center node information and the neighbor node information of observable neighbor nodes, rather than solely on the center node information. This ensures that the predicted missing neighborhood information is not limited by the observable neighbor node information, thereby improving the accuracy of missing neighborhood information prediction and ultimately enhancing the representation enhancement effect of tail nodes.

[0004] According to one aspect of the embodiments of this specification, a method for training a graph representation enhancement model is provided, comprising: constructing a sparse graph based on an original long-tailed graph, the sparse graph including true tail nodes and pseudo tail nodes, the pseudo tail nodes being generated by performing neighborhood pruning on the true head nodes in the original long-tailed graph; using a graph representation enhancement model to perform graph representation enhancement on each node in the sparse graph to obtain enhanced node representations for each node, the enhanced node representations for each node being learned based on the enhanced sparse graph, the enhanced sparse graph being obtained by enhancing neighborhood missing information for each node in the sparse graph, the neighborhood missing information for each node being predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the node; determining a neighborhood information prediction loss function based on the enhanced node representations of the pseudo tail nodes and the node representations of the corresponding true head nodes in the original long-tailed graph; and adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function.

[0005] Optionally, in one example of the above aspects, the graph representation enhancement model includes a single-layer graph representation enhancement model. Using the graph representation enhancement model to enhance the graph representation of each node in the sparse graph may include: using the graph representation enhancement model to predict the neighborhood missing information of each node based on the initial center node representation and initial neighbor node representation of each node in the sparse graph, the predicted neighborhood missing information of each node serving as the initial node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph; and using a graph neural network model to perform graph learning based on the enhanced sparse graph to learn the enhanced node representation of each node.

[0006] Optionally, in one example of the above aspects, the graph representation enhancement model includes a multi-layer graph representation enhancement model. Using the graph representation enhancement model to enhance the graph representation of each node in the sparse graph may include: iteratively performing the following process until enhanced node representations of each node in the sparse graph are obtained: using the current graph representation enhancement model to predict the current neighborhood missing information of each node based on the current center node representation and the current neighbor node representation of each node in the sparse graph, the predicted neighborhood missing information of each node serving as the current node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph; and using a graph neural network model to perform graph learning based on the enhanced sparse graph to learn the current enhanced node representation of each node.

[0007] Optionally, in one example of the above aspects, the graph representation enhancement model training method may further include: using a head node generative adversarial network to perform a head node generative adversarial task, and determining a head node adversarial loss function based on the head node generative task execution result. Correspondingly, adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function may include: adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the head node adversarial loss function.

[0008] Optionally, in one example of the above aspects, the graph representation enhancement model training method may further include: performing a downstream task using the enhanced node representations of each node in the sparse graph and determining a downstream task loss function based on the downstream task execution results. Correspondingly, adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function may include: adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the downstream task loss function.

[0009] Optionally, in one example of the above aspects, the graph representation enhancement model training method may further include: performing a contrastive learning task based on the enhanced node representations of each node in the sparse graph and determining a contrastive learning task loss function based on the execution result of the contrastive learning task. Correspondingly, adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function may include: adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the contrastive learning task loss function.

[0010] Alternatively, in one example of the above aspects, constructing a sparse graph based on the original long-tailed graph may include: using an adaptive sampling model to construct a sparse graph based on the original long-tailed graph.

[0011] Optionally, in one example of the above aspects, the graph representation enhancement model training method may further include: using a tail node generative adversarial network to perform a tail node generative adversarial task, and determining a tail node adversarial loss function based on the tail node generative adversarial task execution result; and adjusting the model parameters of the adaptive sampling model based on the tail node adversarial loss function.

[0012] Alternatively, in one example of the above aspects, the original long-tail graph may include a user-item interaction graph.

[0013] According to another aspect of the embodiments of this specification, a graph representation method for a long-tailed graph is provided, comprising: providing a head node in the long-tailed graph to be represented to a graph neural network model for graph learning to learn a node representation of the head node; and using a graph representation enhancement model to perform graph representation enhancement on the tail nodes in the long-tailed graph to obtain node representations of the tail nodes, wherein the node representations of each tail node are learned based on the enhanced long-tailed graph, the enhanced long-tailed graph being obtained by enhancing neighborhood missing information for each tail node in the long-tailed graph to be represented, and the neighborhood missing information of each tail node being predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the tail node.

[0014] According to another aspect of the embodiments of this specification, an item recommendation method is provided, comprising: providing head nodes in a user-item interaction graph to a graph neural network model for graph learning to learn node representations of the head nodes, the head nodes including head user nodes and head item nodes; using a graph representation enhancement model to perform graph representation enhancement on tail nodes in the user-item interaction graph to obtain node representations of the tail nodes, the node representations of each tail node being learned based on an enhanced user-item interaction graph, the enhanced user-item interaction graph being obtained by enhancing neighborhood missing information for each tail node in the user-item interaction, the neighborhood missing information of each tail node being predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the tail node, the tail nodes including user tail nodes and item tail nodes; and making an item recommendation decision based on the node representations of target users and candidate items in the user-item interaction graph.

[0015] According to another aspect of the embodiments of this specification, a graph representation enhancement model training apparatus is provided, comprising: a sparse graph construction unit for constructing a sparse graph based on an original long-tailed graph, the sparse graph including true tail nodes and pseudo tail nodes, the pseudo tail nodes being generated by neighborhood pruning of the true head nodes in the original long-tailed graph; a graph representation enhancement unit for using a graph representation enhancement model to perform graph representation enhancement on each node in the sparse graph to obtain enhanced node representations for each node, the enhanced node representations for each node being learned based on the enhanced sparse graph, the enhanced sparse graph being obtained by enhancing neighborhood missing information for each node in the sparse graph, the neighborhood missing information for each node being predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the node; a first loss function determination unit for determining a neighborhood information prediction loss function based on the enhanced node representations of the pseudo tail nodes and the node representations of the corresponding true head nodes in the original long-tailed graph; and a first model adjustment unit for adjusting the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function.

[0016] Optionally, in one example of the above aspects, the graph representation enhancement model includes a single-layer graph representation enhancement model. The graph representation enhancement unit includes: a neighborhood missing information prediction module, which uses the graph representation enhancement model to predict the neighborhood missing information of each node based on the initial center node representation and initial neighbor node representation of each node in the sparse graph, and the predicted neighborhood missing information of each node serves as the initial node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph; and a graph learning module, which uses a graph neural network model to perform graph learning based on the enhanced sparse graph to learn the enhanced node representation of each node.

[0017] Optionally, in one example of the above aspects, the graph representation enhancement model includes a multi-layer graph representation enhancement model. The graph representation enhancement unit includes: a neighborhood missing information prediction module, which uses the current graph representation enhancement model to predict the current neighborhood missing information of each node based on the current center node representation and the current neighbor node representation of each node in the sparse graph, wherein the predicted neighborhood missing information of each node is used as the current node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph; and a graph learning module, which uses a graph neural network model to perform graph learning based on the enhanced sparse graph to learn the current enhanced node representation of each node.

[0018] Optionally, in one example of the above aspects, the graph representation model training device may further include a head node generation adversarial task execution unit, which uses a head node generation adversarial network to perform a head node generation adversarial task. Accordingly, the first loss function determination unit further determines a head node adversarial loss function based on the head node generation adversarial task execution result, and the first model parameter adjustment unit adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the head node adversarial loss function.

[0019] Optionally, in one example of the above aspects, the graph representation enhancement model training apparatus may further include: a downstream task execution unit that performs a downstream task using the enhanced node representations of each node in the sparse graph. The first loss function determination unit further determines a downstream task loss function based on the downstream task execution results, and the first model parameter adjustment unit adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the downstream task loss function.

[0020] Optionally, in one example of the above aspects, the graph representation enhancement model training apparatus may further include: a contrastive learning task execution unit, which performs a contrastive learning task based on the enhanced node representations of each node in the sparse graph. The first loss function determination unit further determines a contrastive learning task loss function based on the execution result of the contrastive learning task, and the first model parameter adjustment unit adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the contrastive learning task loss function.

[0021] Alternatively, in one example of the above aspects, the sparse graph building unit uses an adaptive sampling model to build a sparse graph based on the original long-tailed graph.

[0022] Optionally, in one example of the above aspects, the graph representation enhancement model training device may further include: a tail node generation adversarial task execution unit, which uses a tail node generation adversarial network to execute a tail node generation adversarial task; a second loss function determination unit, which determines a tail node adversarial loss function based on the tail node generation adversarial task execution result; and a second model parameter adjustment unit, which adjusts the model parameters of the adaptive sampling model based on the tail node adversarial loss function.

[0023] According to another aspect of the embodiments of this specification, a graph representation apparatus for a long-tailed graph is provided, comprising: a head node representation unit, which provides a head node in the long-tailed graph to be represented to a graph neural network model for graph learning to learn the node representation of the head node; and a tail node representation unit, which uses a graph representation enhancement model to perform graph representation enhancement on the tail nodes in the long-tailed graph to obtain the node representation of the tail nodes, wherein the node representation of each tail node is learned based on the enhanced long-tailed graph, the enhanced long-tailed graph is obtained by enhancing the neighborhood missing information of each tail node in the long-tailed graph to be represented, and the neighborhood missing information of each tail node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representation of the tail node.

[0024] According to another aspect of the embodiments of this specification, a graph representation enhancement model training apparatus is provided, comprising: at least one processor; a memory coupled to the at least one processor; and a computer program stored in the memory, wherein the at least one processor executes the computer program to implement the graph representation enhancement model training method as described above.

[0025] According to another aspect of the embodiments of this specification, a graph representation apparatus for long-tailed graphs is provided, comprising: at least one processor; a memory coupled to the at least one processor; and a computer program stored in the memory, wherein the at least one processor executes the computer program to implement the graph representation method for long-tailed graphs as described above. Attached Figure Description

[0026] A further understanding of the nature and advantages of this specification can be achieved by referring to the following figures. In the figures, similar components or features may have the same reference numerals.

[0027] Figure 1A and Figure 1B This diagram illustrates example illustrations of head and tail nodes in a user-item interaction graph.

[0028] Figure 2 A schematic diagram of an example scheme for enhancing tail node structural information is shown.

[0029] Figure 3 An example schematic diagram of the application architecture of a long-tail graph representation enhancement scheme according to an embodiment of this specification is shown.

[0030] Figure 4 An example flowchart of a graph representation enhancement model training method according to an embodiment of this specification is shown.

[0031] Figure 5 An example flowchart of a sparse graph construction process according to an embodiment of this specification is shown.

[0032] Figure 6 An example schematic diagram of a sparse graph construction process according to an embodiment of this specification is shown.

[0033] Figure 7A and Figure 7B An example architecture diagram of a graph representation enhancement model according to an embodiment of this specification is shown.

[0034] Figure 8 An example flowchart of a graph representation enhancement process according to an embodiment of this specification is shown.

[0035] Figure 9 An example schematic diagram of a graph representation enhancement process according to an embodiment of this specification is shown.

[0036] Figure 10 Another example flowchart of a graph representation enhancement process according to an embodiment of this specification is shown.

[0037] Figure 11 Another example schematic diagram of a graph representation enhancement process according to an embodiment of this specification is shown.

[0038] Figure 12 An example schematic diagram of the training process of a graph representation enhancement model according to an embodiment of this specification is shown.

[0039] Figure 13 An example flowchart of a graph representation method for long-tailed graphs according to an embodiment of this specification is shown.

[0040] Figure 14 An example flowchart of an item recommendation method according to an embodiment of this specification is shown.

[0041] Figure 15 An example block diagram of a graph representation enhancement model training apparatus according to an embodiment of this specification is shown.

[0042] Figure 16 An example block diagram of a graph representation enhancement unit according to an embodiment of this specification is shown.

[0043] Figure 17An example block diagram of a graphical representation device according to an embodiment of this specification is shown.

[0044] Figure 18 An example block diagram of an article recommendation device according to an embodiment of this specification is shown.

[0045] Figure 19 An example schematic diagram of a computer-based graph representation enhancement model training apparatus according to an embodiment of this specification is shown.

[0046] Figure 20 An example schematic diagram of a graph representation device implemented using a computer system according to an embodiment of this specification is shown.

[0047] Figure 21 An example schematic diagram of a computer-based item recommendation device according to an embodiment of this specification is shown. Detailed Implementation

[0048] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0049] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0050] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0051] Recommendation systems are widely used in various online service scenarios for item recommendation, such as e-commerce services and online advertising. In this specification, the term "item" refers to any object that can be provided or recommended to the user, such as goods, advertisements, images, mini-programs, application services, and content such as long / short audio / video recordings and movies. When making item recommendations, user and item representations can be represented based on observable user-item interaction information (e.g., clicks, conversions). Item recommendation predictions are then made based on these representations, and item recommendations are made based on the prediction results.

[0052] Recommendation schemes based on graph neural networks represent user-item interaction data as user-item interaction graphs and iteratively propagate neighborhood information (e.g., higher-order neighborhood information) to efficiently learn high-accuracy user and item representations, thereby improving the accuracy of item recommendations.

[0053] However, this recommendation scheme based on graph neural networks relies on rich graph structure information of nodes when learning graph representations. In the real world, graph structures exhibit a power-law distribution (i.e., a long-tail distribution) in node degree, meaning that a small number of nodes have many neighbors and thus rich neighborhood information, while most nodes have few neighbors and correspondingly less neighborhood information. Nodes with many neighbors are called head nodes, while nodes with few neighbors are called tail nodes. Figure 1A and Figure 1B The diagram illustrates an example of head and tail nodes in a user-item interaction graph. A graph structure with a long-tailed distribution can be called a long-tailed graph. This structural information difference between head and tail nodes in a long-tailed graph results in tail nodes having a poorer graph node representation effect compared to head nodes.

[0054] Therefore, embodiments of this specification propose a long-tail graph representation scheme that considers the long-tail distribution in long-tail graphs. In this scheme, a graph representation enhancement model is modeled to predict the missing neighborhood information of tail nodes, and pseudo-tail nodes obtained by pruning the neighborhood information of the head nodes of the long-tail graph are used as training samples to train the graph representation enhancement model. Subsequently, the trained graph representation enhancement model is used to predict the missing neighborhood information of tail nodes, and the predicted missing neighborhood information is used as the missing neighborhood structure information of the tail nodes to enhance the graph structure information of the long-tail graph. Figure 2A schematic diagram illustrating an example of a tail node structural information enhancement scheme is shown. After the graph structural information enhancement described above, graph representation learning is performed on the enhanced long-tail graph. This alleviates the structural information discrepancies between the head and tail nodes in the long-tail graph structure, enhancing the graph node representation effect of the tail nodes. Furthermore, since the trained graph representation enhancement model can be based on the center node information of the tail nodes and the neighborhood information of the observable neighbor nodes, the predicted missing neighborhood information is not limited by the observable neighbor node information, thereby improving the prediction accuracy of missing neighborhood information and further enhancing the tail node representation enhancement effect.

[0055] The following will describe in detail, with reference to the accompanying drawings, the graph representation enhancement model training method, the graph representation method, the item recommendation method, and the apparatus according to embodiments of this specification.

[0056] Figure 3 An example schematic diagram of an application architecture 300 for a long-tail graph representation enhancement scheme according to an embodiment of this specification is shown.

[0057] like Figure 3 As shown, the application architecture 300 includes a graph representation enhancement model training device 310, a graph representation enhancement model storage device 320, a graph representation device 330, and an item recommendation device 340. The graph representation enhancement model training device 310, graph representation enhancement model storage device 320, graph representation device 330, and item recommendation device 340 can communicate with each other via a network 350 to transmit data. In some embodiments, the network 350 can be any one or more of a wired network or a wireless network. Examples of the network 350 may include, but are not limited to, cable networks, fiber optic networks, telecommunications networks, enterprise intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), Bluetooth networks, ZigZee networks, near field communication (NFC), device internal buses, device internal lines, etc., or any combination thereof. In some embodiments, some or all of the components in the graph representation enhancement model training device 310, graph representation enhancement model storage device 320, graph representation device 330, and item recommendation device 340 can communicate directly without the network 350.

[0058] The graph representation enhancement model training device 310 trains a graph representation enhancement model based on long-tailed graph data to predict the missing neighborhood information of tail nodes in the long-tailed graph. The trained graph representation enhancement model is stored in the graph representation enhancement model storage device 320 for use by downstream devices, such as the graph representation device 330 for graph node representation, or the item recommendation device 340 for item recommendation. It should be noted that downstream devices may also include other types of graph representation application devices.

[0059] Graph representation device 330 uses a graph representation enhancement model to predict the missing neighborhood information of tail nodes in the long-tailed graph, and enhances the graph node representation of the tail nodes based on the predicted missing neighborhood information, thereby improving the representation effect of tail nodes in the long-tailed graph. Item recommendation device 340 uses a graph representation enhancement model to enhance the graph node representation of tail nodes in the user-item interaction graph, and makes item recommendations based on the enhanced node representations of users and items.

[0060] Optionally, in some embodiments, the graph representation device 330 and / or the item recommendation device 340 may have storage capabilities, thereby allowing the graph representation enhancement model trained therefrom to be stored locally on the graph representation device 330 and / or the item recommendation device 340, thus eliminating the need for the graph representation enhancement model storage device 320.

[0061] Figure 4 An example flowchart of a graph representation enhancement model training method 400 according to an embodiment of this specification is shown.

[0062] like Figure 4 As shown in 410, a sparse graph is constructed based on the original long-tail graph. The constructed sparse graph includes true tail nodes and pseudo tail nodes, and the pseudo tail nodes are generated by performing neighborhood pruning on the true head nodes in the original long-tail graph.

[0063] The original long-tailed graph is the training graph data used for training graph representation enhancement models, and it is generated based on real-world application data from various application scenarios. This real-world application data can be modeled as binary graph data. in, Let A represent the set of nodes and A represent the set of edges. Edges in a graph structure represent relationships between nodes. If a predetermined relationship exists between two nodes, then an edge exists between them, and this edge relationship can be measured as 1 or another metric. If no predetermined relationship exists between two nodes, then no edge exists between them, and this edge relationship can also be measured as 0. Following this approach, A can be represented as an adjacency matrix.

[0064] For example, in the context of item recommendation, a long-tail graph can be a user-item interaction graph generated based on user-item interaction data. This user-item interaction graph can be modeled as graph data. in, Let I represent the user set and I represent the item set. Observable user-item interaction data is represented as... Where there is interaction between user u and item i, then the corresponding element r ui =1, if there is no interaction between user u and item i, then the corresponding element r = 1.ui =0, |I| represents the number of users in the user set, |I| represents the number of items in the item set, and |I| represents the number of nodes. In this case, the adjacency matrix It can be expressed as shown in formula (1):

[0065]

[0066] The original long-tail graph has true head nodes (HN) and true tail nodes (TN). In this specification, the term "true node" refers to a node generated based on real data, while nodes obtained through pruning or augmentation are called "pseudo nodes." For example, a node degree threshold K can be set. For a node in the original long-tail graph, if the node's degree > K, then the node is a head node; if the node's degree ≤ K, then the node is a tail node. Here, the node degree refers to the number of the node's neighboring nodes.

[0067] In some embodiments, pseudo-tail nodes can be generated by randomly sampling the neighboring nodes of the head node in the original long-tail graph. For example, for each head node in the original long-tail graph, no more than K neighboring nodes can be randomly sampled from all the neighboring nodes of the head node, and the unsampled neighboring nodes can be pruned, thereby generating the corresponding pseudo-tail node. Alternatively, neighboring nodes that need pruning can be randomly sampled from all the neighboring nodes of the head node and pruned (the number of remaining neighboring nodes after pruning is no more than K), thereby generating the corresponding pseudo-tail node.

[0068] In some embodiments, an adaptive sampling model can be used to construct a sparse graph based on the original long-tailed graph. For example, for each head node in the original long-tailed graph, the adaptive sampling model can be used to prune neighborhood information based on the edge weights of neighboring nodes, thereby generating corresponding pseudo-tailed nodes. The adaptive sampling model has a trainable sampling mechanism that minimizes the distribution deviation between the pseudo-tailed nodes in the constructed sparse graph and the true tailed nodes in the original long-tailed graph.

[0069] Figure 5 An example flowchart of a sparse graph construction process 500 according to an embodiment of this specification is shown. It should be noted that... Figure 5 The processing flow shown is executed by an adaptive sampling model.

[0070] like Figure 5As shown in Figure 510, the node degree of the pseudo-tail node corresponding to each head node in the original long-tail graph is determined. For example, assuming that the node degree threshold used to distinguish between head nodes and tail nodes is K, then for all head nodes, the node degree k of the pseudo-tail node corresponding to each head node i can be randomly sampled from the numerical range [1, K]. i That is, the number of neighboring nodes remaining after pruning, and to make the node degree of each head node after random sampling uniformly distributed.

[0071] In step 520, the neighboring nodes of each head node are sorted based on the edge weights of each head node.

[0072] For example, the edge weight matrix S between node pairs in the original long-tailed graph can be calculated according to formula (2):

[0073] S=(H (0) W s H (0)T A (2)

[0074] in, H represents the pairwise edge weight matrix (i.e., the pairwise correlation matrix) between nodes in the original long-tailed graph. (0) W represents the node feature matrix of the original long-tailed graph. s is the trainable parameters of the adaptive sampling model, and A is the adjacency matrix of the original long-tailed graph.

[0075] In step 530, based on the degree of the pseudo-tail nodes corresponding to each head node and the sorting results of the neighbor nodes of each head node, neighborhood pruning is performed on each head node to construct a sparse graph.

[0076] After performing neighborhood pruning and constructing a sparse graph as described above, the adjacency matrix of the resulting sparse graph is... This can be expressed as shown in formula (3):

[0077]

[0078] Where δ is used to control the adjacency matrix. The hyperparameter of smoothness.

[0079] Figure 6 An example schematic diagram of a sparse graph construction process according to an embodiment of this specification is shown. Figure 6As shown, for the original long-tail graph, the edge weights of the neighboring nodes of each node are calculated. Then, for each head node, neighborhood pruning is performed based on the calculated edge weights of its neighboring nodes and the degree of the selected pseudo-tail node, thereby constructing the corresponding pseudo-tail node. For example, the neighboring nodes corresponding to edge weights 0.2 and 0.3 of the head node HN shown in the figure can be pruned, thus constructing a pseudo-tail node PTN with two neighboring nodes.

[0080] Optionally, a tail node generative adversarial network (also called a tail node generative adversarial network model) can be set up, which includes a first generator and a first discriminator. The goal of the first generator is to classify both pseudo-tail nodes and true tail nodes in the original long-tail graph as tail nodes based on their node representations. The goal of the first discriminator is to distinguish between pseudo-tail nodes and true tail nodes based on their node representations in the original long-tail graph. Here, pseudo-tail nodes are obtained by pruning the neighborhood of the true head nodes in the original long-tail graph.

[0081] In this context, a tail node generative adversarial network (GAN) can be used to perform the tail node generative adversarial task. Specifically, a first generator and a first discriminator are used to perform node classification prediction based on the node representations of pseudo-tail nodes in the sparse graph (i.e., node representations obtained without domain information augmentation) and the node representations of true tail nodes in the original long-tail graph, respectively. The tail node adversarial loss function is then determined based on the results of the tail node generative adversarial task. Subsequently, based on the tail node adversarial loss function Adjust the model parameters of the adaptive sampling model. It should be noted that the first generator can be implemented as the output layer of a graph neural network model, and the first generator and the first discriminator are trained separately, that is, trained based on independent training processes.

[0082] For example, the tail node adversarial loss function can be determined according to formula (4).

[0083]

[0084] Where CROSSENT(·) represents the cross-entropy function, 1(·) represents the indicator function, and f d (·;θ d ) is based on θ d The discriminator function with parameters.

[0085] The discriminator function is used to determine the probability that a node is a tail node according to formula (5):

[0086]

[0087] Where LeakyReLU(·) is the activation function, and σ(·) is the sigmoid function. It is the discriminator function f d (·;θ d The trainable parameters of ).

[0088] After training the adaptive sampling model for multiple rounds as described above, the pseudo-tail nodes sampled by the trained adaptive sampling model can be made to resemble the true tail nodes in the original long-tail graph.

[0089] Back Figure 4 After constructing the sparse graph as described above, in step 420, a graph representation enhancement model is used to enhance the graph representation of each node in the sparse graph to obtain the enhanced node representation of each node. The enhanced node representation of each node is learned based on the enhanced sparse graph. The enhanced sparse graph is obtained by enhancing the neighborhood missing information of each node in the sparse graph, and the neighborhood missing information of each node is predicted by the graph representation enhancement model based on the central node representation and neighbor node representation of the node.

[0090] Figure 7A and Figure 7B An example architecture diagram of a graph representation enhancement model according to an embodiment of this specification is shown. Figure 7A In the example, the graph representation enhancement model is implemented as a single-layer graph representation enhancement model structure, that is, it includes a single-layer graph representation enhancement model, and in Figure 7B In the examples, the graph representation enhancement model is implemented as a multi-layer graph representation enhancement model structure, for example, including an L-layer graph representation enhancement model. Each graph representation enhancement model can be implemented, for example, as a knowledge transfer model with a multi-layer perceptron model structure, i.e., a multi-layer perceptron model.

[0091] Figure 8 An example flowchart of a graph representation enhancement process 800 according to an embodiment of this specification is shown. Figure 8 In the example, the graph representation enhancement model is implemented as a single-layer graph representation enhancement model.

[0092] like Figure 8 As shown in 810, a graph representation enhancement model is used to predict the neighborhood missing information of each node based on the initial center node representation and the initial neighbor node representation of each node in the sparse graph.

[0093] For example, the missing neighborhood information m of node i in a sparse graph can be predicted according to formula (6). i :

[0094]

[0095] Among them, f t (·) denotes the perceptron function. Let i represent the initial node representation (i.e., the initial center node representation). Let represent the set of neighbor nodes of node i in the sparse graph (the set of observable neighbor nodes in the sparse graph). Let θ represent the initial neighbor node representations of each neighbor node of node i (e.g., it can be represented as the average representation of the initial neighbor node representations of all neighbor nodes of node i), and θ t The graph represents the trainable model parameters of the augmented model.

[0096] After predicting the missing neighborhood information for each node as described above, the predicted missing neighborhood information can be used to augment the sparse graph, thereby generating an augmented sparse graph. For example, a pseudo-neighbor node can be augmented for each node in the sparse graph, and the predicted missing neighborhood information for each node can be used as the initial node representation of the augmented pseudo-neighbor node, thereby generating an augmented sparse graph.

[0097] In section 820, a graph neural network model is used to perform graph learning based on an augmented sparse graph, in order to learn augmented node representations for each node. For example, the neighbor information of each node can be aggregated to that node according to various neighborhood information aggregation schemes in this field, thereby completing the graph representation learning based on the augmented sparse graph. For example, the aggregation method described below for the multilayer perceptron model can be used for neighbor information aggregation.

[0098] Figure 9 An example schematic diagram of a graph representation enhancement process according to an embodiment of this specification is shown.

[0099] like Figure 9 As shown, for each node (pseudo-tail node PTN and true tail node TN) in the sparse graph, a graph representation enhancement model is used to predict missing neighborhood information. The predicted missing neighborhood information is then added to the sparse graph as pseudo-neighbor nodes, thereby generating a pseudo-head node PHN, and thus constructing an enhanced sparse graph. The enhanced sparse graph is then fed to a graph neural network model for graph learning to learn the enhanced node representations for each node in the sparse graph.

[0100] Figure 10 Another example flowchart of a graph representation enhancement process 1000 according to an embodiment of this specification is shown. Figure 10 In the example, the graph representation enhancement model is implemented as a multi-layer graph representation enhancement model.

[0101] like Figure 10As shown, operations 1000 to 1030 are performed cyclically from the first layer graph representation enhancement model until the enhanced node representations of each node in the sparse graph are obtained.

[0102] Specifically, in 1010, the current graph representation enhancement model is used to predict the current neighborhood missing information of each node based on the current center node representation and the current neighbor node representation of each node in the sparse graph. The predicted neighborhood missing information of each node is used as the current node representation of the pseudo neighbor nodes of each node in the sparse graph.

[0103] For example, the missing neighborhood information of node i in a sparse graph can be predicted according to formula (7).

[0104]

[0105] Among them, f t (·) denotes the perceptron function. This represents the node representation of node i output at the l-th graph convolutional layer of the graph neural network model. Let represent the set of neighbor nodes of node i in the sparse graph (the set of observable neighbor nodes in the sparse graph). Let represent the neighbor node representations output by each neighbor node of node i on the l-th graph convolutional layer of the graph neural network model (e.g., it can be represented as the pooled average representation output by all neighbor nodes of node i on the l-th convolutional layer of the graph neural network model), and This represents the trainable model parameters of the graph representation enhancement model at layer l.

[0106] After predicting the missing neighborhood information of each node as described above, the predicted missing neighborhood information of each node can be used as the current node representation of the enhanced pseudo-neighbor nodes of each node in the enhanced sparse graph.

[0107] In 1020, a graph neural network model is used to perform graph learning based on the augmented sparse graph to learn the current augmented node representation for each node. Similarly, the neighbor information of each node can be aggregated to that node according to various neighborhood information aggregation schemes in this field, thereby completing the graph representation learning based on the augmented sparse graph.

[0108] For example, neighbor information on the (l+1)th layer can be aggregated according to formula (8):

[0109]

[0110] At 1030, check if the current graph representation enhancement model is the last layer of graph representation enhancement model. If it is, the process ends, and the current enhanced node representation of each node is output as the final node representation of each node. If it is not the last layer of graph representation enhancement model, return to 1010 and re-execute the next loop process.

[0111] Figure 11 Another example schematic diagram of a graph representation enhancement process according to an embodiment of this specification is shown.

[0112] like Figure 11 As shown, for each node (pseudo-tail node PTN and true tail node TN) in the sparse graph, in each iteration, the current graph representation enhancement model is used to predict missing neighborhood information, and the predicted missing neighborhood information is added to the sparse graph as pseudo-neighbor nodes, thereby generating pseudo-head nodes and constructing an enhanced sparse graph. Then, the enhanced sparse graph is provided to the current convolutional layer of the graph neural network model for graph learning to learn the current enhanced node representation of each node in the sparse graph. After obtaining the current enhanced node representation of each node, if all graph representation enhancement models have completed node enhancement processing, the current enhanced node representation of each node is output as the final node representation of each node. If there are still graph representation enhancement models that have not completed node enhancement processing, the next graph representation enhancement model is used to re-predict and re-learn the graph, and this cycle continues until the end of the iteration.

[0113] Back Figure 4 After obtaining the enhanced node representations of each tail node in the sparse graph as described above, at 430, the neighborhood information prediction loss function is determined based on the enhanced node representations of the pseudo-tail nodes in the sparse graph and the node representations of the corresponding true head nodes in the original long-tail graph.

[0114] For example, the neighborhood information prediction loss function can be determined according to formulas (9) and (10).

[0115]

[0116]

[0117] in, And L is the number of graph convolutional layers in the graph neural network model.

[0118] In step 440, the model parameters of the graph representation enhancement model are adjusted based on the neighborhood information prediction loss function. For example, when the graph representation enhancement model is a multi-layer model, backpropagation can be used to adjust the model parameters layer by layer.

[0119] In some embodiments, during graph representation model training, a head node generative adversarial network (also called a head node generative adversarial network model) can be set up. This head node generative adversarial network includes a second generator and a second discriminator. The goal of the second generator is to classify both pseudo-head nodes (i.e., enhanced node representations generated after enhancing each tail node in the sparse graph) and the true head nodes in the original long-tailed graph as head nodes. The goal of the second discriminator is to distinguish between pseudo-head nodes and true head nodes based on the pseudo-head node representations (i.e., enhanced node representations generated after enhancing each tail node in the sparse graph) and the true head nodes in the original long-tailed graph. Here, pseudo-head nodes are obtained by enhancing the pseudo tail nodes and true tail nodes in the sparse graph (i.e., supplementing missing neighborhood information).

[0120] In this context, a head node generative adversarial network can be used to perform the head node generative adversarial task. Specifically, a second generator and a second discriminator are used to perform node classification prediction based on the enhanced node representations of each node in the sparse graph and the node representations of the head nodes in the original long-tail graph, and the head node adversarial loss function is determined based on the execution results of the head node generative adversarial task.

[0121] For example, the adversarial loss function of the head node can be determined according to formula (11).

[0122]

[0123] Where CROSSENT(·) represents the cross-entropy function, 1(·) represents the indicator function, and f d (·;θ d ) is based on θ d The discriminator function with parameters.

[0124] The discriminator function is used to determine the probability that a node is a head node according to formula (12):

[0125]

[0126] Where LeakyReLU(·) is the activation function, and σ(·) is the sigmoid function. It is the discriminator function f d (·;θ d The trainable parameters of ).

[0127] In this scenario, the model parameters of the graph representation augmentation model can be adjusted based on the neighborhood information prediction loss function and the head node adversarial loss function. For example, the neighborhood information prediction loss function and the head node adversarial loss function can be weighted and combined to obtain the total loss function. Subsequently, the model parameters of the graph representation augmentation model can be adjusted based on the total loss function. After training the graph representation augmentation model multiple times in this manner, the pseudo-head nodes augmented by the trained graph representation augmentation model can be made to resemble the true head nodes in the original long-tailed graph more closely.

[0128] Alternatively, the trainable parameters of the discriminator function of the second discriminator can be adjusted based on the head node adversarial loss function or a weighted combination of the neighborhood information prediction loss function and the head node adversarial loss function. Using the above training task, the prediction accuracy of missing neighborhood information can be further improved. It should be noted that the second generator can be implemented as the output layer of a graph neural network model, and the second generator and the second discriminator are trained independently, i.e., trained through independent training processes.

[0129] In some embodiments, during the training of the graph representation enhancement model, the enhanced node representations of each node in the sparse graph can be used to perform downstream tasks, and the loss function of the downstream task can be determined based on the execution results of the downstream task. Downstream task loss function The method for determining the loss function can be tailored to different downstream tasks, following the corresponding loss function determination method. In this case, the model parameters of the graph representation enhancement model can be adjusted based on the predicted loss function and the downstream task loss function.

[0130] In some embodiments, during the training of the graph representation enhancement model, a contrastive learning task can be performed based on the enhanced node representations of each node in the sparse graph, and the loss function of the contrastive learning task can be determined based on the results of the contrastive learning execution. In this case, a loss function based on neighborhood information and a loss function for the contrastive learning task can be used. Adjust the model parameters of the graph representation to enhance the model.

[0131] For example, after obtaining the enhanced node representation for each node based on the graph representation enhancement model, two views can be generated for contrastive learning using a feature perturbation method. Here, feature perturbation can be achieved by adding random noise to the node representations. After generating the two views, a contrastive learning task can be performed, making the node representations of the same node in different views closer together, and the node representations of different nodes farther apart. Since the node representations are generated based on the graph representation enhancement model, there is no need to worry that the graph representation enhancement effect of the tail nodes will deteriorate due to feature perturbation. The training sample construction method and loss function for the contrastive learning task are also discussed. The calculation method can be implemented in any suitable manner in the field, and will not be described here.

[0132] In some embodiments, neighborhood information prediction can be combined with one or more of the head node generation adversarial training task, downstream task, and contrastive learning task to perform graph representation enhancement model training. For example, when neighborhood information prediction is combined with the head node generation adversarial training task, downstream task, and contrastive learning task, the total loss function can be determined based on Equation (13).

[0133]

[0134] Where λ1, λ2, and λ3 are hyperparameters, used to control the head node adversarial task loss function, respectively. Downstream task loss function and contrastive learning task loss function Contribution to the total loss function.

[0135] Then, based on the total loss function This allows for the adjustment of model parameters in the graph representation augmentation model. Furthermore, when training involves using other models with trainable parameters, model parameter adjustments can also be made synchronously with the graph representation augmentation model based on the total loss function.

[0136] Figure 12 An example schematic diagram of the training process of a graph representation enhancement model according to an embodiment of this specification is shown.

[0137] like Figure 12 As shown, the original long-tailed graph is adaptively sampled using an adaptive sampling model to generate a sparse graph. After generating the sparse graph, a tail node generative adversarial task can be performed based on the tail node generative adversarial network, and the tail node adversarial loss function can be determined based on the execution result of the tail node generative adversarial task. The generated tail node adversarial loss function Model parameters used to adjust the adaptive sampling model.

[0138] Subsequently, the generated sparse graph is fed into a graph representation enhancement model to predict missing neighborhood information. The predicted missing neighborhood information is then used as pseudo-neighbor nodes to supplement the sparse graph, thus obtaining the enhanced sparse graph. The resulting enhanced sparse graph can be fed into a graph convolutional network for graph learning, thereby learning the enhanced node representations of each node in the sparse graph. The loss function for predicting missing neighborhood information is then determined based on the enhanced node representations of the pseudo-tail nodes and the corresponding true head nodes in the original long-tail graph.

[0139] The enhanced node representations of each node in the learned sparse graph can be provided to the downstream recommendation module to perform recommendation tasks, and the recommendation loss function can be determined based on the results of the recommendation task.

[0140] Furthermore, two contrastive views can be constructed based on the enhanced node representations of each generated node for the contrastive learning task, and these two constructed contrastive views can be used to perform the contrastive learning task. After obtaining the execution results of the contrastive learning task, the contrastive learning loss function is determined based on these results.

[0141] Furthermore, after obtaining the enhanced sparse graph, a head node generative adversarial task can be performed based on the head node generative adversarial network, and the head node adversarial loss function can be determined based on the execution result of the head node generative adversarial task.

[0142] The neighborhood missing information prediction loss function is obtained as described above. Recommended loss function Contrast learning loss function Adversarial loss function against head nodes Then, the loss function can be predicted based on the missing neighborhood information. Recommended loss function Contrast learning loss function Adversarial loss function against head nodes Determine the total loss function and adjust the model parameters of the graph representation enhancement model based on the total loss function.

[0143] The training process of a graph representation enhancement model according to an embodiment of this specification has been described above with reference to the accompanying drawings.

[0144] By employing the aforementioned graph representation enhancement model training scheme, pseudo-tail nodes are generated by pruning the neighborhood of the true head nodes in the original long-tail graph. A graph representation enhancement model for predicting missing neighborhood information of nodes is then trained using the pseudo-tail nodes and their corresponding true head nodes in the original long-tail graph. This allows the trained graph representation enhancement model to predict missing neighborhood information of tail nodes based on the center node information and the neighbor node information of observable neighbor nodes, rather than solely on the center node information. This ensures that the predicted missing neighborhood information is not limited by the observable neighbor node information, thereby improving the accuracy of missing neighborhood information prediction and ultimately enhancing the representation enhancement effect of tail nodes.

[0145] By using the above-mentioned graph representation enhancement model training scheme, and by using a head node generative adversarial network to perform a head node generative adversarial task, and applying the head node adversarial loss function generated by the head node generative adversarial task to adjust the model parameters of the graph representation enhancement model, the prediction accuracy of neighborhood missing information of the graph representation enhancement model can be further improved.

[0146] By using the above graph representation enhancement model training scheme, compared with the neighborhood pruning scheme based on random sampling, the adaptive sampling model is used to select the neighbor nodes to be pruned, thereby overcoming the problem of biased sampling data caused by the distribution difference between the head and tail nodes in the original long-tail graph.

[0147] By using the above-mentioned graph representation enhancement model training method, the problem of data bias caused by the absence of true tail nodes in the training of the graph representation enhancement model can be overcome by introducing true tail nodes into downstream tasks, contrastive learning tasks, and tail node generation adversarial training tasks, and applying the loss function generated by the above tasks to the model training of the graph representation enhancement model.

[0148] After training the graph representation enhancement model as described above, the trained graph representation enhancement model can be applied to the graph representation process and the item recommendation process.

[0149] Figure 13 An example flowchart of a graph representation method 1300 for long-tailed graphs according to an embodiment of this specification is shown.

[0150] like Figure 13 As shown in 1310, the head node in the long-tail graph to be represented is provided to the graph neural network model for graph learning in order to learn the node representation of the head node.

[0151] In step 1320, a graph representation enhancement model is used to enhance the graph representation of the tail nodes in the long-tailed graph to be represented, thereby obtaining the node representations of the tail nodes. The node representations of each tail node are learned based on the enhanced long-tailed graph, which is obtained by enhancing the missing neighborhood information of each tail node in the long-tailed graph to be represented. Furthermore, the missing neighborhood information of each tail node is predicted by the graph representation enhancement model based on the representations of the center node and neighbor nodes of that tail node. The node feature enhancement process for tail nodes can be referred to the node feature enhancement process described above.

[0152] Figure 14 An example flowchart of an article recommendation method 1400 according to an embodiment of this specification is shown.

[0153] like Figure 14 As shown in 1410, the head nodes in the user-item interaction graph are provided to the graph neural network model for graph learning in order to learn the node representation of the head nodes, which include head user nodes and head item nodes.

[0154] In 1420, a graph representation enhancement model is used to enhance the graph representation of the tail nodes in the user-item interaction graph to obtain the node representation of the tail nodes. The node representation of each tail node is learned based on the enhanced user-item interaction graph. The enhanced user-item interaction graph is obtained by enhancing the neighborhood missing information of each tail node in the user-item interaction graph. The neighborhood missing information of each tail node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representation of the tail node. The tail nodes include user tail nodes and item tail nodes.

[0155] In step 1430, item recommendation decisions are made based on the node representations of the target user and candidate items in the user-item interaction graph. How to make item recommendation decisions based on the node representations of the target user and candidate items can be achieved using any suitable decision-making mechanism in this field.

[0156] Figure 15 An example block diagram of a graph representation enhancement model training apparatus 1500 according to an embodiment of this specification is shown. Figure 15 As shown, the graph representation enhancement model training device 1500 includes a sparse graph construction unit 1510, a graph representation enhancement unit 1520, a first loss function determination unit 1530, and a first model adjustment unit 1540.

[0157] The sparse graph construction unit 1510 is configured to construct a sparse graph based on the original long-tailed graph. The constructed sparse graph includes true tail nodes and pseudo tail nodes. The pseudo tail nodes are generated by performing neighborhood pruning on the true head nodes in the original long-tailed graph. The operation of the sparse graph construction unit 1510 can be referred to the above. Figure 4 410 and Figures 5-6 The described operation.

[0158] The graph representation enhancement unit 1520 is configured to use a graph representation enhancement model to enhance the graph representation of each node in the sparse graph to obtain enhanced node representations for each node. The enhanced node representations are learned based on the enhanced sparse graph, which is obtained by enhancing the missing neighborhood information of each node in the sparse graph. Furthermore, the missing neighborhood information of each node is predicted by the graph representation enhancement model based on the representations of its center node and neighboring nodes. The operation of the graph representation enhancement unit 1520 can be referenced above. Figure 4 420 and Figure 7- Figure 12 The described operation.

[0159] The first loss function determination unit 1530 is configured to determine the neighborhood information prediction loss function based on the enhanced node representation of the pseudo-tail node and the node representation of the corresponding true head node in the original long-tail graph. The operation of the first loss function determination unit 1530 can be referenced above. Figure 4 The operation described in 430.

[0160] The first model adjustment unit 1540 is configured to predict the loss function based on neighborhood information and adjust the model parameters of the graph representation enhancement model. The operation of the first model adjustment unit 1540 can be referenced above. Figure 4 The operation described in 440.

[0161] Figure 16 An example block diagram of a graph representation enhancement unit 1600 according to an embodiment of this specification is shown. Figure 16 As shown, the graph representation enhancement unit 1600 includes a neighborhood missing information prediction module 1610 and a graph learning module 1620.

[0162] When the graph representation enhancement model is implemented as including a single-layer graph representation enhancement model, the neighborhood missing information prediction module 1610 uses the graph representation enhancement model to predict the neighborhood missing information of each node based on the initial center node representation and the initial neighbor node representation of each node in the sparse graph. The predicted neighborhood missing information of each node is used as the initial node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph. Subsequently, the graph learning module 1620 uses a graph neural network model to perform graph learning based on the enhanced sparse graph to learn the enhanced node representation of each node.

[0163] When the graph representation enhancement model is implemented as including a multi-layer graph representation enhancement model, the neighborhood missing information prediction module 1610 and the graph learning module 1620 perform operations cyclically until the final node representation of each node is output. Specifically, in each iteration, the neighborhood missing information prediction module 1610 uses the current graph representation enhancement model to predict the current neighborhood missing information of each node based on the current center node representation and the current neighbor node representation of each node in the sparse graph. The predicted neighborhood missing information of each node is used as the current node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph. Then, the graph learning module 1620 uses a graph neural network model to perform graph learning based on the enhanced sparse graph to learn the current enhanced node representation of each node.

[0164] In some embodiments, the graph representation enhancement model training apparatus may further include a head node generation adversarial task execution unit. The head node generation adversarial task execution unit can use a head node generation adversarial network to perform a head node generation adversarial task. In this case, the first loss function determination unit 1530 further determines a head node adversarial loss function based on the enhanced node representations of each node in the sparse graph and the node representations of the head nodes in the original long-tailed graph. Subsequently, the first model parameter adjustment unit 1540 adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the head node adversarial loss function.

[0165] In some embodiments, the graph representation enhancement model training apparatus may further include a downstream task execution unit. The downstream task execution unit performs a downstream task using the enhanced node representations of each node in the sparse graph. In this case, the first loss function determination unit 1530 further determines a downstream task loss function based on the downstream task execution results. Subsequently, the first model parameter adjustment unit 1540 adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the downstream task loss function.

[0166] In some embodiments, the graph representation enhancement model training apparatus may further include a contrastive learning task execution unit. The contrastive learning task execution unit is configured to perform a contrastive learning task based on the enhanced node representations of each node in the sparse graph. In this case, the first loss function determination unit 1530 further determines a contrastive learning task loss function based on the result of the contrastive learning task execution. Subsequently, the first model parameter adjustment unit 1540 adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the contrastive learning task loss function.

[0167] In some embodiments, the sparse graph construction unit can use an adaptive sampling model to construct a sparse graph based on the original long-tailed graph. Furthermore, the graph representation enhancement model training apparatus may further include a tail node generation adversarial task execution unit, a second loss function determination unit, and a second model adjustment unit. The tail node generation adversarial task execution unit uses a tail node generation adversarial network model to perform a tail node generation adversarial training task. The second loss function determination unit determines the tail node adversarial loss function based on the node representations of the pseudo-tailed nodes in the sparse graph and the node representations of the true tailed nodes in the original long-tailed graph. Subsequently, the second model parameter adjustment unit adjusts the model parameters of the adaptive sampling model based on the tail node adversarial loss function.

[0168] Figure 17 An example block diagram of a graphical representation device 1700 according to an embodiment of this specification is shown. Figure 17 As shown, the graph representation device 1700 includes a head node representation unit 1710 and a tail node representation unit 1720.

[0169] The head node representation unit 1710 is configured to provide the head node of the long-tailed graph to be represented to the graph neural network model for graph learning, so as to learn the node representation of the head node. The operation of the head node representation unit 1710 can be referred to the above reference. Figure 13 The operation described in 1310.

[0170] The tail node representation unit 1720 is configured to use a graph representation enhancement model to perform graph representation enhancement on the tail nodes in the long-tail graph to be represented, thereby obtaining the node representations of the tail nodes. The node representations of each tail node are learned based on the enhanced long-tail graph, which is obtained by enhancing the neighborhood missing information of each tail node in the long-tail graph to be represented. The neighborhood missing information of each tail node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of that tail node. The operation of the tail node representation unit 1720 can be referred to the above reference. Figure 13 The operation described in 1320.

[0171] Figure 18 An example block diagram of an article recommendation device 1800 according to an embodiment of this specification is shown. Figure 18 As shown, the item recommendation device 1800 includes a head node representation unit 1810, a tail node representation unit 1820, and a recommendation decision unit 1830.

[0172] The head node representation unit 1810 is configured to provide the head nodes in the user-item interaction graph to the graph neural network model for graph learning in order to learn the node representation of the head nodes, which include head user nodes and head item nodes.

[0173] The tail node representation unit 1820 is configured to use a graph representation enhancement model to perform graph representation enhancement on the tail nodes in the user-item interaction graph to obtain the node representation of the tail nodes. The node representation of each tail node is learned based on the enhanced user-item interaction graph. The enhanced user-item interaction graph is obtained by enhancing the neighborhood missing information of each tail node in the user-item interaction graph. The neighborhood missing information of each tail node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representation of the tail node. The tail nodes include user tail nodes and item tail nodes.

[0174] The recommendation decision unit 1830 is configured to make item recommendation decisions based on the node representations of the target user and candidate items in the user-item interaction graph.

[0175] As shown in Figure 1 above. Figure 18 This specification describes a graph representation enhancement model training method and apparatus, a graph representation method and apparatus, and an item recommendation method and apparatus according to embodiments thereof. The graph representation enhancement model training apparatus, graph representation apparatus, and item recommendation apparatus described above can be implemented in hardware, software, or a combination of both.

[0176] Figure 19 An example schematic diagram of a graph representation enhancement model training apparatus 1900 implemented using a computer system according to an embodiment of this specification is shown. Figure 19 As shown, the graphical representation enhancement model training apparatus 1900 may include at least one processor 1910, a memory (e.g., non-volatile memory) 1920, a main memory 1930, and a communication interface 1940, and the at least one processor 1910, memory 1920, main memory 1930, and communication interface 1940 are connected together via a bus 1960. At least one processor 1910 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0177] In one embodiment, computer-executable instructions are stored in memory, which, when executed, cause at least one processor 1910 to: construct a sparse graph based on the original long-tailed graph, the constructed sparse graph including true tail nodes and pseudo tail nodes, the pseudo tail nodes being generated by neighborhood pruning of the true head nodes in the original long-tailed graph; use a graph representation enhancement model to perform graph representation enhancement on each node in the sparse graph to obtain enhanced node representations for each node, the enhanced node representations for each node being learned based on the enhanced sparse graph, the enhanced sparse graph being obtained by enhancing the neighborhood missing information of each node in the sparse graph, the neighborhood missing information of each node being predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the node; determine a neighborhood information prediction loss function based on the enhanced node representations of the pseudo tail nodes and the node representations of the corresponding true head nodes in the original long-tailed graph; and adjust the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function.

[0178] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1910 to perform the above-described combinations in the various embodiments of this specification. Figures 3-12 as well as Figures 15-16 The description includes various operations and functions.

[0179] Figure 20 An example schematic diagram of a graph representation device 2000 implemented using a computer system according to an embodiment of this specification is shown. Figure 20 As shown, the device 2000 may include at least one processor 2010, a memory (e.g., non-volatile memory) 2020, a main memory 2030, and a communication interface 2040, and the at least one processor 2010, memory 2020, main memory 2030, and communication interface 2040 are connected together via a bus 2060. At least one processor 2010 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0180] In one embodiment, computer-executable instructions are stored in memory that, when executed, cause at least one processor 2010 to: provide head nodes in a long-tailed graph to be represented to a graph neural network model for graph learning to learn node representations of the head nodes; and use a graph representation enhancement model to perform graph representation enhancement on tail nodes in the long-tailed graph to obtain node representations of the tail nodes, the node representations of each tail node being learned based on the enhanced long-tailed graph, the enhanced long-tailed graph being obtained by enhancing neighborhood missing information for each tail node in the long-tailed graph to be represented, the neighborhood missing information of each tail node being predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the tail node.

[0181] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 2010 to perform the above-described combinations in the various embodiments of this specification. Figure 13 and Figure 17 The description includes various operations and functions.

[0182] Figure 21 An example schematic diagram of a computer-based item recommendation device 2100 implemented according to an embodiment of this specification is shown. Figure 21 As shown, the item recommendation device 2100 may include at least one processor 2110, a memory (e.g., non-volatile memory) 2120, a RAM 2130, and a communication interface 2140, and the at least one processor 2110, memory 2120, RAM 2130, and communication interface 2140 are connected together via a bus 2160. The at least one processor 2110 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0183] In one embodiment, computer-executable instructions are stored in memory that, when executed, cause at least one processor 2110 to: provide head nodes in a user-item interaction graph to a graph neural network model for graph learning to learn node representations of the head nodes, the head nodes including head user nodes and head item nodes; use a graph representation enhancement model to perform graph representation enhancement on tail nodes in the user-item interaction graph to obtain node representations of the tail nodes, the node representations of each tail node being learned based on an enhanced user-item interaction graph, the enhanced user-item interaction graph being obtained by enhancing neighborhood missing information for each tail node in the user-item interaction graph, the neighborhood missing information of each tail node being predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the tail node, the tail nodes including user tail nodes and item tail nodes; and make item recommendation decisions based on the node representations of target users and candidate items in the user-item interaction graph.

[0184] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 2110 to perform the above-described combinations in the various embodiments of this specification. Figure 14 and Figure 18 The description includes various operations and functions.

[0185] According to one embodiment, a program product, such as a machine-readable medium (e.g., a non-transitory machine-readable medium), is provided. The machine-readable medium may have instructions (i.e., the elements implemented in software as described above), which, when executed by a machine, cause the machine to perform the various embodiments described above in conjunction with Figure 1- Figure 18The various operations and functions described. Specifically, a system or apparatus equipped with a readable storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer or processor of the system or apparatus to read and execute the instructions stored in the readable storage medium.

[0186] In this case, the program code itself, which can be read from a readable medium, can perform the functions of any of the above embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0187] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0188] According to one embodiment, a computer program product is provided, the computer program product including a computer program, which, when executed by a processor, causes the processor to perform the various embodiments described above in conjunction with Figure 1- Figure 18 The description includes various operations and functions.

[0189] Those skilled in the art will understand that the various embodiments disclosed above can be modified and varied without departing from the spirit of the invention. Therefore, the scope of protection of this invention should be defined by the appended claims.

[0190] It should be noted that not all steps and units in the above process and system structure diagrams are mandatory; some steps or units can be omitted according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in the above embodiments can be a physical structure or a logical structure; that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0191] In the above embodiments, the hardware units or modules can be implemented mechanically or electrically. For example, a hardware unit, module, or processor may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operation. The hardware unit or processor may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operation. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0192] The specific embodiments described above with reference to the accompanying drawings are exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of the claims. The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" compared to other embodiments. Specific details are included to provide an understanding of the described techniques. However, these techniques can be practiced without these specific details. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0193] The foregoing description of this disclosure is provided to enable any person skilled in the art to implement or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for training a graph representation augmentation model for item recommendation, comprising: A sparse graph is constructed based on the original long-tail graph. The sparse graph includes true tail nodes and pseudo tail nodes. The pseudo tail nodes are generated by performing neighborhood pruning on the true head nodes in the original long-tail graph. The original long-tail graph is a user-item interaction graph generated based on user-item interaction data. The user-item interaction graph has head nodes and tail nodes. The head nodes are nodes in the user-item interaction graph with a degree greater than a node degree threshold, and the tail nodes are nodes in the user-item interaction graph with a degree not greater than the node degree threshold. The node degree refers to the number of neighboring nodes of a node. The graph representation enhancement model is used to enhance the graph representation of each node in the sparse graph to obtain the enhanced node representation of each node. The enhanced node representation of each node is learned based on the enhanced sparse graph. The enhanced sparse graph is obtained by enhancing the neighborhood missing information of each node in the sparse graph. The neighborhood missing information of each node is predicted by the graph representation enhancement model based on the center node representation and the neighbor node representation of the node. Based on the enhanced node representation of the pseudo-tail node and the node representation of the corresponding true head node in the original long-tail graph, the neighborhood information prediction loss function is determined. as well as Based on the neighborhood information prediction loss function, the model parameters of the graph representation enhancement model are adjusted.

2. The graph representation enhancement model training method as described in claim 1, wherein, The graph representation enhancement model includes a single-layer graph representation enhancement model. Using this model to enhance the graph representation of each node in the sparse graph includes: A graph representation enhancement model is used to predict the neighborhood missing information of each node based on the initial center node representation and initial neighbor node representation of each node in the sparse graph. The predicted neighborhood missing information of each node is used as the initial node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph; and A graph neural network model is used to perform graph learning based on the enhanced sparse graph to learn the enhanced node representations for each node.

3. The graph representation enhancement model training method as described in claim 1, wherein, The graph representation enhancement model includes a multi-layer graph representation enhancement model, and the graph representation enhancement model is used to enhance the graph representation of each node in the sparse graph, including: Repeat the following process until the augmented node representations of each node in the sparse graph are obtained: The current graph representation enhancement model is used to predict the current neighborhood missing information of each node based on the current center node representation and the current neighbor node representation of each node in the sparse graph. The predicted neighborhood missing information of each node is used as the current node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph; and A graph neural network model is used to perform graph learning based on the augmented sparse graph to learn the current augmented node representation for each node.

4. The graph representation enhancement model training method as described in claim 1 further includes: A head node generative adversarial network is used to perform a head node generative adversarial task, and the head node adversarial loss function of the head node generative adversarial task is determined based on the execution result of the head node generative adversarial task. Based on the neighborhood information prediction loss function, adjusting the model parameters of the graph representation enhancement model includes: The model parameters of the graph representation enhancement model are adjusted based on the neighborhood information prediction loss function and the head node adversarial loss function.

5. The graph representation enhancement model training method as described in claim 1 further includes: The enhanced node representations of each node in the sparse graph are used to represent the execution of downstream tasks, and the loss function of the downstream tasks is determined based on the execution results of the downstream tasks. Based on the neighborhood information prediction loss function, adjusting the model parameters of the graph representation enhancement model includes: The model parameters of the graph representation enhancement model are adjusted based on the neighborhood information prediction loss function and the downstream task loss function.

6. The graph representation enhancement model training method as described in claim 1 further includes: A contrastive learning task is performed based on the augmented node representations of each node in the sparse graph, and the loss function for the contrastive learning task is determined based on the results of the contrastive learning execution. Based on the neighborhood information prediction loss function, adjusting the model parameters of the graph representation enhancement model includes: The model parameters of the graph representation enhancement model are adjusted based on the neighborhood information prediction loss function and the contrastive learning task loss function.

7. The graph representation enhancement model training method as described in claim 1, wherein, Constructing a sparse graph based on the original long-tailed graph includes: An adaptive sampling model is used to construct a sparse graph based on the original long-tailed graph.

8. The graph representation enhancement model training method as described in claim 7, further comprising: Tail-node generative adversarial networks (GANs) are used to perform tail-node generative adversarial tasks, and the tail-node adversarial loss function of the tail-node generative adversarial task is determined based on the execution result of the tail-node generative adversarial task; and The model parameters of the adaptive sampling model are adjusted based on the tail node adversarial loss function.

9. A graphical representation method for item recommendation, comprising: The head node in the long-tail graph to be represented is provided to the graph neural network model for graph learning to learn the node representation of the head node. The long-tail graph to be represented is a user-item interaction graph generated based on user-item interaction data. The user-item interaction graph has head nodes and tail nodes. The head node refers to the node in the user-item interaction graph whose node degree is greater than the node degree threshold, and the tail node refers to the node in the user-item interaction graph whose node degree is not greater than the node degree threshold. The node degree refers to the number of neighboring nodes of the node. as well as A graph representation enhancement model is used to enhance the graph representation of the tail nodes in the long-tail graph to be represented, thereby obtaining the enhanced node representation of the tail nodes. The enhanced node representation of each tail node is learned based on the enhanced long-tail graph. The enhanced long-tail graph is obtained by enhancing the neighborhood missing information of each tail node in the long-tail graph to be represented. The neighborhood missing information of each tail node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representation of the tail node.

10. A method for recommending items, including: The head nodes in the user-item interaction graph are provided to the graph neural network model for graph learning to learn the node representation of the head nodes. The head nodes include head user nodes and head item nodes. The head nodes refer to the nodes in the user-item interaction graph whose node degree is greater than the node degree threshold. The node degree refers to the number of neighboring nodes of the node. A graph representation enhancement model is used to enhance the graph representation of the tail nodes in the user-item interaction graph to obtain the node representations of the tail nodes. The node representations of each tail node are learned based on the enhanced user-item interaction graph. The enhanced user-item interaction graph is obtained by enhancing the neighborhood missing information of each tail node in the user-item interaction graph. The neighborhood missing information of each tail node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the tail node. The tail nodes include user tail nodes and item tail nodes. A tail node refers to a node in the user-item interaction graph whose node degree is not greater than the node degree threshold. Item recommendation decisions are made based on the node representations of target users and candidate items in the user-item interaction graph.

11. A training apparatus for a graph representation augmentation model for item recommendation, comprising: A sparse graph construction unit constructs a sparse graph based on an original long-tail graph. The sparse graph includes true tail nodes and pseudo tail nodes. The pseudo tail nodes are generated by performing neighborhood pruning on the true head nodes in the original long-tail graph. The original long-tail graph is a user-item interaction graph generated based on user-item interaction data. The user-item interaction graph has head nodes and tail nodes. The head nodes are nodes in the user-item interaction graph with a degree greater than a node degree threshold, and the tail nodes are nodes in the user-item interaction graph with a degree not greater than the node degree threshold. The node degree refers to the number of neighboring nodes of a node. The graph representation enhancement unit uses a graph representation enhancement model to enhance the graph representation of each node in the sparse graph to obtain the enhanced node representation of each node. The enhanced node representation of each node is learned based on the enhanced sparse graph. The enhanced sparse graph is obtained by enhancing the neighborhood missing information of each node in the sparse graph. The neighborhood missing information of each node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representation of the node. The first loss function determination unit determines the neighborhood information prediction loss function based on the enhanced node representation of the pseudo-tail node and the node representation of the corresponding true head node in the original long-tail graph. as well as The first model adjustment unit adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function.

12. The graph representation enhancement model training apparatus as described in claim 11, wherein, The graph representation enhancement model includes a single-layer graph representation enhancement model, and the graph representation enhancement unit includes: The neighborhood missing information prediction module uses a graph representation enhancement model to predict the neighborhood missing information of each node based on the initial center node representation and initial neighbor node representation of each node in the sparse graph. The predicted neighborhood missing information of each node is used as the initial node representation of the pseudo neighbor nodes of each node in the enhanced sparse graph. The graph learning module uses a graph neural network model to perform graph learning based on the enhanced sparse graph, so as to learn the enhanced node representation of each node.

13. The graph representation enhancement model training apparatus as described in claim 11, wherein, The graph representation enhancement model includes a multi-layer graph representation enhancement model, and the graph representation enhancement unit includes: The neighborhood missing information prediction module uses a current graph representation enhancement model to predict the current neighborhood missing information of each node based on the current center node representation and the current neighbor node representation of each node in the sparse graph. The predicted neighborhood missing information of each node is used as the current node representation of the pseudo-neighbor nodes of each node in the enhanced sparse graph; and The graph learning module uses a graph neural network model to perform graph learning based on the enhanced sparse graph, in order to learn the current enhanced node representation of each node.

14. The graph representation enhancement model training apparatus of claim 11, further comprising: The head node generates adversarial task execution units, using a head node generative adversarial network to execute head node generative adversarial tasks. Specifically, the first loss function determination unit further determines the head node adversarial loss function based on the execution result of the head node adversarial task, and the first model adjustment unit adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the head node adversarial loss function.

15. The graph representation enhancement model training apparatus of claim 11, further comprising: The downstream task execution unit uses the augmented node representations of each node in the sparse graph to execute downstream tasks. The first loss function determination unit further determines the downstream task loss function based on the downstream task execution result, and the first model adjustment unit adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the downstream task loss function.

16. The graph representation enhancement model training apparatus of claim 11, further comprising: The contrastive learning task execution unit performs a contrastive learning task based on the enhanced node representations of each node in the sparse graph. The first loss function determination unit further determines the contrastive learning task loss function based on the execution result of the contrastive learning task, and the first model adjustment unit adjusts the model parameters of the graph representation enhancement model based on the neighborhood information prediction loss function and the contrastive learning task loss function.

17. The graph representation enhancement model training apparatus as described in claim 11, wherein, The sparse graph construction unit uses an adaptive sampling model to construct a sparse graph based on the original long-tailed graph.

18. The graph representation enhancement model training apparatus of claim 17, further comprising: The tail node generative adversarial training task execution unit uses the tail node generative adversarial network model to execute the tail node generative adversarial task. The second loss function determination unit determines the tail node adversarial loss function based on the execution result of the adversarial task generated by the tail node; and The second model adjustment unit adjusts the model parameters of the adaptive sampling model based on the tail node adversarial loss function.

19. A graphical representation device for recommending items, comprising: The head node representation unit provides the head node in the long-tail graph to be represented to the graph neural network model for graph learning, so as to learn the node representation of the head node. as well as The tail node representation unit uses a graph representation enhancement model to enhance the graph representation of the tail nodes in the long-tail graph to be represented, thereby obtaining the node representations of the tail nodes. The node representations of each tail node are learned based on the enhanced long-tail graph. The enhanced long-tail graph is obtained by enhancing the neighborhood missing information of each tail node in the long-tail graph to be represented. The neighborhood missing information of each tail node is predicted by the graph representation enhancement model based on the center node representation and neighbor node representations of the tail node. The long-tail graph to be represented is a user-item interaction graph generated based on user-item interaction data. The user-item interaction graph has head nodes and tail nodes. The head node refers to a node in the user-item interaction graph whose node degree is greater than a node degree threshold, and the tail node refers to a node in the user-item interaction graph whose node degree is not greater than the node degree threshold. The node degree refers to the number of neighboring nodes of a node.

20. A graph representation enhancement model training device, comprising: At least one processor; Memory coupled to the at least one processor; as well as A computer program stored in the memory, which is executed by the at least one processor to implement the graph representation enhancement model training method as described in any one of claims 1 to 8.

21. A graphical representation device for item recommendation, comprising: At least one processor; Memory coupled to the at least one processor; as well as A computer program stored in the memory, which is executed by the at least one processor to implement the graph representation method for item recommendation as described in claim 9.

Images