Method and system for training graph neural network

Abstract

The embodiment of the invention relates to a method and system for training a graph neural network, and the method comprises the steps: in an offline stage, firstly, carrying out the distributed sub-graph sampling of a knowledge graph through each computing device in a distributed system by taking a plurality of graph elements in local fragmentation data of the knowledge graph as a sampling center, and obtaining a plurality of sub-graphs; the graph elements comprise nodes and / or relation edges; then, each computing device takes each graph element as a key, takes the sub-graph corresponding to each graph element as a value, and stores the plurality of sub-graphs in a target storage area in a key-value pair form; in a training stage, firstly, training equipment takes a first batch of first graph elements as keys, and queries from the target storage area to obtain a first batch of first sub-graphs; the training device then trains a graph neural network based on the first sub-graph of the first batch.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of knowledge graphs, and in particular, to methods and systems for training graph neural networks. Background Art

[0002] In recent years, knowledge graphs (KGs) have attracted widespread attention in many fields such as recommendation systems and transaction risk control. In order to enhance the representation ability of knowledge graphs, knowledge graph embedding learning has become a key task, which aims to embed entities and relationship edges in knowledge graphs into the potential representation space to assist various downstream tasks, including link prediction, relationship prediction, entity classification, etc. Among them, graph neural networks (GNNs) are a type of neural network model specifically used to process graph structured data, which can well learn the embedded representation of elements in knowledge graphs.

[0003] In the training process of graph neural networks, batches of subgraphs are often generated online from the original graph, and then the graph neural network is trained based on the subgraphs. However, the scale of knowledge graphs is getting larger and larger, reaching tens of billions or even trillions. Traditional online sampling consumes a lot of computing resources, resulting in reduced model training efficiency. Therefore, a method is needed to improve the efficiency of model training and save computing resources. Summary of the invention

[0004] One or more embodiments of this specification describe a method and system for training a graph neural network, separating the steps of subgraph sampling and model training to improve the efficiency of model training.

[0005] In a first aspect, a method for training a graph neural network is provided, comprising:

[0006] In the offline stage, each computing device in the distributed system uses multiple graph elements in the local shard data of the knowledge graph as sampling centers, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs; the graph elements include nodes and / or relationship edges;

[0007] Each computing device uses each graph element as a key and a subgraph corresponding to each graph element as a value, and stores the multiple subgraphs in a target storage area in the form of key-value pairs;

[0008] In the training phase, the training device uses the first graph element of the first batch as a key to query and obtain the first subgraph of the first batch from the target storage area;

[0009] The training device trains the graph neural network based on the first subgraph of the first batch.

[0010] In some possible implementations, before performing distributed subgraph sampling on the knowledge graph, the method further includes:

[0011] In the offline phase, the master device in the distributed system divides the knowledge graph into multiple graph shards.

[0012] The master device distributes each graph slice to each computing device as local slice data of each computing device.

[0013] In some possible implementations, the multiple graph elements include a first node; each computing device uses multiple graph elements in local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including:

[0014] The first computing device having the first node takes the first node as a sampling center, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first node.

[0015] In some possible implementations, the multiple graph elements include a first relationship edge; each computing device uses the multiple graph elements in the local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including:

[0016] The second computing device having the first relationship edge respectively takes the two nodes connected by the first relationship edge as sampling centers, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first relationship edge.

[0017] In some possible implementations, the first graph element is a node; and the training device trains the graph neural network based on the first subgraph of the first batch, including:

[0018] The training device aggregates information on each first subgraph respectively to obtain a first graph representation corresponding to each node of the first batch;

[0019] The training device uses the node attributes of each node as a training label and trains the graph neural network according to each first graph representation.

[0020] In some possible implementations, the first graph element is a relationship edge; and the training device trains the graph neural network based on the first subgraph of the first batch, including:

[0021] The training device aggregates information on each first subgraph respectively to obtain a second graph representation of each second relationship edge of the first batch;

[0022] The training device uses the edge attributes of each relationship edge as a training label and trains the graph neural network according to each second graph representation.

[0023] In some possible implementations, the following further includes:

[0024] In the training phase, the training device trains the graph neural network based on the first subgraph of the first batch, and uses the second graph element of the second batch as a key to query the second subgraph of the second batch from the target storage area and store it in its own target cache area;

[0025] The training device reads the second subgraph of the second batch from the target buffer area, and trains the graph neural network based on the second subgraph of the second batch.

[0026] In some possible implementations, the knowledge graph includes several knowledge points related to users, products and / or transactions; and the node attributes include: user attributes or product attributes.

[0027] In some possible implementations, the knowledge graph includes several knowledge points related to users, commodities and / or transactions; and the edge attributes include: transaction attributes.

[0028] In a second aspect, a method for obtaining and storing training data is provided, comprising:

[0029] In the offline stage, each computing device in the distributed system uses multiple graph elements in the local shard data of the knowledge graph as sampling centers, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs; the graph elements include nodes and / or relationship edges;

[0030] Each computing device uses each graph element as a key and the subgraph corresponding to each graph element as a value, and stores the multiple subgraphs in the form of key-value pairs in a target storage area; the multiple subgraph data are used to train the graph neural network.

[0031] In some possible implementations, before performing distributed subgraph sampling on the knowledge graph, the method further includes:

[0032] In the offline phase, the master device in the distributed system divides the knowledge graph into multiple graph shards.

[0033] The master device distributes each graph slice to each computing device as local slice data of each computing device.

[0034] In some possible implementations, the multiple graph elements include a first node; each computing device uses multiple graph elements in local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including:

[0035] The first computing device having the first node takes the first node as a sampling center, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first node.

[0036] In some possible implementations, the multiple graph elements include a first relationship edge; each computing device uses the multiple graph elements in the local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including:

[0037] The second computing device having the first relationship edge respectively takes the two nodes connected by the first relationship edge as sampling centers, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first relationship edge.

[0038] In a third aspect, a system for training a graph neural network is provided, including a distributed system and a training device, wherein:

[0039] Each computing device in the distributed system is used to, in an offline stage, use multiple graph elements in the local shard data of the knowledge graph as sampling centers to perform distributed subgraph sampling on the knowledge graph to obtain multiple subgraphs; the graph elements include nodes and / or relationship edges;

[0040] Each computing device is further used to store the plurality of subgraphs in a target storage area in the form of key-value pairs, using each graph element as a key and a subgraph corresponding to each graph element as a value;

[0041] The training device is used to, during the training phase, use the first graph element of the first batch as a key to query and obtain the first subgraph of the first batch from the target storage area; and train the graph neural network based on the first subgraph of the first batch.

[0042] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first aspect or the second aspect.

[0043] In a fifth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect or the second aspect is implemented.

[0044] The method and system for training a graph neural network proposed in the embodiments of this specification separate subgraph sampling and model training. In the offline stage before model training, subgraph sampling is first performed on the original knowledge graph in a distributed system, and then the multiple subgraphs obtained by sampling are saved in a storage area in the form of key-value pairs. In this way, in the online model training stage, the training device can directly read the corresponding subgraph from the storage area and process it according to the graph elements currently to be processed, without the need for sampling operations, which significantly improves the efficiency of model training and saves the computing resources of the training device. Furthermore, reading subgraphs and training models can be performed in parallel. While the model is being trained, the next batch of subgraphs to be used are loaded from the storage area into the cache area of ​​the training device in advance to further improve the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the multiple embodiments disclosed in this specification, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only the multiple embodiments disclosed in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram showing an implementation scenario of a method for training a graph neural network according to an embodiment;

[0047] Figure 2 A flowchart showing a method for training a graph neural network according to one embodiment;

[0048] Figure 3 A schematic diagram of a system for training a graph neural network according to one embodiment is shown. DETAILED DESCRIPTION

[0049] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0050] In the graph neural network training method of the related art, if the data volume of the original knowledge graph (hereinafter referred to as the original graph) is too large, the training device cannot read the entire original graph into its own memory at one time. Therefore, the training device usually uses a mini-batch method for training, sampling from the original graph to generate a batch of subgraphs. After a batch of subgraphs is formed, this batch of subgraphs is sent to the graph neural network for training. After the training of this batch of subgraphs is completed, the next batch of subgraphs is generated for training. Usually within one iteration of graph neural network training, the generated batches of subgraphs can cover the entire original graph. After one iteration ends, subgraphs continue to be generated on the original graph for the next iterative training, and several iterative trainings are repeated until the model converges.

[0051] In the related art, subgraphs need to be repeatedly generated between several iterations. For example, after the first iteration is completed, the second iteration still needs to generate the same or similar subgraphs as the previous iteration to perform training for the second iteration. This repeated calculation causes a waste of computing resources and greatly reduces the training efficiency.

[0052] Based on the above analysis, the embodiments of this specification propose a method for training a graph neural network to overcome the above problems. Figure 1 A schematic diagram of an implementation scenario of a method for training a graph neural network according to an embodiment is shown. Figure 1 In the example, the complete training process is divided into two stages: offline and online training. The data of the original knowledge graph is stored in the form of graph shards in various computing devices of the distributed system.

[0053] In the offline stage, each computing device in the distributed system uses multiple graph elements (nodes and / or relationship edges) in its own local graph shard data as sampling centers to perform distributed subgraph sampling on the knowledge graph to obtain multiple subgraphs. Then, since each subgraph is obtained by sampling with a node or relationship edge as the sampling center, the sampling center is used as the key and the corresponding subgraph is used as the value, and the subgraph is stored in the storage area in the form of a key-value pair.

[0054] In the online training phase, the training device determines the subgraph batches to be used in this round of iterative training, uses the graph elements of each subgraph in the batch as keys, queries the graph data of each subgraph in the batch from the storage area, and loads it into the local cache area of ​​the training device, such as memory. Then, based on these graph data, the graph neural network is trained to complete the model training process based on the batch.

[0055] After multiple rounds of query-load-training processes, the training process of the graph neural network is completed.

[0056] Furthermore, since the loading and training processes are separate, during any i-th round of iterative training, the subgraph batches required for the i+1th round of iterative training can be queried from the storage area in advance and loaded into the local cache area, so that the training process and the loading process can be executed in parallel, further accelerating the overall training process.

[0057] The following describes the specific implementation steps of the above-mentioned method for training a graph neural network in conjunction with a specific embodiment.

[0058] Figure 2 A flowchart of a method for training a graph neural network according to an embodiment is shown. The execution subject of the method can be any platform or server or device cluster with computing and processing capabilities. Figure 2 As shown, the method at least includes: step 206, in the offline stage, each computing device in the distributed system uses multiple graph elements in the local shard data of the knowledge graph as sampling centers, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs; the graph elements include nodes and / or relationship edges; step 208, each computing device uses each graph element as a key and the subgraph corresponding to each graph element as a value, and stores the multiple subgraphs in the target storage area in the form of key-value pairs; step 210, in the training stage, the training device uses the first graph element of the first batch as a key, and queries the first subgraph of the first batch from the target storage area to obtain the first subgraph of the first batch; step 212, the training device trains the graph neural network based on the first subgraph of the first batch.

[0059] The specific execution process of each of the above steps is described below.

[0060] First, in step 206, in the offline stage, each computing device in the distributed system uses multiple graph elements in the local shard data of the knowledge graph as sampling centers, performs distributed sub-graph sampling on the knowledge graph, and obtains multiple sub-graphs; the graph elements include nodes and / or relationship edges.

[0061] The graph shard data of the knowledge graph is stored in each computing device in the distributed system, and the overall graph shard data in each computing device can constitute the original knowledge graph.

[0062] The knowledge graph can be a knowledge graph related to any application field and stores relevant knowledge points in the application field. Knowledge points can exist in the knowledge graph in the form of triples (head node, relationship edge, tail node).

[0063] In one embodiment, the knowledge graph is a knowledge graph related to the e-commerce field, which contains knowledge points related to users, products and / or transactions. Among them, users and products can correspond to nodes in the knowledge graph, and transactions can correspond to relationship edges in the knowledge graph. Transactions between users can be transfers, payments, collections, etc.; transactions between users and products can be purchases, returns, etc.

[0064] The nodes and edges of the knowledge graph can also have their own attribute information. For example, the attributes of a user node can be the user's personality, preferences, transfer frequency, etc.; the attributes of a product can be the category, price, etc.; the attributes of a transaction can be a normal transaction, a risky transaction, a large transaction, etc.

[0065] The graph neural network can be trained based on the knowledge points in the knowledge graph to obtain the embedded representation of each node and relationship edge, such as an embedded vector. These embedded vectors can be used for specific downstream tasks, such as entity classification, relationship prediction, link prediction, etc. Entity classification is used to predict whether a specific entity node has a specific attribute, such as predicting whether a user has a specific preference; relationship prediction is used to predict whether a specific relationship edge has a specific attribute, such as predicting whether a transaction is a risky transaction; link prediction is used to predict whether there is a specific relationship edge between two nodes, such as predicting whether a user will buy a certain product.

[0066] Many types of graph neural networks can be trained, such as Graph Convolutional Network (GCN), Graph Attention Network (GAT), Graph Isomorphism Network (GIN), etc., without limitation here.

[0067] Subgraph sampling can be a neighbor-based subgraph sampling method, such as fixed number of neighbor sampling, k-order neighbor sampling, k-order neighbor sampling sorted by importance, etc. These subgraph sampling methods use a graph element (node ​​or relationship edge) as the sampling center, radiate outward to collect one or more layers of neighbor nodes and relationship edges, and then form a subgraph centered on the graph element. When training a graph neural network, the embedding vector of the graph element can be determined based on the subgraph.

[0068] In distributed subgraph sampling, each computing device uses multiple graph elements in its local shard data as sampling centers and performs subgraph sampling in parallel.

[0069] In one embodiment, the multiple graph elements include a first node; step 206 specifically includes: a first computing device having the first node uses the first node as a sampling center, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first node.

[0070] Specifically, first, first-order neighbor sampling is performed, with the first node as the root node, and each first-order neighbor node directly connected to the first node is obtained, and these nodes and the relationship edges between these nodes and the first node are added to the subgraph. Optionally, a sampling quantity limit can also be set, for example, each node samples a maximum of N first-order neighbors.

[0071] Then, based on each node obtained by the first-order sampling, first-order neighbor sampling is performed to obtain second-order neighbor nodes, and these second-order neighbor nodes and the relationship edges between the second-order neighbor nodes and the first-order neighbor nodes are added to the subgraph. Optionally, a sampling quantity limit can also be set for each node.

[0072] Repeat the operation in the previous step, expanding outward layer by layer until the k-th neighbor of the first node is reached, completing the k-th neighbor sampling for the first node.

[0073] In another embodiment, the multiple graph elements include a first relationship edge; step 206 specifically includes: the second computing device having the first relationship edge respectively takes the two nodes connected by the first relationship edge as sampling centers, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first relationship edge.

[0074] Specifically, k-order neighbor sampling is performed on the second node and the third node connected by the first relationship edge, and the obtained two subgraphs are merged to obtain the subgraph corresponding to the first relationship edge.

[0075] The step of performing k-order neighbor sampling on the second node and the third node may refer to the previous embodiment and will not be described in detail here.

[0076] By performing sub-graph sampling on multiple graph elements in the knowledge graph, the corresponding sub-graph of each graph element can be obtained.

[0077] Then, in step 208, in the offline stage, each computing device uses each graph element as a key and a subgraph corresponding to each graph element as a value to store the multiple subgraphs in the target storage area in the form of key-value pairs.

[0078] Key-Value Store is a data storage model that uses a key as an index to quickly retrieve the corresponding value. Each key is unique and is stored in pairs with a value, similar to the structure of a dictionary or hash table.

[0079] The embodiment of this specification uses the graph element as the key and its corresponding subgraph as the value to form a graph element-subgraph storage structure and store it in the target storage area. The target storage area can be any storage device, which can be accessed by various computing devices and training devices in the distributed system.

[0080] Next, in step 210, during the training phase, the training device uses the first graph element of the first batch as a key to query and obtain the first subgraph of the first batch from the target storage area.

[0081] After the training device obtains the first subgraph of the first batch through query, it loads it into its own local target buffer area, which may be a memory, for example.

[0082] The first graph element of the first batch may include nodes or relationship edges, or both nodes and relationship edges, depending on the specific tasks in the training graph neural network in the subsequent training stage. For example, if it is an entity classification or link prediction task, such as predicting whether a user has a specific preference, or predicting whether a user will buy a certain product, the first graph element may only include nodes; if it is a relationship prediction task, such as predicting whether a transaction is a risky transaction, the first graph element may only include relationship edges. If both entity classification and relationship prediction tasks are included, such as predicting whether a user is a risky user and whether a transaction is a risky transaction, the first graph element may include both nodes and relationship edges.

[0083] Then, in step 212, in the training phase, the training device trains the graph neural network based on the first subgraph of the first batch.

[0084] The specific process of training graph neural networks varies depending on the specific downstream tasks.

[0085] In one embodiment, the first graph element is a node; step 212 specifically includes:

[0086] The training device aggregates information on each first subgraph respectively to obtain a first graph representation corresponding to each node of the first batch.

[0087] The specific process of information aggregation is related to the specific graph neural network model. For example, when the graph neural network is a graph convolutional network (GCN), information aggregation can be performed by averaging or weighted averaging the features of neighboring nodes; when the graph neural network is a graph attention network (GAT), information aggregation can be performed by assigning different weights to different neighboring nodes through the attention mechanism.

[0088] After obtaining the first graph representation corresponding to each node, the training device uses the node attributes of each node as a training label and trains the graph neural network according to each first graph representation.

[0089] The first graph representation can be input into a multilayer perceptron (MLP) to obtain a predicted value of the node attribute for the first node. Then, a training loss is determined based on the predicted value and the training label, and the graph neural network is trained based on the training loss.

[0090] In a more specific embodiment, the knowledge graph includes several knowledge points related to users, products and / or transactions; the node attributes include: user attributes or product attributes.

[0091] In this embodiment, the downstream task may be an entity classification task of predicting user attributes or commodity attributes, for example, predicting whether a certain user is a risky user.

[0092] In another embodiment, the first graph element is a relationship edge; step 212 specifically includes:

[0093] The training device aggregates information of each first subgraph to obtain a second graph representation of each second relationship edge of the first batch. The training device uses the edge attribute of each relationship edge as a training label and trains the graph neural network according to each second graph representation.

[0094] Similar to the previous embodiment, the first graph representation can be input into a multi-layer perceptron to obtain a predicted value for the edge attribute of the first relationship edge, and then the training loss is determined based on the predicted value and the training label, and the graph neural network is trained based on the training loss.

[0095] In a more specific embodiment, the knowledge graph includes several knowledge points related to users, commodities and / or transactions; the edge attributes include: transaction attributes.

[0096] In this embodiment, the downstream task may be a relationship prediction task of predicting transaction attributes, such as predicting whether a transaction is a risky transaction.

[0097] The above describes the process in which the distributed system in the offline phase performs distributed subgraph sampling based on graph sharding, saves the sampling results in the target storage area in the form of key-value pairs, and then the training device in the training phase queries and loads the subgraph from the target storage area and trains the graph neural network.

[0098] In some possible implementations, before step 206 , the method further includes a process of segmenting the original knowledge graph to obtain graph fragments, which specifically includes steps 202 and 204 .

[0099] Step 202: In the offline stage, the master device in the distributed system segments the knowledge graph to obtain multiple graph fragments.

[0100] To partition the knowledge graph, you can use vertex partition or edge partition to evenly divide the knowledge graph into multiple graph slices.

[0101] Then, in step 204, in the offline phase, the master device allocates each graph shard to each computing device as local shard data of each computing device.

[0102] It should be noted that each step in the offline phase, including step 202, step 204, step 206 and step 208, only needs to be executed once in the entire graph neural network training process to obtain a target storage area storing multiple subgraphs. That is, even in the online training phase, the graph neural network will perform multiple rounds of iterative training, but each round of iterative training will directly query and load the subgraph from the target storage area without repeatedly sampling the subgraph in the original knowledge graph, so as to separate the sampling and training processes, reduce the pressure on the memory device and computing device in the training device, and improve the overall training efficiency.

[0103] In some possible implementations, the method further comprises the steps of:

[0104] During the training phase, the training device executes step 214 while executing step 212: using the second graph element of the second batch as a key, querying the second subgraph of the second batch from the target storage area, and storing it in its own target cache area.

[0105] The first subgraph of the first batch in step 210 may be data required for training the graph neural network in the i-th iteration, and the second subgraph of the second batch in step 214 may be data required for training the graph neural network in the i+1-th iteration. The computing device may further accelerate the training process of the whole by querying and loading the second subgraph of the second batch from the target storage area and into its own target cache area in parallel while using the first subgraph of the first batch to perform the i-th round of training the graph neural network in step 212.

[0106] Then, in step 216, the training device reads the second subgraph of the second batch from the target buffer area, and trains the graph neural network based on the second subgraph of the second batch.

[0107] While training the graph neural network based on the second subgraph of the second batch, the training device can also, in parallel, query and load the third subgraph of the third batch required for the next round of iteration into its own target cache area in accordance with step 212.

[0108] The embodiments of this specification collect subgraphs offline and store them using key-value pairs, and then query and load them directly during online training. On the one hand, a distributed system can be used to efficiently generate and store subgraphs in the offline stage, and on the other hand, the problem of repeated generation is avoided, thereby greatly improving the training efficiency of large-scale graph learning. In addition, by executing a certain round of iterative training process in parallel with the query and loading of subgraphs in the next round, the overall training efficiency can be further improved.

[0109] According to another aspect of the embodiment, a method for obtaining and storing training data is also provided, comprising:

[0110] In the offline stage, each computing device in the distributed system uses multiple graph elements in the local shard data of the knowledge graph as sampling centers, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs; the graph elements include nodes and / or relationship edges;

[0111] Each computing device uses each graph element as a key and the subgraph corresponding to each graph element as a value, and stores the multiple subgraphs in the form of key-value pairs in a target storage area; the multiple subgraph data are used to train the graph neural network.

[0112] According to an embodiment of another aspect, a device for training a language model is also provided. Figure 3 A schematic diagram of a system for training a graph neural network according to an embodiment is shown, wherein the system 300 for training a graph neural network includes a distributed system 302 and a training device 304, wherein:

[0113] Each computing device in the distributed system 302 is used to, in an offline stage, use multiple graph elements in the local shard data of the knowledge graph as sampling centers to perform distributed subgraph sampling on the knowledge graph to obtain multiple subgraphs; the graph elements include nodes and / or relationship edges;

[0114] Each computing device is further used to store the plurality of subgraphs in a target storage area in the form of key-value pairs, using each graph element as a key and a subgraph corresponding to each graph element as a value;

[0115] The training device 304 is used to, during the training phase, use the first graph element of the first batch as a key to query and obtain the first subgraph of the first batch from the target storage area; and train the graph neural network based on the first subgraph of the first batch.

[0116] According to another aspect of the embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in any of the above embodiments.

[0117] According to yet another embodiment, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in any one of the above embodiments is implemented.

[0118] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0119] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0120] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0121] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0122] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for training a graph neural network, comprising: In the offline stage, each computing device in the distributed system uses multiple graph elements in the local shard data of the knowledge graph as sampling centers, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs; the graph elements include nodes and / or relationship edges; Each computing device uses each graph element as a key and a subgraph corresponding to each graph element as a value, and stores the multiple subgraphs in a target storage area in the form of key-value pairs; In the training phase, the training device uses the first graph element of the first batch as a key to query and obtain the first subgraph of the first batch from the target storage area; The training device trains the graph neural network based on the first subgraph of the first batch.

2. According to the method of claim 1, before performing distributed subgraph sampling on the knowledge graph, the method further comprises: In the offline phase, the master device in the distributed system divides the knowledge graph into multiple graph shards. The master device distributes each graph slice to each computing device as local slice data of each computing device.

3. The method according to claim 1, wherein: The multiple graph elements include a first node; each computing device uses multiple graph elements in the local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including: The first computing device having the first node takes the first node as a sampling center, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first node.

4. The method according to claim 1, wherein: The multiple graph elements include a first relationship edge; each computing device uses the multiple graph elements in the local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including: The second computing device having the first relationship edge respectively takes the two nodes connected by the first relationship edge as sampling centers, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first relationship edge.

5. The method according to claim 1, wherein: The first graph element is a node; The training device trains the graph neural network based on the first subgraph of the first batch, including: The training device aggregates information on each first subgraph respectively to obtain a first graph representation corresponding to each node of the first batch; The training device uses the node attributes of each node as a training label and trains the graph neural network according to each first graph representation.

6. The method according to claim 1, wherein: The first graph element is a relationship edge; The training device trains the graph neural network based on the first subgraph of the first batch, including: The training device aggregates information on each first subgraph respectively to obtain a second graph representation of each second relationship edge of the first batch; The training device uses the edge attributes of each relationship edge as a training label and trains the graph neural network according to each second graph representation.

7. The method according to claim 1, further comprising: In the training phase, the training device trains the graph neural network based on the first subgraph of the first batch, and uses the second graph element of the second batch as a key to query the second subgraph of the second batch from the target storage area and store it in its own target cache area; The training device reads the second subgraph of the second batch from the target buffer area, and trains the graph neural network based on the second subgraph of the second batch.

8. The method according to claim 5, wherein: The knowledge graph includes several knowledge points related to users, commodities and / or transactions; the node attributes include: user attributes or commodity attributes.

9. The method according to claim 6, wherein: The knowledge graph includes several knowledge points related to users, commodities and / or transactions; the edge attributes include: transaction attributes.

10. A method for obtaining and storing training data, comprising: In the offline stage, each computing device in the distributed system uses multiple graph elements in the local shard data of the knowledge graph as sampling centers, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs; the graph elements include nodes and / or relationship edges; Each computing device uses each graph element as a key and the subgraph corresponding to each graph element as a value, and stores the multiple subgraphs in the form of key-value pairs in a target storage area; the multiple subgraph data are used to train the graph neural network.

11. According to the method of claim 10, before performing distributed subgraph sampling on the knowledge graph, the method further comprises: In the offline phase, the master device in the distributed system divides the knowledge graph into multiple graph shards. The master device distributes each graph slice to each computing device as local slice data of each computing device.

12. The method according to claim 10, wherein: The multiple graph elements include a first node; each computing device uses multiple graph elements in the local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including: The first computing device having the first node takes the first node as a sampling center, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first node.

13. The method according to claim 10, wherein: The multiple graph elements include a first relationship edge; each computing device uses the multiple graph elements in the local shard data of the knowledge graph as a sampling center, performs distributed subgraph sampling on the knowledge graph, and obtains multiple subgraphs, including: The second computing device having the first relationship edge respectively takes the two nodes connected by the first relationship edge as sampling centers, performs k-order neighbor sampling based on local shard data, and obtains a subgraph corresponding to the first relationship edge.

14. A system for training a graph neural network, comprising a distributed system and a training device, wherein: Each computing device in the distributed system is used to, in an offline stage, use multiple graph elements in the local shard data of the knowledge graph as sampling centers to perform distributed subgraph sampling on the knowledge graph to obtain multiple subgraphs; the graph elements include nodes and / or relationship edges; Each computing device is further used to store the plurality of subgraphs in a target storage area in the form of key-value pairs, using each graph element as a key and a subgraph corresponding to each graph element as a value; The training device is used to, during the training phase, use the first graph element of the first batch as a key to query and obtain the first subgraph of the first batch from the target storage area; and train the graph neural network based on the first subgraph of the first batch.

15. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 13.

16. A computing device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 13 is implemented.

Images