Training methods, training systems, and storage media for graph neural networks

Abstract

This specification provides a training method, training system, and storage medium for a graph neural network. The training method for the graph neural network includes: the training system determining a target graphics processor (GPU) corresponding to a sampling task based on the label of the starting vertex of the sampling task; wherein the root queue of the target GPU includes at least one root vertex, the label of which is the same as the label of the starting vertex; the at least one root vertex includes hot vertices; the feature data of the hot vertices is cached in the local cache of the target GPU; and the sampling probability of the hot vertices satisfies a preset condition. Furthermore, the training system can utilize the target GPU to execute the sampling task based on the feature data of the hot vertices in the local cache to train the graph neural network.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a training method, training system and storage medium for graph neural networks. Background Technology

[0002] Graph Neural Networks (GNNs) are deep learning models capable of processing graph data. GNNs have broad application prospects in multiple fields. Before using GNNs to handle related tasks in these fields, they need to be trained on graph data from the target domain. During training, the training system needs to sample from the graph data of the target domain and iteratively update the GNN multiple times based on the sampled feature data, resulting in a GNN capable of performing tasks related to the target domain.

[0003] Currently, to reduce the computational burden during training, training systems typically employ a mini-batch sampling method. This involves the training system sampling feature data from a subset of vertices in the target domain's graph data each time, and then using this sampled data for training. Specifically, when sampling from the target domain's graph data, the training system needs to load the sampled vertex feature data from the memory corresponding to the Central Processing Unit (CPU) into the Graphics Processing Unit (GPU).

[0004] However, due to the constraints of communication protocols and physical bandwidth on the data transfer rate between the CPU and GPU, a large amount of data transfer will result in a long transmission time, which in turn will affect the training efficiency of graph neural networks.

[0005] The information in the background section is merely information known only to the inventor and does not imply that such information had entered the public domain before the date of this application, nor does it imply that it can be considered prior art in this disclosure. Summary of the Invention

[0006] This specification provides a training method, training system, and storage medium for graph neural networks. It can preload the feature data corresponding to hot vertices in the target graph data into the cache of the target graphics processor, and perform sampling tasks based on the feature data of hot vertices in the local cache of the target graphics processor when training the graph neural network using the target graphics processor, thereby improving the training efficiency of the graph neural network.

[0007] To achieve the above objectives, the embodiments in this specification adopt the following technical solutions: In a first aspect, this specification provides a training method for a graph neural network, comprising: determining a target graphics processor corresponding to a sampling task based on the label of the starting vertex of the sampling task, wherein the root queue of the target graphics processor includes: at least one root vertex, the label of the at least one root vertex being the same as the label of the starting vertex, the at least one root vertex including: hot vertices, the feature data of the hot vertices being cached in the local cache of the target graphics processor, and the sampling probability of the hot vertices satisfying a preset condition; and using the target graphics processor to execute the sampling task based on the feature data of the hot vertices in the local cache to train the graph neural network.

[0008] In some embodiments, the at least one root vertex further includes a cold vertex with the same label as the hot vertex. The label of the cold vertex is determined based on a preset label propagation algorithm. The correlation between cold vertices and hot vertices with the same label is greater than the correlation between cold vertices and hot vertices with different labels. The feature data of the cold vertex is stored in the storage space corresponding to the central processing unit.

[0009] In some embodiments, the step of using the target graphics processor to perform the sampling task based on the feature data of the hot vertices in the local cache includes: using the target graphics processor to obtain the feature data of the target hot vertices from the local cache, obtaining the feature data of the target cold vertices from the storage space corresponding to the central processing unit, and performing the sampling task based on the feature data of the target hot vertices and the feature data of the target cold vertices, wherein the target hot vertices and the target cold vertices are i-hop vertices of the starting vertex in the target graph data, and i is an integer greater than or equal to 1.

[0010] In some embodiments, before obtaining the feature data of target hot vertices from the local cache using the target graphics processor and obtaining the feature data of target cold vertices from the storage space corresponding to the central processing unit, the method further includes: determining whether all target hot vertices are included in the root queue of the target graphics processor; when not all target hot vertices are included in the root queue of the target graphics processor, the method further includes: determining other graphics processors corresponding to other target hot vertices based on the labels of other target hot vertices not included in the root queue of the target graphics processor, and obtaining the feature data of other target hot vertices from the local cache of the other graphics processor.

[0011] In some embodiments, the target graphics processor is one of a graphics processor group, which includes N graphics processors, where N is an integer greater than or equal to 1. Before determining the target graphics processor corresponding to the sampling task based on the label of the starting vertex of the sampling task, the method further includes: identifying hot vertices in the target graph data to obtain hot vertex graph data corresponding to the hot vertices; and dividing the hot vertex graph data into N subgraphs under a first preset condition, wherein the hot vertices in each subgraph have the same label, and loading the feature data corresponding to the hot vertices in the N subgraphs into the local caches of the N graphics processors respectively.

[0012] In some embodiments, the method further includes: for any graphics processor: obtaining the root vertex corresponding to the graphics processor, the root vertex including: hot vertices in the subgraph corresponding to the graphics processor, and cold vertices with the same label as the hot vertices; and adding the root vertex to the root queue of the graphics processor.

[0013] In some embodiments, the method further includes: determining the label of the cold vertex based on the preset label propagation algorithm and the labels of each hot vertex.

[0014] In some embodiments, the target graph data includes multiple vertices and their respective topological relationships; identifying hot vertices in the target graph data and obtaining hot vertex graph data corresponding to the hot vertices includes: obtaining the degree of each vertex in the target graph data according to the respective topological relationships of the multiple vertices; determining hot vertices based on the degree of each vertex in the target graph data; and obtaining the hot vertex graph data based on the hot vertices and their corresponding topological relationships.

[0015] In some embodiments, determining hot vertices based on the degree of each vertex in the target graph data includes: determining vertices whose degree satisfies a second preset condition as hot vertices, wherein the second preset condition includes: determining the top k vertices by degree as hot vertices, where k is an integer greater than or equal to 1; or, determining vertices whose degree is greater than a preset threshold as hot vertices.

[0016] In some embodiments, dividing the hot vertex graph data into N subgraphs under the constraint of a first preset condition includes: determining the target weight of each hot vertex according to its degree; and dividing the hot vertex graph data into N subgraphs under the constraint of the first preset condition according to the target weight of each hot vertex and the topological relationship between the hot vertices.

[0017] In some embodiments, the first preset condition includes one or more of the following: the number of hot vertices in each subgraph is the same or similar; the number of hot vertices with a topological relationship between two subgraphs is minimized; and the sum of the target weights of all hot vertices in each subgraph is the same or similar.

[0018] In some embodiments, the degree corresponding to the hot vertex is the initial weight of the hot vertex; determining the target weight corresponding to each hot vertex according to the degree corresponding to each hot vertex includes: obtaining a plurality of initial weight sets corresponding to the hot vertex according to the degree corresponding to each hot vertex; and normalizing the initial weight sets to map the initial weights in the initial weight sets to the target weight interval to obtain the target weights corresponding to each of the hot vertexes.

[0019] Secondly, this specification also provides a training system, comprising: at least one storage medium storing at least one instruction set for training a graph neural network; and at least one processor communicatively connected to the at least one storage medium, wherein, when the training system is running, the at least one processor reads the at least one instruction set and implements the method provided in the first aspect according to the instructions of the at least one instruction set.

[0020] Thirdly, this specification also provides a computer-readable non-volatile storage medium, wherein the computer-readable non-volatile storage medium stores at least one instruction set, which, when executed by at least one processor, implements the method provided in the first aspect.

[0021] Other features of the graph neural network training methods, training systems, and storage media provided in this specification will be partially listed in the following description. The inventive aspects of the graph neural network training methods, training systems, and storage media provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A schematic diagram illustrating an application scenario of the graph neural network training method provided according to embodiments of this specification is shown. Figure 2A hardware structure diagram of a computing system provided according to an embodiment of this specification is shown; Figure 3 A flowchart of a training method for a graph neural network according to an embodiment of this specification is shown; Figure 4 A schematic diagram of target image data provided according to an embodiment of this specification is shown; Figure 5 A schematic diagram is shown illustrating the determination of hot vertex graph data based on target graph data in an embodiment provided according to this specification; Figure 6 A schematic diagram illustrating the partitioning of a graph based on hot vertex graph data according to an embodiment of this specification is shown. Figure 7 A schematic diagram illustrating the propagation of labels from hot vertices to cold vertices according to an embodiment of this specification is shown. Figure 8 A schematic diagram illustrating the allocation of root vertices to a graphics processor according to an embodiment provided in this specification is shown; and Figure 9 A schematic diagram is shown illustrating the performance of a sampling task using a target graphics processor according to an embodiment provided in this specification. Detailed Implementation

[0024] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0025] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0026] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0027] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0028] For ease of description, the terms that will appear later in this manual will be explained first.

[0029] Term 1: Graph Neural Network. A Graph Neural Network (GNN) is a deep learning model used to process graph data. It learns low-dimensional vector representations (i.e., "embedded representations") that capture the topological structure and feature information of the graph data by passing information between the vertices. These captured "embedded representations" can then be used to perform downstream tasks such as vertex classification and link prediction.

[0030] Term 2: Graph Data. Graph data is structured data stored in units of vertices (nodes) and edges. The attributes of vertices in graph data can be the feature data corresponding to that vertex, while the edges represent the topological relationships between two vertices. Edges can be directed (i.e., in a directed graph) or undirected (i.e., in an undirected graph).

[0031] Term 3: Hot Vertex. A hot vertex is a vertex in graph data that has high connectivity (e.g., height number) and is frequently accessed during sampling tasks. In other words, a hot vertex is a vertex in graph data with a high probability of being accessed (sampled).

[0032] Term 4: Cold vertex. In graph data, a cold vertex is a vertex that is visited (sampled) less frequently than a hot vertex. In some cases, a cold vertex can be any other vertex in the graph data besides a hot vertex.

[0033] Term 5: Root vertex. This refers to the starting vertex of each mini-batch sampled during the training of a GNN based on a mini-batch sampling task. The training objective of a GNN is to compute the final embedding representation for each root vertex.

[0034] Term 6: Root Queue. This is a queue that stores the root vertices to be processed. In this scheme, each GPU has a corresponding root queue, containing root vertices with a high affinity to the data cached by that GPU.

[0035] Term 7: Affinity. In a multi-GPU environment, this refers to the degree of physical correlation between a sampling task executed on a GPU (i.e., the root vertex being processed) and the data required by the sampling task (such as the i-hop node of the root vertex). Higher affinity indicates a higher proportion of the data required by the sampling task is stored in the local cache of the GPU executing the sampling task.

[0036] Term 8: Data Locality. This refers to the fact that when a GPU performs computations, the data it needs is stored in a physically close storage medium (such as a local cache rather than other GPU caches or CPU memory). High data locality can significantly reduce data access latency and is key to achieving high-performance computing. This solution improves data locality by enhancing "affinity".

[0037] Term 9: Hot Vertex Graph Data. Hot vertex graph data, often simply called a heatmap, refers to graph data consisting of all identified hot vertices and the topological relationships between them.

[0038] Term 10: Label Propagation Algorithm. A semi-supervised algorithm for community detection (a "community" is a set of vertices with the same label) in graph data. For each vertex, the label propagation algorithm updates the vertex's label based on the labels of its neighboring vertices. In this specification, the label propagation algorithm can be used to propagate the labels of hot vertices into the graph data, thereby classifying cold vertices in the graph data into their corresponding hot vertices, thus extending the community structure from the hot graph to the entire graph.

[0039] The following section introduces the application scenarios of this manual.

[0040] The technical solutions provided in this specification are applicable to scenarios involving training graph neural networks. For example, when a training system uses a mini-batch sampling task to train a GNN, the system needs to perform a sampling task once for each vertex in the graph data. Each vertex, when used as the initial vertex for the sampling task, becomes a root vertex. The training system calls the GPU to sample the root vertex and its neighbors to obtain feature data for multiple vertices. Then, based on the feature data of multiple vertices, the training system calculates the embedding representation corresponding to the root vertex. Finally, after multiple iterations, the training system can obtain the final embedding representation of each root vertex based on the graph data, thus obtaining the trained GNN.

[0041] In related approaches, the training system needs to sample from graph data. Since graph data is large in volume, it is typically stored in the CPU's RAM. This means that each time the training system performs a sample, it needs to load the feature data of the sampled vertices from the CPU's RAM to the GPU. However, because the data transfer rate between the CPU and GPU is constrained by communication protocols and physical bandwidth, large data transfers result in long transmission times, thus affecting the training efficiency of the graph neural network.

[0042] Therefore, this specification provides a training method for a graph neural network, which can be executed by a training system. The method includes: the training system determining a target graphics processor (GPU) corresponding to the sampling task based on the label of the starting vertex of the sampling task; wherein the root queue of the target GPU includes at least one root vertex, the label of which is the same as the label of the starting vertex; the at least one root vertex includes hot vertices, the feature data of which is cached in the local cache of the target GPU; and the sampling probability of which is hot vertices satisfies a preset condition. Furthermore, the training system can utilize the target GPU to execute the sampling task based on the feature data of the hot vertices in the local cache to train the graph neural network.

[0043] The solution provided in this specification allows the training system to pre-load the feature data corresponding to hot vertices in the target graph data into the cache of the target graphics processor (GPU). When training the graph neural network using the GPU, the system performs sampling tasks based on the feature data of the hot vertices in the GPU's local cache. In this way, the training system can directly and quickly read the feature data of frequently accessed vertices from the GPU's local cache, reducing latency and bandwidth consumption caused by retrieving data from CPU memory or other GPU caches. This effectively reduces data transmission overhead, thereby improving data access efficiency and overall training speed during graph neural network training.

[0044] Figure 1 A schematic diagram illustrating an application scenario of the graph neural network training method provided according to embodiments of this specification is shown. For example... Figure 1 As shown, the application scenario 100 may include a training system 11. The training system 11 includes at least one graphics processor (GPU), and the GPU's local cache stores hot vertices. When the root vertex in the GPU's root queue is used as the starting vertex of a sampling task, the feature data corresponding to the hot vertex is read from the GPU's local cache for training the graph neural network.

[0045] refer to Figure 1The training system 11 can be deployed on a device or cluster of devices with data processing capabilities. For example, the training system 11 can be deployed on physical devices such as servers, server clusters, and cloud servers. In this case, the physical device corresponding to the training system 11 can store data or instructions for executing the training method of the graph neural network described in this specification, and can execute or be used to execute the data or instructions. In some embodiments, the training system 11 may include hardware devices with data information processing capabilities and the necessary programs required to drive the hardware devices to work. The training system 11 may include multiple graphics processors (GPUs), which can collaborate through corresponding group communication channels and protocols to train the graph neural network. This specification does not limit the number of GPUs or the group communication protocol in the training system 11.

[0046] Figure 2 A hardware structure diagram of a computing system provided according to an embodiment of this specification is shown. The computing system 200 can serve as... Figure 1 The training system 11 in this specification performs the training method for the graph neural network described herein.

[0047] like Figure 2 As shown, the computing system 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, the computing system 200 may also include a communication port 250 and an internal communication bus 210. The computing system 200 may also include I / O components 260.

[0048] The internal communication bus 210 can connect to different system components. For example, the internal communication bus 210 can connect to storage medium 230, processor 220, communication port 250, and I / O component 260, etc.

[0049] I / O component 260 supports input / output between computing system 200 and other components.

[0050] Communication port 250 is used for data communication between computing system 200 and the outside world. For example, communication port 250 can be used for data communication between computing system 200 and a network. Communication port 250 can be a wired communication port or a wireless communication port.

[0051] Storage medium 230 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 235. Storage medium 230 also includes at least one instruction set stored in the data storage device. The instruction set may include computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc.

[0052] At least one processor 220 may be communicatively connected to at least one storage medium 230. When the computing system 200 is running, at least one processor 220 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the graph neural network training method provided in this specification. Processor 220 may execute the steps included in the graph neural network training method. Processor 220 may be in the form of one or more processors. In some embodiments, processor 220 may include one or more hardware processors, such as microcontrollers, microprocessors, reduced instruction set computers (RISC), application-specific integrated circuits (ASICs), application-specific instruction set processors (ASIPs), central processing units (CPUs), graphics processing units (GPUs), physical processing units (PPUs), microcontroller units, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), advanced RISC machines (ARMs), programmable logic devices (PLDs), any circuit or processor capable of performing one or more functions, or any combination thereof. For example, in this specification, processor 220 may include at least one or more GPUs and a CPU.

[0053] For illustrative purposes only, the accompanying drawings show only one processor 220 for the computing system 200. However, it should be noted that the computing system 200 may also include multiple processors; therefore, the operations and / or method steps disclosed herein may be executed by one processor or by multiple processors in combination. For example, if the processor 220 of the computing system 200 described in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 220 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0054] Figure 3 A flowchart illustrating a graph neural network training method according to an embodiment of this specification is shown. As previously described, the computing system 200 can serve as a training system to execute the graph neural network training method.

[0055] like Figure 3 As shown, training methods for graph neural networks can include: S310: Based on the label of the starting vertex of the sampling task, determine the target graphics processor corresponding to the sampling task, wherein the root queue of the target graphics processor includes: at least one root vertex, the label of at least one root vertex is the same as the label of the starting vertex, the at least one root vertex includes: hot vertices, the feature data of the hot vertices are cached in the local cache of the target graphics processor, and the sampling probability of the hot vertices meets the preset conditions.

[0056] In some embodiments, the training system includes multiple graphics processing units (GPUs), which form a GPU group. The GPUs within the GPU group are physically connected and interact with each other via appropriate communication protocols. For example, the GPUs may be NVIDIA H-series GPUs. In this case, the multiple GPUs can be physically connected based on a Server PCI Express Module (SXM) architecture and interact with each other via the NVIDIA Connect (NVlink) protocol.

[0057] As an example, the target graphics processor is one of a group of graphics processors. Assume the group contains N graphics processors, where N is an integer greater than or equal to 1. Before determining the target graphics processor in the group to perform the sampling task, the training system can first identify hot vertices in the target graph data, obtaining the corresponding hot vertex graph data. Then, the training system can divide the hot vertex graph data into N subgraphs, each with hot vertices having the same label, and load the feature data corresponding to the hot vertices in the N subgraphs into the local caches of the N graphics processors respectively.

[0058] In some embodiments, the target graph data may be pre-collected graph data used to train a graph neural network. The target graph data includes multiple vertices and the corresponding topological relationships of each vertex.

[0059] Figure 4 A schematic diagram of target image data provided according to an embodiment of this specification is shown.

[0060] refer to Figure 4The target graph data is an undirected graph, including multiple vertices (numbered 0-7) and the topological relationships (i.e., edges) between these vertices. When the training system identifies hot vertices in the target graph data, it can obtain the degree of each vertex based on the corresponding topological relationships. Then, the training system can determine the hot vertices based on their degrees and obtain the hot vertex graph data based on the hot vertices and their corresponding topological relationships. A higher degree for a vertex indicates a greater probability that it will be an intermediate vertex, a sampling path point, or a sampling endpoint; that is, the sampling probability of a vertex is positively correlated with its degree.

[0061] In some embodiments, when the target graph data is an undirected graph, for any vertex: the training system can obtain the number of topological relations that the vertex has and use the number of topological relations as the degree of the vertex.

[0062] In some embodiments, when the target graph data is a directed graph, for any vertex: the training system can obtain the in-degree of the vertex and use the in-degree as the degree of the vertex. Alternatively, the training system can obtain the out-degree of the vertex and use the out-degree as the degree of the vertex.

[0063] For example, in target graph data, vertices with high out-degree connect to more downstream vertices. In a sampling task, vertices with high out-degree have a higher probability of acting as intermediate vertices connecting to other vertices. In this case, vertices with high out-degree are also more likely to be sampled; that is, out-degree can be used as the degree of a vertex to identify hot vertices in the target graph data.

[0064] For example, in target graph data, vertices with high in-degree are connected by more upstream vertices. In sampling tasks, vertices with high in-degree have a higher probability of becoming the convergence point or endpoint of multiple paths. In this case, vertices with high in-degree are also more likely to be sampled; that is, in-degree can be used as the degree of a vertex to identify hot vertices in the target graph data.

[0065] Figure 5 A schematic diagram is shown illustrating the determination of hot vertex graph data based on target graph data in an embodiment provided according to this specification.

[0066] refer to Figure 5 The target graph data is an undirected graph, where the degree of each vertex is the number of topological relationships it has. The training system can be based on... Figure 5 The target image data is statistically analyzed and ranked to obtain the following data (displayed in tabular form in this embodiment; however, the data storage method is not limited in actual applications): Table 1: Vertex Degree Sort Table

[0067] In this table, vertices 4 and 2 both have a degree of 2, and vertices 0 and 5 both have a degree of 1. The training system can randomly sort vertices with the same degree, or sort them according to the order in which they were counted, or sort them in descending or ascending order based on their vertex numbers. This specification does not impose any restrictions on this. The sorting of vertices 4 and 2, and the sorting of vertices 0 and 5 in Table 1 are for illustrative purposes only and are not intended to be limiting.

[0068] In some embodiments, the training system may determine vertices whose degree satisfies a second preset condition as hot vertices. The second preset condition includes: the training system determines the top k vertices by degree as hot vertices, where k is an integer greater than or equal to 1. Alternatively, the training system determines vertices with a degree greater than a preset threshold as hot vertices.

[0069] As an example, the training system can determine hot vertices based on the sorting in Table 1 and according to the second preset conditions.

[0070] For example, the training system can determine k vertices as hot vertices based on the number of graphics processors (GPUs), meaning the value of k can be positively correlated with the number of GPUs. For instance, suppose the training system needs to allocate 2 hot vertices to each GPU. When the number of GPUs is 2, k is 4, meaning the first 4 vertices in Table 1: vertex 6, vertex 1, vertex 3, and vertex 4 are all determined as hot vertices by the training system.

[0071] For example, the training system can identify vertices with a high sampling probability (i.e., vertices with a degree greater than a preset threshold) as hot vertices. For example, when the preset threshold is 2, all vertices in Table 1 with a degree greater than or equal to 2: vertex 6, vertex 1, vertex 3, vertex 4, and vertex 2 are identified as hot vertices by the training system.

[0072] refer to Figure 5 If vertex 6, vertex 1, vertex 3 and vertex 4 in the target graph data are all identified as hot vertices by the training system, then the training system can generate hot vertex graph data based on the topological relationship between these four hot vertices.

[0073] exist Figure 5 In the hot vertex graph data, the topological relationships between hot vertices are represented by solid lines, while the remaining vertices and the topological relationships between hot vertices are represented by dashed lines. Multiple vertices represented by dashed lines (such as vertex 0, vertex 2, and vertex 5) can also be called cold vertices; that is, cold vertices are the vertices in the target graph data other than the hot vertices.

[0074] In some embodiments, the training system can determine the target weight corresponding to each hot vertex based on the degree corresponding to each hot vertex.

[0075] As an example, the training system can use the degree corresponding to a hot vertex as its initial weight, and obtain multiple sets of initial weights for each hot vertex based on its degree. Then, the training system can normalize these initial weight sets, mapping them to target weight intervals to obtain the target weights for each hot vertex.

[0076] For example, referring to Table 1, where the hot vertices are vertex 6, vertex 1, vertex 3, and vertex 4, the set of hot vertices corresponding to the target graph data is:

[0077] The initial weight set corresponding to the hot vertex set is:

[0078] in, This represents the i-th vertex. This represents the initial weight corresponding to the i-th vertex, referring to Table 1, where i ∈ (1, 3, 4, 6). That is, , , , .

[0079] As an example, the target weight interval can be the (0, 1) interval. The training system can map the initial weights in the initial weight set to the target weight interval through the following steps to obtain the target weights corresponding to each of the multiple hot vertices: First, the training system can calculate Standard deviation of all hot vertices and variance Among them, standard deviation and variance It can be calculated using the following formula:

[0080]

[0081] Then, the training system can process the initial weights using z-score normalization to obtain the z-score corresponding to each hot vertex. , where z-score The calculation formula is as follows:

[0082] Finally, the training system can use the tanh function to normalize the z-score, scaling it to the (0, 1) interval, to obtain the target weight corresponding to each hot vertex. Among them, the target weight The calculation formula is as follows:

[0083] In some embodiments, the training system can divide the hot vertex graph data into N subgraphs based on the target weights corresponding to each hot vertex and the topological relationships between hot vertices, constrained by a first preset condition.

[0084] The first preset condition includes one or more of the following: the number of hot vertices in each subgraph is the same or similar; the number of hot vertices with topological relationships between two subgraphs is minimized; and the sum of the target weights of all hot vertices in each subgraph is the same or similar.

[0085] As an example, the training system can use the METIS multi-constraint graph segmentation algorithm to divide the hot vertex graph data into subgraphs equal to the number of GPUs, with a first preset condition as a constraint, and assign the same label to vertices in the same subgraph.

[0086] Figure 6 A schematic diagram of subgraph partitioning based on hot vertex graph data is shown according to an embodiment of this specification.

[0087] The first precondition is that the number of hot vertices in each subgraph is the same or similar, the number of hot vertices with topological relationships between two subgraphs is minimized, and the sum of the target weights of all hot vertices in each subgraph is the same or similar.

[0088] refer to Figure 6 The hot vertex graph data includes 4 hot vertices. Based on the constraint that the number of hot vertices in each subgraph is the same or similar, the hot vertex graph data can be divided into 4 subgraphs (each subgraph...) by the training system. Figure 1 One hot vertex) or two subgraphs (each subgraph has one ... Figure 2 (Two hot vertices). When the graphics processing unit includes two graphics processors (i.e., N=2), the training system can divide the hot vertex graph data into two subgraphs, each containing two hot vertices.

[0089] If the sum of the target weights of vertices 1 and 3 is the same as the sum of the target weights of vertices 4 and 6, then the training system can assign vertices 1 and 3 to one subgraph and assign vertices 4 and 6 to another subgraph.

[0090] Finally, the training system can determine the number of hot vertices that have a topological relationship between two subgraphs based on the principle of minimizing the number of such vertices. Figure 6 The subgraph dividing lines are shown in the figure, and the range of each of the two subgraphs is finally determined based on the subgraph dividing lines.

[0091] In this embodiment, the training system constrains the segmentation of the subgraph using a first preset condition, resulting in a more balanced distribution of data volume and weights in the segmented subgraphs. This reduces the load differences between different graphics processors, thereby improving hardware resource utilization and overall computational stability. Furthermore, the constraint of the first preset condition minimizes the topological relationships between boundary vertices of the subgraphs, thus reducing the frequency of cross-graphics processor communication, minimizing latency caused by data synchronization and transmission during training, and improving the training efficiency of the graph neural network.

[0092] In some embodiments, after obtaining two subgraphs, the training system can generate a corresponding label for each subgraph, for example, referring to... Figure 6 The subgraph containing vertices 1 and 3 can be labeled "Label 1", and the subgraph containing vertices 4 and 6 can be labeled "Label 2".

[0093] In some embodiments, after obtaining the subgraphs corresponding to hot vertices and their corresponding labels, the training system can further propagate labels based on these subgraphs, spreading the subgraph labels to the entire target graph data and adding labels to cold vertices in the target graph data. The labels of the cold vertices can be determined by the training system through a preset label propagation algorithm that propagates the labels of the hot vertices. The correlation between cold and hot vertices with the same label is greater than the correlation between cold and hot vertices with different labels.

[0094] Figure 7 A schematic diagram illustrating the propagation of labels from hot vertices to cold vertices according to an embodiment of this specification is shown.

[0095] In some embodiments, the training system can determine the label of a cold vertex based on a preset label propagation algorithm and the labels of each hot vertex.

[0096] As an example, see reference Figure 7 Vertices 1 and 3 are hot vertices in the same subgraph, labeled "Label 1". Vertices 4 and 6 are also hot vertices in the same subgraph, labeled "Label 2". The training system can use a pre-defined label propagation algorithm to propagate the labels of vertices 1 and 3 to vertex 0, and the labels of vertices 4 and 6 to vertices 2 and 5. After label propagation, the target graph data includes two subgraphs: one subgraph contains two hot vertices (vertices 1 and 3) and one cold vertex (vertices 0), with all vertices in this subgraph labeled "Label 1". The other subgraph contains two hot vertices (vertices 4 and 6) and two cold vertices (vertices 2 and 5), with all vertices in this subgraph labeled "Label 2".

[0097] In this embodiment, the training system uses a label propagation algorithm to spread the labels of hot vertices to cold vertices in the target graph data, assigning a corresponding label to each cold vertex. The correlation between hot and cold vertices with the same label is greater than the correlation between hot and cold vertices with different labels. By propagating labels to cold vertices, vertices within each subgraph have high correlation, while vertices in different subgraphs have low correlation. In this way, the training system improves the affinity of data in subgraphs, thereby enhancing the data locality of vertices in the subgraphs. That is, when acquiring feature data corresponding to cold vertices, the traversal range can be reduced by using labels, decreasing the retrieval time when reading feature data corresponding to cold vertices, improving data acquisition efficiency, and thus improving the training efficiency of the graph neural network.

[0098] In some embodiments, when the training system trains the graph neural network based on the target graph data, it can perform a sampling task using each vertex in the target graph data as a starting vertex. After the training system traverses the target graph data, performs the sampling task with each vertex as a starting vertex, and calculates the embedding representation of the starting vertex based on the feature data collected by the sampling task, one round of training on the target graph data is completed. The sampling task and the calculation of the embedding representation are performed using a graphics processing unit (GPU). The GPU maintains a root queue, which includes at least one root vertex.

[0099] In some embodiments, for any graphics processor: the training system can obtain the root vertex corresponding to the graphics processor, the root vertex including: hot vertices in the subgraph corresponding to the graphics processor, and cold vertices with the same label as the hot vertices; and add the root vertex to the root queue of the graphics processor, and load the feature data corresponding to the hot vertices in the root queue into the local cache of N graphics processors.

[0100] Figure 8 A schematic diagram is shown illustrating the allocation of root vertices to a graphics processor according to an embodiment provided in this specification.

[0101] refer to Figure 8 Assuming there are two graphics processors (GPUs), the training system can treat every vertex in each subgraph as the root vertex corresponding to one GPU and add it to the root queue of that GPU, ensuring that the root vertices in the root queue have the same label. For example, refer to... Figure 8 The training system can add vertices 0, 1, and 3 from the target graph data as root vertices to the root queue of graphics processor 1. It can also add vertices 2, 4, 5, and 6 from the target graph data as root vertices to the root queue of graphics processor 2.

[0102] When the training system adds the root vertex to the root queue, it can add the root vertices in an out-of-order manner. For example, refer to... Figure 8 When the training system adds vertex 0, vertex 1, and vertex 3 to the root queue of graphics processor 1, it can randomly sort vertex 0, vertex 1, and vertex 3. If the random sorting order is "vertex 3, vertex 0, and vertex 1", then the training system can add the root vertex to the root queue in the order of vertex 3, vertex 0, and vertex 1.

[0103] The training system can then load the feature data corresponding to the hot vertices in the root queue into the local cache of the corresponding graphics processor. For example, see [reference needed]. Figure 8 The target graph data is stored in the CPU's corresponding storage space (such as RAM). If vertices 1 and 3 are hot vertices in the root queue of GPU 1, the training system can retrieve the feature data corresponding to vertices 1 and 3 from the CPU's storage space and store it in GPU 1's local cache. If vertices 4 and 6 are hot vertices in the root queue of GPU 2, the training system can retrieve the feature data corresponding to vertices 4 and 6 from the CPU's storage space and store it in GPU 2's local cache.

[0104] Once the training system has finished loading, the hot vertices in the target graph data are assigned to multiple graphics processors according to their labels. Each graphics processor's cache loads the feature data of the corresponding hot vertices. The feature data corresponding to the cold vertices in the root vertex is stored in the same location as the target graph data, i.e., in the CPU's memory space.

[0105] In some embodiments, when performing a sampling task, the training system can determine the corresponding target graphics processor based on the label of the starting vertex of the sampling task. For example, refer to... Figure 8 When the starting vertex of the sampling task is vertex 0, vertex 0 is labeled "Label 1". Since the root vertices stored in the root queue of graphics processor 1 are all labeled "Label 1", graphics processor 1 can be identified as the target graphics processor for the sampling task. When the starting vertex of the sampling task is vertex 6, vertex 6 is labeled "Label 2". Since the root vertices stored in the root queue of graphics processor 2 are all labeled "Label 2", graphics processor 2 can be identified as the target graphics processor for the sampling task.

[0106] In this embodiment, the training system assigns vertices with the same label to the root queue of the same graphics processor (GPU) as root vertices, and loads the feature data corresponding to the hot vertices in the root vertices into the GPU's local cache. During training, the training system can determine the corresponding target GPU based on the label of the starting vertex of the sampling task. The target GPU's cache pre-loads the feature data of the hot vertices that the sampling task may sample. When executing the sampling task, the target GPU can directly read the feature data of the hot vertices from its local cache, reducing the probability of obtaining data from the CPU's corresponding storage space, thereby reducing latency and bandwidth usage caused by data transmission. This effectively reduces data handling overhead, improves the execution efficiency of the sampling task during graph neural network training, reduces training time, and increases training efficiency.

[0107] Furthermore, the training system, based on a first preset condition constraining subgraph segmentation, utilizes label propagation to enhance the correlation between cold and hot vertices within a subgraph. In this way, the training system improves the affinity and data locality of cached data in each graphics processor. This shortens the access path for high-frequency data (feature data of hot vertices) and significantly improves cache hit rate during graph neural network training. Simultaneously, the acquisition of low-frequency data (feature data of cold vertices) can utilize labels to reduce acquisition time, thereby collaboratively achieving efficient execution of the sampling task, effectively reducing global communication overhead, and ultimately improving resource utilization and training efficiency during graph neural network training.

[0108] S320: Utilizes the target graphics processor to perform a sampling task based on feature data of hot vertices in the local cache to train a graph neural network.

[0109] In some embodiments, the training system can load the feature data corresponding to the hot vertices into the local caches of multiple graphics processors based on the steps in S310. After loading is complete, the training system can use the graphics processors to execute sampling tasks to train the graph neural network. During this process, the training system can determine the target graphics processor corresponding to each sampling task based on the steps in S310. Then, the training system can determine whether all the target hot vertices involved in the sampling task are contained in the root queue of the target graphics processor.

[0110] In some embodiments, when all target hot vertices are contained in the root queue of the target graphics processor, the training system can utilize the target graphics processor to obtain feature data of the target hot vertices from the local cache and feature data of the target cold vertices from the storage space corresponding to the central processing unit. Then, the training system can perform a sampling task based on the feature data of the target hot vertices and the target cold vertices, where the target hot vertices and target cold vertices are i-hop vertices of the starting vertex in the target graph data, and i is an integer greater than or equal to 1.

[0111] Figure 9 A schematic diagram is shown illustrating the performance of a sampling task using a target graphics processor according to an embodiment provided in this specification.

[0112] As an example, see reference Figure 9 The example shown is Sampling Task 1. The starting node of Sampling Task 1 is vertex 0 (label 1), and the sampling range is 1-hop vertices (i.e., the 1-hop neighbors of the starting node). The training system can determine that the target graphics processor for Sampling Task 1 is graphics processor 1. Furthermore, based on the target graph data, the training system can determine that the target hot vertex is vertex 1 and the target cold vertex is vertex 0. Since all target hot vertices are contained in the root queue of graphics processor 1, the training system can use graphics processor 1 to obtain the feature data of vertex 1 from its local cache and the feature data of vertex 0 from the storage space corresponding to the central processing unit. Then, the training system can perform the sampling task based on the feature data of vertex 1 and vertex 0 to obtain the embedding representation corresponding to vertex 0.

[0113] In some embodiments, when the target hot vertices are not all contained in the root queue of the target graphics processor, the method further includes: determining other graphics processors corresponding to other target hot vertices based on the labels of other target hot vertices not contained in the root queue of the target graphics processor, and obtaining feature data of the other target hot vertices from the local cache of the other graphics processors.

[0114] When not all target hot vertices are contained in the root queue of the target graphics processor, the training system can determine the other graphics processors corresponding to the other target hot vertices based on the labels of the remaining vertices that are not included in the root queue of the target graphics processor, and obtain the feature data of the other target hot vertices from the local cache of the other graphics processors. Then, the training system can obtain the feature data of the target cold vertices from the storage space corresponding to the central processing unit, and perform a sampling task based on the feature data of the target hot vertices and the feature data of the target cold vertices.

[0115] As an example, see reference Figure 9The sampling task 2 is shown in the diagram. The starting node of sampling task 2 is vertex 6 (label 2), and the sampling range is 1-hop vertices (i.e., the 1-hop neighbors of the starting node). The training system can determine that the target graphics processor for sampling task 2 is graphics processor 2. Furthermore, based on the target graph data, the training system can determine that the target hot vertices are vertex 1, vertex 3, vertex 4, and vertex 6, and the target cold vertices are vertex 2 and vertex 5. Since not all target hot vertices are contained in the root queue of graphics processor 2, the training system can utilize graphics processor 2 to obtain the feature data of vertex 4 and vertex 5 from its local cache, and obtain the feature data of vertex 1 and vertex 3 from the local cache of graphics processor 1 through inter-graphics processor communication. Then, the training system can obtain the feature data of vertex 2 and vertex 5 from the storage space corresponding to the central processing unit. Finally, the training system can perform the sampling task based on the feature data of vertex 1, vertex 3, vertex 4, and vertex 6, as well as the feature data of vertex 2 and vertex 5, thereby obtaining the embedding representation corresponding to vertex 6.

[0116] In some implementations, when loading root vertices from the root queue, the training system can load the feature data corresponding to both hot and cold vertices in the root queue into the local cache of the graphics processor. In this case, the training system can obtain the feature data of cold vertices corresponding to the sampling task from the local cache of the target graphics processor, and obtain the feature data of missing cold vertices from the local caches of other graphics processors. In this way, the training system can reduce the probability of obtaining the feature data of cold vertices from the CPU's storage space, further improving the data access efficiency during the graph neural network training process.

[0117] In summary, the graph neural network training method provided in this specification allows the training system to pre-load the feature data corresponding to hot vertices in the target graph data into the cache of the target graphics processor (GPU). When training the graph neural network using the GPU, the system performs sampling tasks based on the feature data of the hot vertices in the GPU's local cache. In this way, the training system can directly and quickly read the feature data of frequently accessed vertices from the GPU's local cache, reducing latency and bandwidth consumption caused by retrieving data from CPU memory or other GPU caches. This effectively reduces data transmission overhead, thereby improving data access efficiency and overall training speed during graph neural network training.

[0118] Meanwhile, the target graph data obtained using graph neural network training methods is labeled with hot and cold vertices and their corresponding labels. This target graph data can be reused for training other graph neural networks or reused repeatedly. This not only improves the training efficiency of graph neural networks but also reduces their training cost and expands the application scope of target graph data.

[0119] This specification, in another aspect, provides a computer-readable non-transitory storage medium storing at least one instruction set for training a graph neural network. When the at least one instruction set is executed by a processor, it instructs the processor to perform the steps of the graph neural network training method described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing system 200, the program code causes the computing system 200 to perform the steps of the graph neural network training method described herein. The program product for implementing the above method may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on the computing system 200. However, the program product of this specification is not limited thereto. In this specification, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing system 200, partially on computing system 200, as a standalone software package, partially on computing system 200 and partially on a remote computing system, or entirely on a remote computing system.

[0120] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0121] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0122] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0123] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art, upon reading this specification, may readily identify some of the devices as separate embodiments. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. And the content of each secondary embodiment is valid even if it contains fewer than all the features of a single foregoing disclosed embodiment.

[0124] Every patent, patent application, publication of a patent application, and other material such as articles, books, specifications, publications, documents, articles, etc., cited herein, except for those inconsistent with or conflicting with this document, or those having a restrictive effect on the widest scope of the claims, may be incorporated herein by reference for all purposes now or hereafter associated with this document. Furthermore, in the event of any inconsistency or conflict between the description, definition, and / or use of relevant terms in any material and the description, definition, and / or use of relevant terms in this document, the terms in this document shall prevail.

[0125] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for training a graph neural network, the method comprising: Based on the label of the starting vertex of the sampling task, the target graphics processor corresponding to the sampling task is determined. The root queue of the target graphics processor includes at least one root vertex, the label of which is the same as the label of the starting vertex. The at least one root vertex includes hot vertices, the feature data of which are cached in the local cache of the target graphics processor, and the sampling probability of which is a hot vertex satisfies a preset condition. The target graphics processor performs the sampling task based on the feature data of the hot vertices in the local cache to train the graph neural network.

2. The method according to claim 1, wherein, The at least one root vertex also includes a cold vertex with the same label as the hot vertex. The label of the cold vertex is determined based on a preset label propagation algorithm. The correlation between cold vertices and hot vertices with the same label is greater than the correlation between cold vertices and hot vertices with different labels. The feature data of the cold vertex is stored in the storage space corresponding to the central processing unit.

3. The method according to claim 2, wherein performing the sampling task using the target graphics processor based on the feature data of the hot vertices in the local cache comprises: The target graphics processor obtains the feature data of the target hot vertex from the local cache and the feature data of the target cold vertex from the storage space corresponding to the central processing unit. The sampling task is performed based on the feature data of the target hot vertex and the feature data of the target cold vertex. The target hot vertex and the target cold vertex are i-hop vertices of the starting vertex in the target graph data, where i is an integer greater than or equal to 1.

4. The method according to claim 3, wherein, Before the step of obtaining feature data of target hot vertices from the local cache using the target graphics processor and obtaining feature data of target cold vertices from the storage space corresponding to the central processing unit, and based on the feature data of the target hot vertices and the feature data of the target cold vertices, the method further includes: Determine whether all the target hot vertices are contained in the root queue of the target graphics processor; When the target hot vertices are not all contained in the root queue of the target graphics processor, the method further includes: Based on the labels of other target hot vertices not included in the root queue of the target graphics processor, determine the other graphics processors corresponding to the other target hot vertices, and obtain the feature data of the other target hot vertices from the local cache of the other graphics processors.

5. The method according to any one of claims 1-4, wherein, The target graphics processor is one of the graphics processor groups, which includes N graphics processors, where N is an integer greater than or equal to 1. Before determining the target graphics processor corresponding to the sampling task based on the label of the starting vertex of the sampling task, the method further includes: Identify hot vertices in the target graph data to obtain the hot vertex graph data corresponding to the hot vertices; as well as Under the constraint of the first preset condition, the hot vertex graph data is divided into N subgraphs, and the hot vertices in each subgraph have the same label. The feature data corresponding to the hot vertices in the N subgraphs are loaded into the local caches of the N graphics processors respectively.

6. The method according to claim 5, wherein, The method further includes: For any graphics processor: Obtain the root vertex corresponding to the graphics processor, the root vertex including: hot vertices in the subgraph corresponding to the graphics processor, and cold vertices with the same label as the hot vertices; and Add the root vertex to the root queue of the graphics processor.

7. The method according to claim 6, wherein, Also includes: Based on the preset label propagation algorithm and the labels of each hot vertex, the label of the cold vertex is determined.

8. The method according to claim 5, wherein, The target graph data includes multiple vertices and the topological relationships corresponding to each of the multiple vertices; The process of identifying hot vertices in the target graph data and obtaining the corresponding hot vertex graph data includes: Based on the topological relationships corresponding to each of the multiple vertices, the degree of each vertex in the target graph data is obtained; Hot vertices are determined based on the degree of each vertex in the target graph data; and Based on the hot vertices and the corresponding topological relationships, the hot vertex graph data is obtained.

9. The method according to claim 8, wherein, The process of determining hot vertices based on the degree of each vertex in the target graph data includes: Vertices whose degrees satisfy a second preset condition are identified as hot vertices, wherein the second preset condition includes: identifying the top k vertices by degree as hot vertices, where k is an integer greater than or equal to 1; or identifying vertices whose degrees are greater than a preset threshold as hot vertices.

10. The method of claim 5, wherein, The process of dividing the hot vertex graph data into N sub-graphs under the constraint of a first preset condition includes: Based on the degree of each hot vertex, determine the target weight corresponding to each hot vertex; and Based on the target weights corresponding to each hot vertex and the topological relationships between the hot vertices, and constrained by a first preset condition, the hot vertex graph data is divided into N subgraphs.

11. The method according to claim 10, wherein, The first preset condition includes one or more of the following: The number of hot vertices in each of the subgraphs is the same or similar; Minimize the number of hot vertices that have a topological relationship between the two subgraphs; as well as The sum of the target weights of all hot vertices in each subgraph is the same or similar.

12. The method according to claim 10, wherein, The degree corresponding to the hot vertex is the initial weight of the hot vertex; Based on the degree of each hot vertex, the target weight corresponding to each hot vertex is determined, including: Based on the degree corresponding to each of the hot vertices, an initial weight set corresponding to multiple hot vertices is obtained; as well as The initial weight set is normalized, and the initial weights in the initial weight set are mapped to the target weight range to obtain the target weights corresponding to each of the multiple hot vertices.

13. A training system, comprising: At least one storage medium storing at least one instruction set for training a graph neural network; as well as At least one processor is communicatively connected to the at least one storage medium, wherein, when the training system is running, the at least one processor reads the at least one instruction set and implements the method as described in any one of claims 1-12 according to the instructions of the at least one instruction set.

14. A computer-readable non-volatile storage medium, wherein, The computer-readable non-volatile storage medium stores at least one instruction set, which, when executed by at least one processor, implements the method as described in any one of claims 1-12.

Images