Sub-graph sampling method and device of graph neural network

Abstract

The embodiment of the invention provides a sub-graph sampling method and device of a graph neural network, and belongs to the technical field of training of the graph neural network. The method comprises the following steps: determining a probability that a neighbor node is selected in a current sampling round according to an offset value of the neighbor node of a central node in the current sampling round and / or a weight of an edge associated with the neighbor node; sorting the neighbor nodes according to the selected probability; and judging whether the selected neighbor nodes in the current sampling round enter the next sampling round or not according to the sorting result of the neighbor nodes. According to the method, through collaborative design of the data structure and the selection algorithm of the neighbor nodes, repeated selection of high-probability nodes is avoided from a mechanism, and the method is an efficient and extensible GNN parallel sampling method.

Description

Technical Field

[0001] This application belongs to the field of computer information technology, and in particular relates to the field of large-scale graph data processing technology (the training technology of graph neural networks), specifically relating to a subgraph sampling method and apparatus for graph neural networks. Background Technology

[0002] Currently, Graph Neural Networks (GNNs) have become a core technology for processing and learning graph-structured data, achieving great success in numerous key applications such as recommender systems, social network analysis, drug discovery, and software engineering. With the expansion of these application scenarios, the scale of graph data is exploding, with the number of nodes reaching hundreds of millions and the number of edges reaching billions. Understandably, performing full graph training on such a massive graph would consume enormous computational and memory resources, which is often impractical in real-world deployments.

[0003] To overcome the aforementioned technical problems, sampling-based GNN training emerged. However, existing subgraph sampling methods for graph neural networks generally overlook a key factor that leads to serious performance issues: selection collisions of high-probability nodes. This problem manifests as follows: in biased sampling, nodes with higher weights (i.e., "hot" nodes) have a higher probability of being selected. When sampling multiple nodes from a neighbor set, parallel sampling threads or consecutive sampling attempts generate multiple random numbers. These random numbers have a high probability of falling into the selection range of the same high-probability node, leading to repeated selection of that node. Since sampling tasks typically require selecting a specified number of non-repeating neighbor nodes, this repeated selection constitutes a failed attempt and must be retried until a different node is selected. This retry-based mechanism has the following serious drawbacks:

[0004] 1. Huge performance overhead: A large number of retry operations are themselves invalid calculations, directly wasting valuable computation cycles and significantly reducing the efficiency of the sampling phase.

[0005] 2. Exacerbating performance bottlenecks in parallel hardware: On massively parallel processors like GPUs, different threads execute different code paths depending on whether a retry is needed, which can cause severe warp divergence. Warp divergence disrupts the GPU's SIMT (Single Instruction, Multithreaded) execution model, causing a sharp drop in hardware utilization and further amplifying the performance degradation caused by retries. Summary of the Invention

[0006] One object of the present invention is to construct a subgraph sampling method and apparatus for graph neural networks, which aims to solve at least a part of the above-mentioned technical problems.

[0007] Another object of the present invention is to provide a subgraph sampling device for a graph neural network. Another object of the present invention is to provide an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the steps of the above-described subgraph sampling method for a graph neural network. A further object of the present invention is to provide a readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described subgraph sampling method for a graph neural network.

[0008] To address the technical problems in the background section of this application, the present invention provides the following technical solutions:

[0009] In a first aspect, the present invention provides a subgraph sampling method for a graph neural network, comprising:

[0010] The probability of a neighboring node being selected in the current sampling round is determined based on the bias values ​​of the neighboring nodes of the center node in the current sampling round and / or the weights of the edges associated with the neighboring nodes.

[0011] The neighboring nodes are sorted according to the probability of being selected;

[0012] Based on the sorting results of the neighbor nodes, it is determined whether the selected neighbor node in the current sampling round should enter the next sampling round.

[0013] In one embodiment of the present invention, determining whether a selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes includes:

[0014] If the ranking of the selected neighbor node in the sorting result exceeds a preset threshold, the neighbor node selected in the current sampling round is deleted in the next sampling round.

[0015] Otherwise, add the neighboring nodes selected in the current sampling round in the next sampling round.

[0016] In one embodiment of the present invention, sorting the neighboring nodes according to the selected probability includes:

[0017] The neighbor node array for the current sampling round is generated based on the probability of the neighbor node being selected and the neighbor node itself; in the neighbor node array, the elements are the neighbor nodes, and the elements are sorted according to the probability of being selected.

[0018] In one embodiment of the present invention, the neighbor node with the highest probability of being selected is located at the end of the array.

[0019] In one embodiment of the present invention, a subgraph sampling method for a graph neural network further includes:

[0020] After the first sampling round is executed, the positions of the neighboring nodes with the fixed, predetermined probability of being selected are fixed in the array.

[0021] In one embodiment of the present invention, the step of determining the selected neighbor node with an unchanged probability includes:

[0022] The probability-invariant neighbor node is determined based on the edges, degree, connectivity, path, and cycle of the neighbor node.

[0023] In a second aspect, the present invention provides a subgraph sampling device for a graph neural network, the device comprising:

[0024] The current selection probability determination module is used to determine the probability that a neighboring node is selected in the current sampling round based on the bias value of the neighboring nodes of the center node in the current sampling round and / or the weight of the edge associated with the neighboring node.

[0025] The neighbor node sorting module is used to sort the neighbor nodes according to the probability of being selected.

[0026] The neighbor node filtering module is used to determine whether the selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes.

[0027] In one embodiment of the present invention, the neighbor node filtering module includes:

[0028] The neighbor node deletion unit is used to delete the neighbor node selected in the current sampling round in the next sampling round if the position of the selected neighbor node in the sorting result exceeds a preset threshold.

[0029] The neighbor node adding unit is used to add the neighbor node selected in the current sampling round in the next sampling round if otherwise.

[0030] In one embodiment of the present invention, the neighbor node sorting module includes:

[0031] The neighbor node sorting unit generates a neighbor node array for the current sampling round based on the probability of the neighbor node being selected and the neighbor node; in the neighbor node array, the elements are the neighbor nodes, and the elements are sorted according to the probability of being selected.

[0032] In one embodiment of the present invention, the neighbor node with the highest probability of being selected is located at the end of the array.

[0033] In one embodiment of the present invention, a subgraph sampling device for a graph neural network further includes:

[0034] A static node fixing module is used to fix the positions of predetermined neighbor nodes with unchanged selection probabilities in the array after the first sampling round is executed.

[0035] In one embodiment of the present invention, a subgraph sampling device for a graph neural network further includes:

[0036] A static node determination module is used to determine the neighboring nodes whose selection probability remains unchanged; the static node determination module includes:

[0037] A static node determination unit is used to determine the neighboring nodes with invariant probabilities based on the edges, degrees, connectivity, paths, and cycles of the neighboring nodes.

[0038] Thirdly, the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of a subgraph sampling method for a graph neural network.

[0039] Fourthly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a subgraph sampling method for a graph neural network.

[0040] Fifthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a subgraph sampling method for a graph neural network.

[0041] As described above, the subgraph sampling method and apparatus for a graph neural network proposed in this application firstly determines the probability of a neighboring node being selected in the current sampling round based on the bias values ​​of the neighboring nodes of the central node in the current sampling round and / or the weights of the edges associated with the neighboring nodes; then, the neighboring nodes are sorted according to the probability of being selected; finally, the neighboring nodes selected in the current sampling round are determined to enter the next sampling round based on the sorting result of the neighboring nodes.

[0042] The method provided in this application avoids the repeated selection of high-probability nodes by co-designing the data structure and selection algorithm of neighbor nodes, and is an efficient and scalable parallel sampling method for GNNs. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the subgraph sampling method of the graph neural network in an embodiment of the present invention. Figure 1 ;

[0045] Figure 2 This is a flowchart illustrating step 300 of the subgraph sampling method of the graph neural network in an embodiment of the present invention;

[0046] Figure 3 This is a flowchart illustrating the subgraph sampling method of the graph neural network in an embodiment of the present invention. Figure 2 ;

[0047] Figure 4 This is a flowchart illustrating a subgraph sampling method for a graph neural network according to a specific embodiment of the present invention.

[0048] Figure 5 A schematic diagram illustrating the principle of biased subgraph sampling conflict in existing technologies;

[0049] Figure 6 This is a schematic diagram illustrating the principle of the selection probability distribution of neighboring nodes and the arrangement of graph nodes in the existing subgraph sampling method.

[0050] Figure 7 This is a schematic diagram illustrating the principle of the probability rearrangement and sliding selection method in a specific embodiment of the present invention;

[0051] Figure 8 This is a block diagram of a subgraph sampling device for a graph neural network in an embodiment of the present invention. Figure 1 ;

[0052] Figure 9 This is a block diagram of the neighbor node filtering module 30 in an embodiment of the present invention;

[0053] Figure 10 This is a block diagram of a subgraph sampling device for a graph neural network in an embodiment of the present invention. Figure 2 ;

[0054] Figure 11 This is a block diagram of a subgraph sampling device for a graph neural network in an embodiment of the present invention. Figure 3 ;

[0055] Figure 12 This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] It should be noted that the terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0059] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0060] The core idea of ​​sampling-based GNN training is to efficiently sample a smaller subgraph from the original large graph in each training iteration, and then perform GNN computation and gradient updates only on this subgraph. Because the training phase is performed on a small subgraph, its execution time is greatly reduced. However, the randomness of subgraph sampling creates a new performance bottleneck in the data preparation phase, especially the graph sampling phase. Many advanced GNN models employ biased sampling strategies to more accurately capture the structural information of the graph. This strategy uses the weights (or biases) of nodes or edges as probabilities for non-uniform selection. Neighbor sampling samples only a subset of the target node's neighbors in each propagation, rather than taking all neighbors from the entire graph. Structured / cluster-based sampling divides the graph into several clusters (or subgraphs), using only a subset of these clusters for forward / backward propagation during the training phase. Information is passed between clusters through edges (e.g., cross-cluster fusion during multi-stage aggregation).

[0061] As described in the background section of this application, both neighbor sampling (representative work GraphSAGE) and existing structured / cluster-level sampling (representative work Cluster-GCN) generally overlook a key factor that leads to serious performance problems: selection conflict of high-probability nodes (see the background section of this application for specific technical issues). That is, the selection conflict problem that is common in the subgraph sampling methods of graph neural networks in the prior art is a key bottleneck restricting the biased sampling performance of large-scale GNNs.

[0062] To address the aforementioned technical problems, this invention presents a parallel sampling method aimed at optimizing the performance of the data preparation phase during GNN training. See also... Figure 1 The method specifically includes the following:

[0063] Step 100: Determine the probability that a neighboring node is selected in the current sampling round based on the bias values ​​of the neighboring nodes of the center node in the current sampling round and / or the weights of the edges associated with the neighboring nodes.

[0064] Step 200: Sort the neighboring nodes according to the probability of being selected;

[0065] Step 300: Determine whether the selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes.

[0066] As described above, the subgraph sampling method and apparatus for a graph neural network proposed in this application firstly determines the probability of a neighboring node being selected in the current sampling round based on the bias values ​​of the neighboring nodes of the central node in the current sampling round and / or the weights of the edges associated with the neighboring nodes; then, the neighboring nodes are sorted according to the probability of being selected; finally, the neighboring nodes selected in the current sampling round are determined to enter the next sampling round based on the sorting result of the neighboring nodes.

[0067] The method provided in this application, through the collaborative design of data structures and selection algorithms, fundamentally avoids the repeated selection of high-probability nodes, making it an efficient and scalable parallel sampling method for GNNs. Specifically, compared with existing technologies, this application has the following advantages:

[0068] 1. Improved Sampling Performance: By fundamentally eliminating selection conflicts and retries, the sampling efficiency of this invention far surpasses that of related methods in the prior art. In the evaluation of the Papers100M and MAG240M ultra-large-scale graph datasets, the HSampler system integrated with this invention achieved a significant performance improvement of 2.1 to 6.2 times in end-to-end training performance compared to the industry-leading DGL system.

[0069] 2. Highly compatible with parallel hardware (GPUs): The advantages of this invention are particularly prominent on parallel hardware such as GPUs. This is specifically reflected in the following two aspects:

[0070] Eliminating branch divergence: The no-retry design avoids complex control flow and ensures the consistency of thread execution paths within the GPU thread bundle, thereby avoiding the performance killer—branch divergence.

[0071] Optimizing memory access patterns: The probabilistic rearrangement step physically groups high-probability nodes (which are usually also frequently accessed nodes). When a GPU warp performs a binary search on the CDF array in parallel, the accessed data is contiguous or adjacent in memory, which greatly improves the coalesced memory access. Coalesced access allows a single memory transaction to serve the requests of multiple threads, significantly reducing memory bandwidth pressure and access latency, and further improving the GPU's computational efficiency.

[0072] The entire training process of a graph neural network can be described as an iterative process of epochs with multiple steps. Specifically, an epoch includes the following steps executed in sequence: sampling, forward propagation, loss calculation, back propagation, and optimizer update. Therefore, steps 100 to 300 in this application are also an iterative process, and their execution cycle is one epoch as described above.

[0073] For step 100, the central node can be obtained through random sampling, and the neighboring nodes of the central node can be determined by the sampling strategy.

[0074] The bias values ​​of neighboring nodes can be randomly initialized to small values ​​(such as normally distributed random values ​​with a mean of 0, or uniformly distributed random values). This initialization helps break symmetry, allowing different nodes to have different bias values. Then, optimization can be performed using backpropagation. In each iteration, the bias values ​​are updated along with other model parameters (such as the weight matrix) to minimize the loss function.

[0075] The edge weights are updated similarly to the biases of neighboring nodes, using optimization methods such as gradient descent. The biases are adaptively adjusted during training until the model converges.

[0076] For step 200, multiple neighbor nodes can be grouped together in an array. In this data group, the order of the elements (neighbor nodes) is sorted according to the probability of being selected (either from smallest to largest or from largest to smallest).

[0077] Regarding step 300, as mentioned above, if the already selected "hot spots" are deleted, then the "hot spots" will only be selected once during the entire sampling process, thereby completely eliminating selection conflicts and related retry overhead, and greatly improving the effectiveness and efficiency of parallel sampling.

[0078] In some embodiments of the present invention, see Figure 2 Step 300 includes:

[0079] Step 301: If the position of the selected neighbor node in the sorting result exceeds a preset threshold, delete the neighbor node selected in the current sampling round in the next sampling round.

[0080] For example, assuming the preset threshold is 5 positions, if the selected neighbor node ranks in the top 5 of the array in the sorting results, that is, in the top 5 of the probability of being selected, then the currently selected neighbor node will be deleted to prevent it from entering the next sampling round.

[0081] Step 302: Otherwise, add the neighbor node selected in the current sampling round in the next sampling round.

[0082] Taking a preset threshold of 5 as an example, if the selected neighbor node does not rank in the top 5 of the array in the sorting result, that is, it is not in the top 5 of the probability of being selected, then the currently selected neighbor node will be added to the next sampling round (in fact, this addition action does not need to be executed, the sampling step defaults to all neighbor nodes entering the next sampling round).

[0083] In a more preferred manner, the preset threshold in steps 301 and 302 is 1, that is, the "hot spot" is the neighbor node with the highest probability of being selected.

[0084] In some embodiments of the present invention, step 200 includes:

[0085] The neighbor node array for the current sampling round is generated based on the probability of the neighbor node being selected and the neighbor node itself; in the neighbor node array, the elements are the neighbor nodes, and the elements are sorted according to the probability of being selected.

[0086] Assuming the array of neighboring nodes is of length N, the sampling task requires selecting K neighboring nodes. The first selection is performed on the entire array of length N. If the node with the highest probability of selection (located at the end of the array) is selected, the search range for the next selection will be updated to the first N-1 elements of the array, logically excluding the already selected node. This process continues, with each successful selection leading to a reduction in the search range.

[0087] In some embodiments of the present invention, the neighbor node with the highest selected probability is located at the end of the array. The advantage of this arrangement is that the system overhead consumed in retrieving the neighbor node with the highest probability can be minimized.

[0088] In some embodiments of the present invention, see Figure 3 A subgraph sampling method for graph neural networks further includes:

[0089] Step 400: After the first sampling round is executed, fix the positions of the neighboring nodes in the array with the predetermined probability of being selected.

[0090] For neighboring nodes whose selection probability remains constant throughout training (e.g., whose selection probability is determined solely by static graph structure information such as the degree of the neighboring node), the reordering of neighboring nodes in the array can be performed as a one-time preprocessing operation before training begins. This avoids repeatedly sorting such neighboring nodes at each sampling, spreading the overhead to almost zero.

[0091] In some embodiments of the present invention, the step of determining the selected neighbor node with an unchanged probability includes:

[0092] The probability-invariant neighbor node is determined based on the edges, degree, connectivity, path, and cycle of the neighbor node.

[0093] Edge: An edge connects two nodes in a graph, representing a relationship or interaction between them. Edges can be directed or undirected: a directed edge represents a relationship in one direction, while an undirected edge represents a bidirectional relationship. Edge properties may include weight, capacity, type, etc.

[0094] Degree: The degree of a node represents the number of edges connected to that node. For a directed graph, the degree is divided into in-degree (the number of edges pointing to that node) and out-degree (the number of edges pointing from that node to other nodes).

[0095] Connectivity: A graph is connected if any two nodes can be connected by a path. A graph is disconnected if there are at least two nodes that cannot be connected by any path.

[0096] Path: A sequence of edges from one node to another. Shortest path: The shortest path between two nodes in the graph.

[0097] A cycle: A path that starts from a node and eventually returns to itself.

[0098] The method provided in this application, through the collaborative design of data structures and selection algorithms, fundamentally avoids the repeated selection of high-probability nodes, making it an efficient and scalable parallel sampling method for GNNs. Specifically, compared with existing technologies, this application has the following advantages:

[0099] 1. Improved Sampling Performance: By fundamentally eliminating selection conflicts and retries, the sampling efficiency of this invention far surpasses that of related methods in the prior art. In the evaluation of the Papers100M and MAG240M ultra-large-scale graph datasets, the HSampler system integrated with this invention achieved a significant performance improvement of 2.1 to 6.2 times in end-to-end training performance compared to the industry-leading DGL system.

[0100] 2. Highly compatible with parallel hardware (GPUs): The advantages of this invention are particularly prominent on parallel hardware such as GPUs. This is specifically reflected in the following two aspects:

[0101] Eliminating branch divergence: The no-retry design avoids complex control flow and ensures the consistency of thread execution paths within the GPU thread bundle, thereby avoiding the performance killer—branch divergence.

[0102] Optimizing memory access patterns: The probabilistic rearrangement step physically groups high-probability nodes (which are usually also frequently accessed nodes). When a GPU warp performs a binary search on the CDF array in parallel, the accessed data is contiguous or adjacent in memory, which greatly improves the coalesced memory access. Coalesced access allows a single memory transaction to serve the requests of multiple threads, significantly reducing memory bandwidth pressure and access latency, and further improving the GPU's computational efficiency.

[0103] To further illustrate the solution, this invention also provides a specific implementation of a subgraph sampling method for graph neural networks, see [link to relevant documentation]. Figure 4 This includes the following steps:

[0104] S1: Probabilistically rearrange the elements in the neighbor node array of each central node.

[0105] Before sampling and selection, the neighbor node array of each central node is preprocessed. Specifically, the array is sorted according to the probability of each neighbor node being selected (i.e., the weight of its associated edge or the node bias value). The neighbor nodes are arranged in ascending order of probability, so that all high-probability "hotspot" nodes are placed at the end of the array.

[0106] S2: Optimize static probabilistic neighbor nodes.

[0107] For scenarios where the selected probability values ​​remain constant throughout training (e.g., the probabilities are determined solely by static graph structure information such as the degree of neighboring nodes), this rearrangement step can be performed as a one-time preprocessing operation before training begins. This avoids repeated sorting at each sampling, distributing the overhead to almost zero.

[0108] S3: Slide to select neighboring nodes.

[0109] Sliding selection performs sampling selection on an array of neighboring nodes that have been probabilistically rearranged. It is important to note that this selection process differs fundamentally from existing methods: once a node located at a high probability end of the array (e.g., the first or last segment) is selected, the search range for subsequent selections is immediately and dynamically narrowed (or "slid").

[0110] Implementation details: Assume the neighbor node array has a length of N, and the sampling task requires selecting K neighbors. The first selection is performed on the entire array of length N. If the node with the highest probability (located at the end of the array) is selected, the search range for the next selection will be updated to the first N-1 elements of the array, logically excluding the already selected node. This process continues, with each successful selection narrowing the search range. This "selection-exclusion" mechanism fundamentally ensures that in a single sampling task (i.e., selecting K neighbors for a single central node), any node, especially the node with the highest probability, will be selected at most once. This completely eliminates selection conflicts and related retry overhead, greatly improving the effectiveness and efficiency of parallel sampling.

[0111] See Figure 5 The diagram visually illustrates the shortcomings of existing related technologies: On the left is the central node C and its four weighted neighbor nodes N1 to N4. In the two independent sampling attempts on the right, the generated random numbers 3.7 and 5.8, according to the cumulative distribution, both fall within the probability interval representing neighbor N2. This results in the second attempt being a failed repeated selection, i.e., a "selection conflict," which must be retried.

[0112] Further, see next. Figure 6 It demonstrates the probability distribution of neighbor nodes being selected and the arrangement of graph nodes in existing subgraph sampling methods. Figure 7 This is a schematic diagram illustrating the principle of a subgraph sampling method for a graph neural network provided in this application. During the dynamic selection process, the selected graph nodes are placed on the far right, and the selection range is limited to the unselected portion. Figure 7 This also demonstrates how the probabilistic rearrangement of this invention optimizes GPU memory access patterns. The upper part shows the probability distribution without rearrangement. When multiple threads access the data corresponding to the selected graph nodes in parallel (as indicated by the arrows), the access points are relatively scattered in memory, resulting in a high level of the common parent node of the search tree and discrete memory access. Figure 7 After the above probabilistic rearrangement operation, high-probability nodes are moved to one side, and the target data accessed in parallel is more concentrated in memory. This makes the common parent node of the search tree lower in level, thus achieving more efficient GPU merged memory access.

[0113] To address the technical problems of selection conflicts, numerous retries, and low parallel efficiency caused by high-probability nodes in existing biased sampling techniques of GNNs, this application provides a subgraph sampling method for graph neural networks:

[0114] First, this method is based on probabilistic rearrangement and sliding selection: through the synergistic effect of the two steps of probabilistic rearrangement and sliding selection, conflict-free and efficient parallel sampling is achieved.

[0115] Secondly, this method preprocesses the set of neighboring nodes of a central node by sorting them according to their sampling probabilities, in order to physically or logically cluster high-probability nodes.

[0116] Finally, this method employs an algorithm mechanism that dynamically excludes a selected node from subsequent candidate search ranges when making selections on the rearranged neighbor set. Specifically, compared with existing technologies, this application has the following advantages:

[0117] 1. Improved Sampling Performance: By fundamentally eliminating selection conflicts and retries, the sampling efficiency of this invention far surpasses that of related methods in the prior art. In the evaluation of the Papers100M and MAG240M ultra-large-scale graph datasets, the HSampler system integrated with this invention achieved a significant performance improvement of 2.1 to 6.2 times in end-to-end training performance compared to the industry-leading DGL system.

[0118] 2. Highly compatible with parallel hardware (GPUs): The advantages of this invention are particularly prominent on parallel hardware such as GPUs. This is specifically reflected in the following two aspects:

[0119] Eliminating branch divergence: The no-retry design avoids complex control flow and ensures the consistency of thread execution paths within the GPU thread bundle, thereby avoiding the performance killer—branch divergence.

[0120] Optimizing memory access patterns: The probabilistic rearrangement step physically groups high-probability nodes (which are usually also frequently accessed nodes). When a GPU warp performs a binary search on the CDF array in parallel, the accessed data is contiguous or adjacent in memory, which greatly improves the coalesced memory access. Coalesced access allows a single memory transaction to serve the requests of multiple threads, significantly reducing memory bandwidth pressure and access latency, and further improving the GPU's computational efficiency.

[0121] Based on the same inventive concept, this application also provides a subgraph sampling device for graph neural networks, which can be used to implement the method described in the above embodiments, as shown in the following embodiments. Since the principle of the subgraph sampling device for graph neural networks in solving the problem is similar to that of the subgraph sampling method for graph neural networks, the implementation of the subgraph sampling device for graph neural networks can refer to the implementation of the subgraph sampling method for graph neural networks, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0122] Embodiments of the present invention provide a specific implementation of a subgraph sampling device for a graph neural network capable of implementing a subgraph sampling method for graph neural networks. See [link to specific implementation details]. Figure 8 A subgraph sampling device for a graph neural network specifically includes the following components:

[0123] The current selection probability determination module 10 is used to determine the probability that a neighboring node is selected in the current sampling round based on the bias value of the neighboring nodes of the center node in the current sampling round and / or the weight of the edge associated with the neighboring node.

[0124] The neighbor node sorting module 20 is used to sort the neighbor nodes according to the probability of being selected;

[0125] The neighbor node filtering module 30 is used to determine whether the selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes.

[0126] In one embodiment of the present invention, see Figure 9 The neighbor node filtering module 30 includes:

[0127] The neighbor node deletion unit 30a is used to delete the neighbor node selected in the current sampling round in the next sampling round if the position of the selected neighbor node in the sorting result exceeds a preset threshold.

[0128] The neighbor node adding unit 30b is used to add the neighbor node selected in the current sampling round in the next sampling round if otherwise.

[0129] In one embodiment of the present invention, the neighbor node sorting module 20 includes:

[0130] The neighbor node sorting unit generates a neighbor node array for the current sampling round based on the probability of the neighbor node being selected and the neighbor node; in the neighbor node array, the elements are the neighbor nodes, and the elements are sorted according to the probability of being selected.

[0131] In one embodiment of the present invention, the neighbor node with the highest probability of being selected is located at the end of the array.

[0132] In one embodiment of the present invention, see Figure 10 A subgraph sampling device for a graph neural network further includes:

[0133] The static node fixing module 40 is used to fix the positions of predetermined neighbor nodes with unchanged selection probabilities in the array after the first sampling round is executed.

[0134] In one embodiment of the present invention, see Figure 11 A subgraph sampling device for a graph neural network further includes:

[0135] A static node determination module 50 is used to determine the neighboring nodes whose selection probability remains unchanged; the static node determination module 50 includes:

[0136] A static node determination unit is used to determine the neighboring nodes with invariant probabilities based on the edges, degrees, connectivity, paths, and cycles of the neighboring nodes.

[0137] As described above, the subgraph sampling device for a graph neural network proposed in this application includes: a current selection probability determination module, used to determine the probability of a neighbor node being selected in the current sampling round based on the bias values ​​of the neighbor nodes of the center node in the current sampling round and / or the weights of the edges associated with the neighbor nodes; a neighbor node sorting module, used to sort the neighbor nodes according to the selection probability; and a neighbor node filtering module, used to determine whether the selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result.

[0138] The present application provides a mechanism to avoid repeated selection of high-probability nodes through the collaborative design of data structures and selection algorithms for neighboring nodes.

[0139] Embodiments of this application also provide a specific implementation of an electronic device capable of implementing all steps in the subgraph sampling method of the graph neural network in the above embodiments, see [link to implementation details]. Figure 12 The electronic devices specifically include the following:

[0140] Processor 1201, memory 1202, communications interface 1203, and bus 1204;

[0141] The processor 1201, memory 1202, and communication interface 1203 communicate with each other via bus 1204; the communication interface 1203 is used to realize information transmission between server-side devices and client-side devices and other related devices.

[0142] The processor 1201 is used to call the computer program in the memory 1202. When the processor executes the computer program, it implements all the steps in the subgraph sampling method of the graph neural network in the above embodiment. For example, when the processor executes the computer program, it implements the following steps:

[0143] The probability of a neighboring node being selected in the current sampling round is determined based on the bias values ​​of the neighboring nodes of the center node in the current sampling round and / or the weights of the edges associated with the neighboring nodes.

[0144] The neighboring nodes are sorted according to the probability of being selected;

[0145] Based on the sorting results of the neighbor nodes, it is determined whether the selected neighbor node in the current sampling round should enter the next sampling round.

[0146] In one embodiment of the present invention, determining whether a selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes includes:

[0147] If the ranking of the selected neighbor node in the sorting result exceeds a preset threshold, the neighbor node selected in the current sampling round is deleted in the next sampling round.

[0148] Otherwise, add the neighboring nodes selected in the current sampling round in the next sampling round.

[0149] In one embodiment of the present invention, sorting the neighboring nodes according to the selected probability includes:

[0150] The neighbor node array for the current sampling round is generated based on the probability of the neighbor node being selected and the neighbor node itself; in the neighbor node array, the elements are the neighbor nodes, and the elements are sorted according to the probability of being selected.

[0151] In one embodiment of the present invention, the neighbor node with the highest probability of being selected is located at the end of the array.

[0152] In one embodiment of the present invention, a subgraph sampling method for a graph neural network further includes:

[0153] After the first sampling round is executed, the positions of the neighboring nodes with the fixed, predetermined probability of being selected are fixed in the array.

[0154] In one embodiment of the present invention, the step of determining the selected neighbor node with an unchanged probability includes:

[0155] The probability-invariant neighbor node is determined based on the edges, degree, connectivity, path, and cycle of the neighbor node.

[0156] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the subgraph sampling method for graph neural networks in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the subgraph sampling method for graph neural networks in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0157] The probability of a neighboring node being selected in the current sampling round is determined based on the bias values ​​of the neighboring nodes of the center node in the current sampling round and / or the weights of the edges associated with the neighboring nodes.

[0158] The neighboring nodes are sorted according to the probability of being selected;

[0159] Based on the sorting results of the neighbor nodes, it is determined whether the selected neighbor node in the current sampling round should enter the next sampling round.

[0160] In one embodiment of the present invention, determining whether a selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes includes:

[0161] If the ranking of the selected neighbor node in the sorting result exceeds a preset threshold, the neighbor node selected in the current sampling round is deleted in the next sampling round.

[0162] Otherwise, add the neighboring nodes selected in the current sampling round in the next sampling round.

[0163] In one embodiment of the present invention, sorting the neighboring nodes according to the selected probability includes:

[0164] The neighbor node array for the current sampling round is generated based on the probability of the neighbor node being selected and the neighbor node itself; in the neighbor node array, the elements are the neighbor nodes, and the elements are sorted according to the probability of being selected.

[0165] In one embodiment of the present invention, the neighbor node with the highest probability of being selected is located at the end of the array.

[0166] In one embodiment of the present invention, a subgraph sampling method for a graph neural network further includes:

[0167] After the first sampling round is executed, the positions of the neighboring nodes with the fixed, predetermined probability of being selected are fixed in the array.

[0168] In one embodiment of the present invention, the step of determining the selected neighbor node with an unchanged probability includes:

[0169] The probability-invariant neighbor node is determined based on the edges, degree, connectivity, path, and cycle of the neighbor node.

[0170] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0171] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0172] While this application provides method operation steps as shown in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the method can be executed in the order shown in the embodiments or drawings or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0173] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this specification, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0174] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0175] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0176] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0177] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0178] The above description is merely an embodiment of the present specification and is not intended to limit the embodiments of the present specification. For those skilled in the art, various modifications and variations can be made to the embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present specification should be included within the scope of the claims of the embodiments of the present specification.

Claims

1. A subgraph sampling method for graph neural networks, characterized in that, include: The probability of a neighboring node being selected in the current sampling round is determined based on the bias values ​​of the neighboring nodes of the center node in the current sampling round and / or the weights of the edges associated with the neighboring nodes. The neighboring nodes are sorted according to the probability of being selected; Based on the sorting results of the neighbor nodes, it is determined whether the selected neighbor node in the current sampling round should enter the next sampling round.

2. The subgraph sampling method according to claim 1, characterized in that, Determining whether a selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes includes: If the ranking of the selected neighbor node in the sorting result exceeds a preset threshold, the neighbor node selected in the current sampling round is deleted in the next sampling round. Otherwise, add the neighboring nodes selected in the current sampling round in the next sampling round.

3. The subgraph sampling method according to claim 1, characterized in that, Sort the neighboring nodes according to the probability of being selected, including: The neighbor node array for the current sampling round is generated based on the probability of the neighbor node being selected and the neighbor node itself; in the neighbor node array, the elements are the neighbor nodes, and the elements are sorted according to the probability of being selected.

4. The subgraph sampling method according to claim 3, characterized in that, The neighbor node with the highest probability of being selected is located at the end of the array.

5. The subgraph sampling method according to claim 3, characterized in that, Also includes: After the first sampling round is executed, the positions of the neighboring nodes with the fixed, predetermined probability of being selected are fixed in the array.

6. The subgraph sampling method according to claim 5, characterized in that, The steps for determining the selected neighbor nodes with unchanged probabilities include: The probability-invariant neighbor node is determined based on the edges, degree, connectivity, path, and cycle of the neighbor node.

7. A subgraph sampling device for a graph neural network, characterized in that, include: The current selection probability determination module is used to determine the probability that a neighboring node is selected in the current sampling round based on the bias value of the neighboring nodes of the center node in the current sampling round and / or the weight of the edge associated with the neighboring node. The neighbor node sorting module is used to sort the neighbor nodes according to the probability of being selected. The neighbor node filtering module is used to determine whether the selected neighbor node in the current sampling round should enter the next sampling round based on the sorting result of the neighbor nodes.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the subgraph sampling method of the graph neural network according to any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the subgraph sampling method of the graph neural network according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the subgraph sampling method for the graph neural network as described in any one of claims 1 to 6.

Images