Rapid data segmentation method for large-scale parallel graph neural network calculation

Abstract

The invention discloses a rapid data segmentation method for large-scale parallel graph neural network calculation, and relates to the field of data segmentation. The method comprises the steps that when streaming graph processing is carried out on vertexes, a super adjacency list of k-Hop is constructed, and therefore the complex dependency relationship in the GNN calculation process can be better captured. The influence of different types of neighbors on vertex distribution is particularly concerned, in the GNN training process, nodes are usually divided into training points, test points and verification points, and the influences of different types of neighbors on vertexes are different. Due to the fact that multiple rounds of iteration are usually carried out in the training stage and only one round is carried out in the testing stage, for a certain node, the node and a training neighbor are more expected to be located in the same partition, and communication overhead is reduced. Through partition information of the distributed nodes, expected values of neighbors in the partitions are increased. Namely, through the combined action of the distribution information of the inner neighbors of each node and the distribution information of the out neighbors in the k-Hop adjacency list, the effectiveness of division is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of data partitioning, and particularly to a fast data partitioning method for large-scale parallel graph neural network computing. Background Art

[0002] In recent years, deep learning has been widely recognized for its ability to automatically learn features and patterns from large amounts of data through simulating the human brain neural network and perform complex task prediction and decision-making. However, it has limitations in processing data with graph structures, and graphs are ubiquitous in the real world, such as social networks, molecular structures, knowledge graphs, etc. Graph neural networks (GNNs) have achieved better results than traditional machine learning and data mining methods by learning the rich knowledge contained in graph structures. In the era of information explosion, the emergence of large graphs has become common, such as the user social network graphs generated by platforms like Twitter and Facebook. These graphs may contain billions of vertices and hundreds of billions of edges, and each vertex is also attached with a feature vector of thousands of dimensions, making it impractical to store and train graph data on a single machine. Therefore, distributed graph neural networks have emerged to meet the demand for large-scale graph data processing.

[0003] Existing distributed systems can be divided into two categories in terms of data partitioning methods. One is to directly adopt traditional graph partitioning methods, such as hash partitioning or the METIS method. These traditional methods usually do not take into account the particularity of the GNN computing process, resulting in unsatisfactory effects in practical applications. Although hash partitioning is relatively fast, it is often difficult to effectively capture the structural features of the graph, which leads to poor partitioning effects and thus huge communication overhead. In contrast, using the METIS method can obtain a relatively high-quality partitioning result, but it only guarantees that the total number of vertices in each partition is the same. However, the difference in the number of training nodes in each partition may still be significant (the same is true for test and validation nodes). This imbalance not only affects the training efficiency but may also cause computational resource overload in some partitions, thus affecting the performance of the entire distributed GNN training process; the other is to make customized partitioning for its GNN system, such as P3 which uses the method of splitting features and evenly distributes the features of each vertex by dimension to each partition. This method often has low generalization ability and is difficult to adapt to different types of graph data and task requirements. ByteGNN proposed a new streaming graph partitioning method for GNN, which preprocesses nodes into blocks through the BFS method and then performs streaming graph partitioning using a method similar to LDG. However, this method does not fully utilize the k-Hop neighbor information of each node, so the partitioning effect is not ideal.

[0004] In the data partitioning of graph neural networks, given a graph G=(V, E), where V is the vertex set and E is the edge set. For traditional edge-cut graph partitioning methods, the graph G is evenly partitioned into p non-overlapping subgraphs / partitions P i ={V i , E i}, where i, j ∈ {1, 2,..., p}. |V i | and |E i | represent the number of vertices and edges in P i respectively. For any i ≠ j, there is and P1 ∪ P2 ∪... ∪ P p = G. To ensure the balance of graph partitioning, it is usually required to balance the load of each subgraph during the graph partitioning process. The load balance can be measured by the following formula, and the imbalance degree of its workload is represented by the parameter δ v . The specific formula is as follows:

[0005]

[0006] where δ v = 1.0 represents the perfect balance of the number of vertices in each partition in the ideal state.

[0007] In addition to load balance, the number of cross-edges is also an important consideration in graph partitioning. Cross-edges refer to those edges that connect different partitions, and the number of cross-edges largely determines the communication overhead between different partitions. Reducing the number of cross-edges means that the dependence between different partitions can be reduced, thereby reducing the amount of data exchange in distributed computing. Therefore, the goal of partitioning is to minimize the number of cross-edges between partitions as much as possible, thereby reducing the communication overhead. In traditional graph partitioning, it is usually desired to minimize the number of cross-edges on the premise of satisfying load balance. For example, in Figure 3 (a), the cutting method shown by the dotted line minimizes the cross-edges between partitions, and the number of vertices in the two partitions is equal, reflecting the goals of load balance and minimizing the number of cross-edges.

[0008] However, the optimization goals considered by traditional graph partitioning methods show certain limitations in GNN application scenarios. Suppose there is a 3-layer GNN model, Figure 3 in which A and B are training points, the green points represent all k-Hop neighbors of A, and the blue points represent all k-Hop neighbors of B. Using the traditional graph partitioning method in Figure 3 (a) to partition the graph data, although the number of cross-edges is the least, it will still cause 1 network communication; if the partitioning method in Figure 3 (b) is adopted and cut into two partitions at the dotted line, although 3 cross-edges are generated, since node A and node B do not receive information from nodes outside the partition, no network communication will occur.

[0009] In addition to using traditional graph partitioning methods to partition nodes, many distributed systems also use the method of slicing feature vectors to achieve graph data partitioning. For example, the typical distributed GNN system P3 does this by evenly distributing each feature vector to each partition.

[0010] At the same time, the load balancing problem of distributed GNN is also different from that of traditional graph partitioning. Since the vertices in GNN graph data will be divided into training points, test points, and validation points, only considering roughly the same total number of nodes will result in a huge difference in the number of training nodes between partitions, which will seriously slow down the overall training progress. Therefore, we need to consider the load balancing of each type of node, so we need to rewrite the above load balancing formula:

[0011]

[0012] Among them, δ train 、δ test and δ val represent the load balancing factors of the training set, test set, and validation set respectively.

[0013] In summary, the optimization goal of graph data partitioning for GNN is to minimize the number of remote k-Hop neighbors of each node and ensure load balancing for each type of node. Summary of the Invention

[0014] The present invention proposes a fast data segmentation method for large-scale parallel graph neural network computing, which adopts a new GNN-oriented streaming graph partitioning method to better support the computing requirements of distributed GNN on large-scale graphs.

[0015] The present invention specifically adopts the following technical solutions:

[0016] A fast data segmentation method for large-scale parallel graph neural network computing, including:

[0017] (1) Topology-enhanced partitioning based on k-hop neighbors; first, preprocess the graph topology structure to construct a k-hop super adjacency list for each node. Specifically, given a node v, define its k-hop neighbor set N k-Hop (v) as the set of all nodes reachable within k hops. This set is used to measure the number of nodes directly or indirectly associated with v in a partition during the node allocation process, and the allocation decision is made through an optimized heuristic function (1):

[0018]

[0019] Among them, pid represents the partition number assigned to node v, which is selected from all possible partitions through the argmax operation, and is the partition number that makes the objective function reach the maximum value;

[0020] i ∈ [1, p] means that the index i of the partition ranges from 1 to p, where p is the total number of partitions. All possible partitions are traversed, the values of the expressions corresponding to each partition are calculated, and then the corresponding partition number with the maximum value is selected;

[0021] represents the partition and the size of the intersection with the set N k-Hop (v) of the k-hop neighbors of node v. Specifically, is the set of nodes in the i-th partition at time step t, and N k-Hop (v) is the set of k-hop neighbors of node v. The size of the intersection represents the number of nodes directly or indirectly associated with v in partition i;

[0022] W t (i, v) represents the weight of node v assigned to partition i at time step t. This weight can be used to measure the priority or cost of assigning node v to partition i;

[0023] t represents the time step or the number of iterations;

[0024] (2) Neighbor type weighted influence; when assigning nodes, different types of nodes in the neighborhood of the target node are weighted, so that the training nodes have higher weights in the partition decision. Specifically, when the target node accesses its neighborhood nodes, the partition weight of the training node is set to δ train , usually greater than 1, while the weights of the test and validation nodes remain 1. When selecting a partition, the optimal partition is selected according to the sum of the weights in each partition as cur;

[0025] (3) Attraction mechanism of the assigned nodes; the attraction mechanism utilizes the implicit information of the k-Hop inner neighbors of the assigned nodes to optimize the partitioning effect by enhancing the partition convergence of the k-hop neighbors. Specifically, when a node is assigned to a certain partition, the information of this partition will spread layer by layer to the unassigned nodes through its k-hop neighbors;

[0026] To implement the attraction mechanism, the heuristic function is improved. Given an unassigned node v, not only the neighbors in the k-Hop adjacency list of v are considered, but also the distribution information of the k-Hop inner neighbors of node v in each partition is calculated. The assignment information of the k-Hop inner neighbors reaching the node and the assignment information of the k-Hop outer neighbors work together to guide the assignment of the node. The following partition selection criteria are defined:

[0027]

[0028] in Denotes how many external neighbors of v have arrived and been placed in the i-th partition P so far at time instance t. i middle, Indicates how many of v’s inner neighbors have arrived and been placed in the i-th partition P so far at time instance t. i middle;

[0029] In this paper, we design an expected value tracking mechanism. When a node v is assigned to a partition pid, the expected value of its k-hop neighbor u will be updated accordingly.

[0030] Preferably, it is characterized in that

[0031] In order to improve the efficiency of adjacency table construction, a sampling-based adjacency table construction method is adopted. When constructing the k-hop adjacency table, a layer-by-layer sampling strategy is adopted to randomly select some nodes from the neighborhood of each hop for expansion according to the set probability. Specifically, for the neighbor nodes of each hop, according to the set sampling ratio p k Select some nodes to enter the adjacency table, and sample layer by layer to gradually build the adjacency table; this sampling process is regarded as a hierarchical recursive sampling: first, sample with probability p1 in the first-hop neighbor set, and select some nodes as the expanded initial node set; then sample with probability p2 in the second-hop neighbor set, and so on, sampling and building the neighbor set of each hop layer by layer.

[0032] Preferably, in order to reduce the computational overhead, a neighbor type weighting strategy is proposed, which optimizes the partition allocation of the target node by giving the training neighbors a higher influence. Specifically, the length of the sampled k-Hop adjacency table of the target node is set to The total number of partitions is p, then ideally the average score of the target node in each partition should be In order to effectively improve the priority of training neighbors in partitioning decisions, a higher weight δ is given to training neighbors. train , and the weights of non-training neighbors remain at 1.

[0033] The present invention has the following beneficial effects:

[0034] The partitioning quality is improved by considering the k-hop neighbor distribution information of the node, the impact of neighbor type on node allocation, and the attraction mechanism of the allocated node to its k-hop neighbors, thereby improving the effectiveness of the partitioning and the balance of the load.

[0035] By assigning higher partition weights to training neighbors, the target node is guided to be in the same partition as the training neighbors preferentially. Meanwhile, the improved load balancing formula (Formula 2) introduces independent balancing factors for training, testing, and validation nodes respectively to ensure the balance of the number of different types of nodes in each partition and avoid resource skew caused by the concentration of training nodes.

[0036] By introducing an expected value tracking mechanism, when node v is assigned to a certain partition, the partition expected value of its k-hop external neighbors will be updated dynamically, forming an "attraction effect". Specifically, the assignment of node v will be propagated layer by layer through the k-hop adjacency list, making unassigned nodes tend to choose the partition with more associated nodes.

[0037] By adopting a hierarchical recursive sampling strategy, key neighbors are screened probabilistically hop by hop to avoid the exponential explosion of the number of neighbors. For example, the sampling rate is increased in the dense degree area to retain the key structure, while the sampling rate is decreased in the sparse area to save resources. This strategy fits well with sampling training models such as GraphSAGE, and the preprocessed adjacency list can be directly used in the training phase to reduce redundant communication. In addition, the dynamic pruning mechanism terminates the calculation in advance when the partition score far exceeds the threshold, further reducing the time complexity and ensuring the real-time performance of the algorithm in large-scale graphs. Brief Description of the Drawings

[0038] Figure 1 It is a schematic flowchart of a fast data segmentation method for large-scale parallel graph neural network computing;

[0039] Figure 2 It is a partition diagram of GNN;

[0040] Figure 3 It is a traditional graph partitioning method. Detailed Implementation Modes

[0041] The following further describes the detailed implementation modes of the present invention in conjunction with the drawings and specific embodiments:

[0042] Combined with Figure 1Graph neural networks (GNNs) are a type of deep learning model specifically designed to process graph-structured data. The core idea is to leverage the graph's topological structure and node attributes to gradually update node representations to capture local or even global graph information, thereby performing tasks such as graph classification, node classification, and edge prediction. GNN models typically update the state of each node by aggregating information from neighboring nodes. The node representation of each layer depends not only on the node's own features, but is also closely related to the features of its neighbors. This aggregation process allows the node representation to gradually include graph structure information from a wider range. By stacking k GNN layers, the model can capture information about the node within a k-hop neighborhood. This multi-layer structure gives GNNs powerful expressive power to cope with complex relationships and feature distributions in large-scale graph data. The update of each layer includes information about more distant neighbors, allowing the node representation to gradually aggregate multi-level contextual information.

[0043] Take the GraphSAGE model as an example. It is a GNN method based on sampling and aggregation, which aims to process large-scale graphs through efficient sampling algorithms. Unlike traditional GNN models, it does not process all neighbor nodes at once, but randomly samples neighbors, which enables it to scale to larger graphs. Specifically, for each node v, its representation is The update at layer k can be expressed as:

[0044]

[0045] Among them, N(v) is the direct neighbor of vertex v, represents the representation of node v at the kth layer, W is the weight matrix to be learned, and σ is the nonlinear activation function. Different aggregation functions (such as mean, maximum, or LSTM) can be selected according to the application scenario to adapt to the characteristics of different graph structures. First, each node v aggregates the representations of its direct neighbor nodes into Then, the current representation of each node will be combined with the aggregated neighborhood vector Perform the connection operation and send the connection vector to the fully connected layer with a nonlinear activation function σ to obtain the representation of node v in the kth layer Through this layer-by-layer information aggregation and feature update process, GraphSAGE is able to capture local information about the graph structure in its node representations, and gradually cover a wider range of structural information through multi-layer stacking. This design demonstrates excellent scalability and expressiveness in practical applications, enabling graph neural networks to effectively process large-scale graph data containing numerous nodes and complex relationships, achieving efficient representation learning.

[0046] Distributed GNN computing requirements

[0047] As the scale of graph data continues to increase, single-machine GNN training and inference face performance bottlenecks, and distributed GNN computing emerges as the times require. Distributed GNNs work together through multiple machines to jointly complete the processing and analysis tasks of large-scale graph data. However, in a distributed environment, the graph data partitioning and sampling strategies are crucial for system performance.

[0048] The design of distributed graph neural networks generally falls into two modes: The first mode is to store different parts of the graph data on each machine. In this mode, when accessing information of remote neighbors, the system aggregates the information of neighbors through network communication. Although this method is flexible and each node can dynamically access the required data, it inevitably incurs significant network communication overhead. A large amount of data transmission not only reduces the overall performance of the system but may also cause bottlenecks under high load, thereby affecting the training efficiency of GNNs; The second mode is to locally store the node feature information that needs to be communicated in advance before the calculation starts to reduce the network communication requirements during the actual calculation process. This method attempts to improve the calculation efficiency through local storage. However, this also brings additional storage overhead and synchronization problems because each node needs to maintain a local copy of its neighbor information, resulting in increased memory usage, and when the graph structure changes, additional synchronization operations are required to ensure information consistency. Different from traditional graph computing tasks, the communication and storage requirements during the GNN calculation process not only include the graph topology but also involve rich graph feature information. This means that when GNNs process data, they need to frequently exchange feature information between each node to capture the complex dependencies between nodes. This complexity places greater pressure on GNNs in terms of communication and storage, so traditional solutions often struggle to meet their needs. Therefore, designing a reasonable data partitioning method is particularly important. This can not only effectively reduce network communication overhead but also reduce redundant storage requirements, thereby improving the training efficiency and overall performance of distributed GNNs.

[0049] Graph Partitioning Techniques and Graph Sampling Techniques

[0050] Graph partitioning techniques accelerate distributed GNN computing by reducing communication. Graph partitioning aims to split large-scale graph data into multiple subgraphs, which are distributed to different machines for processing, optimize the communication between partitions, and reduce the cross-machine data transmission overhead. Graph sampling techniques accelerate distributed GNN computing by reducing [the amount of data processed]. Graph sampling performs subset sampling on the neighbors of each node to reduce the single-node computational load.

[0051] Note: There seems to be a missing word in the description of what graph sampling techniques reduce in the original Chinese text. I filled it in as "[the amount of data processed]" to make the English translation more complete and meaningful in the context. You can adjust it according to the actual situation.In recent years, the optimization of distributed GNN computing has become a research hotspot, and related technologies have emerged continuously. For example, DSP proposed an efficient GNN training method based on multiple GPUs, which improved the training performance by optimizing data loading and communication; PaGraph improved the efficiency of large-scale GNN training through computing-aware caching technology; ByteGNN proposed an efficient GNN training strategy for large-scale graph data; P3 focused on distributed deep graph learning and achieved efficient processing of large-scale graph data. These studies have provided useful references for the optimization of distributed GNN computing, but there are still deficiencies and need further in-depth exploration.

[0052] This application proposes a graph data partitioning method for GNN, which has the following three main characteristics: (1) Different from traditional graph partitioning methods, when performing streaming graph processing on vertices, we construct a k-Hop super adjacency list, considering the distribution information of the k-hop neighbors of each node, so as to better capture the complex dependencies in the GNN computing process, which is different from traditional graph partitioning that only cares about 1-Hop neighbors; (2) We particularly focus on the impact of different types of neighbors on vertex allocation. During the GNN training process, nodes are usually divided into training nodes, test nodes, and validation nodes, and different types of neighbors have different impacts on vertices. Since multiple rounds of iteration are usually performed in the training phase, while generally only one round is performed in the test phase. Therefore, for a certain node, it would prefer to be in the same partition as its training neighbors to reduce communication overhead; (3) To further optimize the partitioning quality, we increase the expected value of its neighbors in this partition through the partition information of the already allocated nodes. That is, by jointly acting on the distribution information of the in-neighbors of each node and the out-neighbor distribution information in the k-Hop adjacency list, the effectiveness of the partitioning is enhanced.

[0053] A fast data segmentation method for large-scale parallel graph neural network computing, including:

[0054] (1) Topology-enhanced partitioning based on k-hop neighbors; First, preprocess the graph topology structure and construct a k-hop super adjacency list for each node. Specifically, given a node v, define its k-hop neighbor set N k-Hop (v) as the set of all nodes reachable within k hops, which is used to measure the number of nodes directly or indirectly associated with v in a certain partition during the node allocation process, and make allocation decisions through the optimized heuristic function (1):

[0055]

[0056] Among them, pid represents the partition number assigned to node v, which is selected from all possible partitions through the argmax operation as the partition number that makes the objective function reach the maximum value;

[0057] $i\in[1,p]$ indicates that the index $i$ of the partition is between 1 and $p$, where $p$ is the total number of partitions. Traverse all possible partitions, calculate the values of the expressions corresponding to each partition, and then select the partition number corresponding to the largest one.

[0058] represents the partition the size of the intersection with the set $N$ k-Hop (v) of the $k$-hop neighbors of node $v$. Specifically, is the set of nodes in the $i$-th partition at time step $t$, and $N$ k-Hop (v) is the set of $k$-hop neighbors of node $v$. The size of the intersection represents the number of nodes directly or indirectly associated with $v$ in partition $i$.

[0059] $W$ t (i, v) represents the weight of assigning node $v$ to partition $i$ at time step $t$. This weight can be used to measure the priority or cost of assigning node $v$ to partition $i$.

[0060] $t$ represents the time step or the number of iterations.

[0061] (2) Neighbor type weighted influence;

[0062] In GNN calculations, one of the core differences from traditional graph calculation methods is that nodes in GNNs have different types, mainly including training points, test points, and validation points. The distributions and roles of different types of nodes in the network structure are different, which has an important impact on the partition decision of target nodes in GNNs. However, existing graph partitioning methods usually do not distinguish node types. Their designs aim to minimize the number of cross-edges, mainly by partitioning nodes with more neighbors into the same partition to reduce the communication overhead across partitions. This traditional strategy is effective in general graph calculations, but it may lead to an additional burden on communication costs during GNN training. Since GNNs often require multiple rounds of iteration (e.g., 50 rounds) during the training phase, while only one round is performed during the test phase, this means that training nodes have a greater impact on the overall computational performance. Therefore, in the partitioning scheme of GNNs, how to reasonably distinguish the influence of different types of nodes during partitioning, especially increasing the partitioning weight of training nodes to reduce the overall communication cost, has become an important optimization direction. As Figure 2 shown:

[0063] Suppose the currently arrived node cur has three neighbors distributed in different partitions: one training node is in partition Par1, and two test nodes are in partition Par2. Without considering the node type, traditional graph partitioning methods tend to assign cur to the partition with more neighbors, i.e., Par2, which can reduce one cross-partition edge. However, the particularity of GNN calculation lies in that the high-frequency iterations in the training phase bring a large number of cross-partition communication requirements. If cur is assigned to Par2, although one cross-edge is reduced, the communication frequency is still high, which will increase the overall computational burden. On the contrary, if cur is assigned to Par1 containing the training node, although the number of cross-edges increases, the communication frequency of the cross-edges is low, thus reducing the communication requirements in 48 out of 50 iterations. Therefore, in the partition design of GNN, considering the influence of different types of neighbors on the partition of the target node, especially increasing the influence weight of training nodes to reduce the cost of high-frequency communication, is the key to achieving efficient distributed training.

[0064] For this reason, a partition optimization strategy based on neighbor type weighting is proposed. When allocating nodes, this strategy weights different types of nodes in the neighborhood of the target node, so that training nodes have a higher weight in the partition decision. Specifically, when the target node visits its neighborhood nodes, the partition weight of the training node is set to δ train (usually greater than 1), while the weights of test and validation nodes remain 1. When choosing a partition, the optimal partition for cur is selected according to the total weight in each partition. Through this weighting method, although the number of cross-edges may increase slightly, due to the significant decrease in communication frequency, the overall communication cost is effectively controlled.

[0065] (3) For this reason, a partition optimization strategy based on neighbor type weighting is proposed. When allocating nodes, this strategy weights different types of nodes in the neighborhood of the target node, so that training nodes have a higher weight in the partition decision. Specifically, when the target node visits its neighborhood nodes, the partition weight of the training node is set to δ train (usually greater than 1), while the weights of test and validation nodes remain 1. When choosing a partition, the optimal partition for cur is selected according to the total weight in each partition. Through this weighting method, although the number of cross-edges may increase slightly, due to the significant decrease in communication frequency, the overall communication cost is effectively controlled.

[0066] The attraction mechanism of the allocated nodes; the attraction mechanism utilizes the implicit information of the k-Hop neighbors of the allocated nodes to optimize the partitioning effect by enhancing the partition convergence of the k-hop neighbors. Specifically, when a node is assigned to a certain partition, the information of this partition will spread layer by layer to the unallocated nodes through its k-hop neighbors;

[0067] To implement the attraction mechanism, the heuristic function is improved. Given a node v to be assigned, not only the neighbors in the k-Hop adjacency list of v are considered, but also the distribution information of the neighbors within k-Hop of node v in each partition is calculated. The allocation information of the neighbors within k-Hop of the node and the allocation information of the neighbors outside k-Hop jointly act to guide the allocation of the node, and the following partition selection criteria are defined:

[0068]

[0069] Among them represents how many external neighbors of v have arrived and been placed in the i-th partition P i by time instance t so far; represents how many internal neighbors of v have arrived and been placed in the i-th partition P i by time instance t so far;

[0070] Specifically, an expected value tracking mechanism is designed. When node v is assigned to partition pid, the expected value of the k-hop external neighbor u will be updated accordingly.

[0071] In large-scale graph data, directly constructing a complete k-hop adjacency list will significantly increase the computational and storage costs. To improve the efficiency of adjacency list construction, a sampling-based adjacency list construction method is adopted, and the adjacency list structure is simplified by sampling key nodes in the neighborhood. This sampling construction method can not only improve the efficiency, but also retain the key structural information of the graph, and naturally fits with the sampling training process of the GraphSage model. When constructing the k-hop adjacency list, a layer-by-layer sampling strategy is adopted, and a part of the nodes are randomly selected from the neighborhood of each hop according to the set probability for expansion. Specifically, for the neighbor nodes of each hop, according to the set sampling ratio p k select part of the nodes to enter the adjacency list. Layer-by-layer sampling gradually constructs the adjacency list, which can effectively limit the exponential growth of the number of neighbor nodes, thereby reducing the storage overhead while ensuring that enough structural information is retained; this sampling process is regarded as a hierarchical recursive sampling: first, sample with probability p1 in the set of 1-hop neighbors, and select part of the nodes as the initial node set for expansion; then sample again with probability p2 in the 2-hop neighbors, and so on, sampling and constructing the neighbor sets of each hop layer by layer.

[0072] In practical applications, the sampling probability p kIt can be dynamically adjusted according to the graph structure characteristics and the importance of nodes. For example, in regions with a higher degree or complex neighborhoods, we can appropriately increase the sampling probability to ensure the inclusion of key nodes and their connection structures; while in regions with sparse neighborhoods and simple structures, we can reduce the sampling probability to compress the storage volume. This dynamic adjustment strategy can balance information retention and resource conservation in the adjacency list constructed by sampling, improving the partitioning efficiency. Through the k-hop super adjacency list constructed by layer-by-layer sampling, we can accurately evaluate the neighborhood distribution of nodes in different partitions during the partitioning process, while significantly reducing the construction time of the adjacency list. The constructed sampled k-Hop adjacency list not only plays a key role in graph partitioning but can also be directly applied to the first round of training of sampling-based GNN models (such as GraphSAGE). In the traditional implementation of GraphSAGE, each node needs to randomly sample k-hop neighbors to update the representation, while the preprocessed k-Hop adjacency list already contains optimized neighbor information. In this way, the sampled adjacency list is not only applicable to streaming graph partitioning tasks but can also significantly reduce the communication and computational costs in the subsequent GNN training phase.

[0073] The main purpose of the layer-by-layer sampling design is to optimize the computational efficiency. By reducing the number of nodes in the adjacency list, we can significantly reduce the storage requirements of the k-hop adjacency list, reducing the complexity and computational volume during node allocation. In addition, the hierarchical structure of sampling helps to retain key nodes within each hop of the neighborhood relationship, enabling the generated adjacency list to retain important topological information at a lower storage cost. This structured sampling method can not only accelerate the graph partitioning process but also reduce the transmission requirements for each hop during streaming partitioning, thus improving the response speed of the entire system.

[0074] While layer-by-layer sampling reduces the cost of constructing the adjacency list, it may lead to a certain loss of topological information, especially for dense local structures in the graph. Since only a partial set of neighbors is retained at each hop, some important nodes that are not sampled may be ignored, resulting in a decline in the partitioning effect when the k-hop adjacency information is incomplete. In addition, due to the randomness of sampling, the results of different samplings may lead to instability in partitioning, affecting the consistency of experimental results. To achieve a balance between benefits and costs, we designed a dynamic sampling strategy: for high-importance regions of the graph (such as nodes with a higher degree or complex neighborhood relationships), we appropriately increase the sampling probability p k to reduce the risk of information loss; for relatively sparse or information-redundant regions, we reduce the sampling probability p k to further save resources. This differential sampling strategy can achieve a better balance between information retention and computational efficiency, reducing the information loss caused by sampling and ensuring that the adjacency list constructed by sampling has high stability and scalability.

[0075] In order to reduce the computational overhead, a neighbor type weighting strategy is proposed. This strategy optimizes the partition allocation of the target node by giving the training neighbors a higher influence. Specifically, the length of the sampled k-Hop adjacency table of the target node is set to The total number of partitions is p, then ideally the average score of the target node in each partition should be In order to effectively improve the priority of training neighbors in partitioning decisions, a higher weight δ is given to training neighbors. train , and the weights of non-training neighbors remain at 1.

[0076] In the actual calculation process, when the cumulative score of the target node on a partition reaches , this partition already has a significant score advantage over other partitions. At this point, we believe that this partition is likely to be the best allocation choice for the target node. Therefore, the algorithm can terminate subsequent cumulative calculations in advance and ignore the remaining neighbors in the target node's adjacency list, thereby saving computing resources. This pruning strategy not only reduces the overhead of traversing the entire k-HOP adjacency table, but also speeds up the partition decision process without significantly affecting the partition accuracy. Since partition allocation prioritizes the score weights of training neighbors, this strategy helps to achieve balanced partitions in a shorter time. However, in order to find a balance between the efficiency of the pruning strategy and the partition accuracy, we only prune the neighbor scores δ train Pruning is only triggered when the number of neighbors exceeds the average. This way, while reducing computational overhead, the global information provided by other types of neighbors is preserved as much as possible, ensuring the integrity of the partitioning process. In general, the pseudo code is as follows:

[0077]

[0078] The input in the pseudocode is the arrival node v, its sampled k-Hop outer neighbors N k-Hop (v) and expected value expect[v], the output is the partition number pid selected by node v. Lines 1-7 calculate the scores of the outer neighbors of each partition. Specifically: Line 1 traverses the sampled k-Hop outer neighbors u of v, and lines 2-5 assign different weights to the partition where node u is located according to the node type of its outer neighbor u. Here, the training node is set to δ train , and 1 for other node types. Lines 6-7 check whether a partition's score is greater than a threshold. If so, the algorithm terminates the subsequent accumulation calculations early. Lines 8-12 obtain the PID of the partition assigned to the arrival node v. Lines 8-10 traverse all partitions. If the partition capacity of the arrival node's type is full, the partition is skipped. Lines 11-12 select the partition with the highest score based on the combined outer and inner neighbor scores and return it.

[0079] Certainly, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions, or substitutions made by those skilled in the art within the scope of the essence of the present invention shall also fall within the protection scope of the present invention.

Claims

1. A fast data segmentation method for large-scale parallel graph neural network computing, characterized in that, Including: (1) Topology Enhancement Partition Based on k-hop Neighbors; First, preprocess the graph topology to construct the k-hop super adjacency list of each node. Specifically, given a node v, define its k-hop neighbor set N k-Hop (v) as the set of all nodes reachable within k hops. This set is used to measure the number of nodes directly or indirectly associated with v in a partition during the node allocation process, and the allocation decision is made through the optimized heuristic function (1): Among them, pid represents the partition number assigned to node v, which is selected from all possible partitions through the argmax operation, and is the partition number that makes the objective function reach the maximum value; i ∈ [1, p] means that the index i of the partition is between 1 and p, where p is the total number of partitions. Traverse all possible partitions, calculate the expression values corresponding to each partition, and then select the corresponding partition number with the largest value; Indicates a partition The intersection size with the set N of k-hop neighbors of node v k-Hop (v), specifically Is the set of nodes in the i-th partition at time step t, N k-Hop (v) is the set of k-hop neighbors of node v, and the intersection size represents the number of nodes directly or indirectly associated with v in partition i; w t (i, v) represents the weight of node v assigned to partition i at time step t, which can be used to measure the priority or cost of assigning node v to partition i; t represents the time step or the number of iterations; (2) Neighbor type weighted influence; when allocating nodes, different types of nodes in the neighborhood of the target node are weighted, so that the training nodes have higher weights in the partition decision; specifically, when the target node accesses its neighborhood nodes, the partition weight of the training node is set to δ train , usually greater than 1, while the weights of the test and validation nodes remain 1. When selecting a partition, the optimal partition is selected for cur according to the sum of weights in each partition; (3) The attraction mechanism of the assigned nodes; the attraction mechanism utilizes the implicit information of the k-Hop inner neighbors of the assigned nodes to optimize the partitioning effect by enhancing the partition convergence of the k-hop neighbors. Specifically, when a node is assigned to a certain partition, the information of this partition will spread layer by layer to the unassigned nodes through its k-hop neighbors; To implement the attraction mechanism, the heuristic function is improved. Given an unassigned node v, not only consider the neighbors in the k-Hop adjacency list of v, but also calculate the distribution information of the k-Hop inner neighbors of node v in each partition. The allocation information of the k-Hop inner neighbors reaching the node and the allocation information of the k-Hop outer neighbors work together to guide the allocation of the node, and the following partition selection criteria are defined: Among them indicates how many external neighbors of v have arrived and been placed in the i-th partition P so far at time instance t i in indicates how many internal neighbors of v have arrived and been placed in the i-th partition P so far at time instance t i in; In particular, an expected value tracking mechanism is designed. When node v is assigned to partition pid, the expected value of the k-hop outer neighbor u will be updated accordingly.

2. The fast data segmentation method for large-scale parallel graph neural network computing according to claim 1, characterized in that To improve the efficiency of constructing the adjacency list, a sampling-based adjacency list construction method is adopted; when constructing the k-hop adjacency list, a layer-by-layer sampling strategy is used to randomly select some nodes from the neighborhood of each hop for expansion. Specifically, for the neighbor nodes of each hop, according to the set sampling ratio p k Select some nodes to enter the adjacency list and gradually construct the adjacency list layer by layer; This sampling process is regarded as a hierarchical recursive sampling: first sample with probability p1 in the set of 1-hop neighbors, and select some nodes as the initial node set for expansion; Then sample with probability p2 in the 2-hop neighbors, and so on, sampling and constructing the neighbor sets of each hop layer by layer.

3. The fast data segmentation method for large-scale parallel graph neural network computing according to claim 1, wherein In order to reduce the computational overhead, a neighbor type weighting strategy is proposed. This strategy optimizes the partition allocation of the target node by giving the training neighbors a higher influence. Specifically, the length of the sampled k-Hop adjacency table of the target node is set to The total number of partitions is p, then ideally the average score of the target node in each partition should be In order to effectively improve the priority of training neighbors in partitioning decisions, a higher weight δ is given to training neighbors. train , and the weights of non-training neighbors remain at 1.

Images