Graph neural network-based graph partitioning method and system for heterogeneous environment

Abstract

The application discloses a heterogeneous environment graph neural network degree classification-based graph partitioning method and system, and comprises the following steps: determining the classification threshold of high-degree vertices and low-degree vertices according to the power-law characteristics of the vertex degree distribution in the input graph neural network, and classifying the vertices into low-degree vertices and high-degree vertices; using a constrained METIS method to perform initial subgraph partitioning on the low-degree vertices; performing initial subgraph partitioning on the high-degree vertices, calculating the locality score and the load balancing score for the high-degree vertices partitioned into different partitions, weighting and summing the locality score and the load balancing score into the comprehensive score of the corresponding partition, and selecting the partition with the highest comprehensive score as the partitioning result, each partition corresponding to a computing node in the heterogeneous environment. The application aims to achieve heterogeneous device-aware load balancing while optimizing the cutting rate, and improve the overall performance and resource utilization of the distributed graph neural network training system.

Description

Technical Field

[0001] This invention relates to the field of computer resource scheduling technology in heterogeneous environments, specifically to a graph partitioning method and system based on degree classification for graph neural networks in heterogeneous environments. Background Technology

[0002] Graph Neural Networks (GNNs), as a key technology supporting data-intensive applications such as social network analysis and intelligent recommendation systems, are facing bottlenecks in computing and storage resources caused by the exponential growth in the scale of graph data structures. Traditional centralized GNN architectures exhibit drawbacks such as excessive GPU memory consumption during model training, making it difficult to meet the real-world demands of large-scale graph data processing. Distributed GNN architectures based on data parallelism improve training efficiency by dividing the original graph data into multiple subgraphs and implementing a multi-node collaborative computing mechanism, possessing the scalability to process ultra-large-scale graph datasets in the cloud. However, in heterogeneous computing environments, the heterogeneity between devices and the power-law property of the vertex degree distribution of graph data cause significant load skew in traditional load balancing strategies based on the uniform partitioning assumption during subgraph partitioning tasks. This leads to systemic bottlenecks such as decreased computing resource utilization efficiency and a surge in communication overhead, severely restricting the actual performance of distributed GNN training systems. In distributed training scenarios, significant advancements have been made in methods for partitioning large-scale graph data in recent years to reduce training time. Traditional methods such as METIS and LDG aim to reduce the number of cut edges between subgraphs (cutting rate) to lower cross-node communication costs, i.e., reducing the total training time by decreasing the overall load on each subgraph. However, current mainstream methods still face two problems in heterogeneous environments: First, they do not flexibly allocate subgraph load based on device performance differences, easily leading to load imbalance. For example, METIS assumes all devices have the same computing performance, resulting in subgraphs with similar vertex counts. However, for nodes with different performance levels, almost identical loads can cause high-performance training nodes to wait for low-performance training nodes to complete their training tasks, resulting in a "weakest link" effect. Second, using vertex count instead of edge count to estimate load leads to a mismatch between load allocation and actual performance. For example, SUGAR's core assumption is flawed: it calculates the number of vertices to measure the load of each subgraph. In reality, the degree of vertices in a graph often follows a power-law distribution, meaning most vertices have relatively few neighbors, while a few vertices have many neighbors. The load of high-degree vertices, which aggregate a large amount of neighbor vertex information, is much higher than that of low-degree vertices; simply calculating the number of vertices cannot reflect the true load of each subgraph. In fact, the computational and communication load of each partition is linearly related to the number of edges, because the training of graph neural networks aggregates information through edges. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a graph partitioning method and system based on degree classification for graph neural networks in heterogeneous environments, which addresses the above-mentioned problems in the prior art. This invention aims to achieve load balancing of heterogeneous device perception while optimizing the edge cutting rate, thereby improving the overall performance and resource utilization of the distributed graph neural network training system.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0005] A degree-based graph partitioning method for graph neural networks in heterogeneous environments includes the following steps: determining classification thresholds for high-degree vertices and low-degree vertices based on the power-law characteristics of the degree distribution of vertices in the input graph neural network; classifying vertices in the graph neural network into low-degree vertices and high-degree vertices according to the classification thresholds; performing initial subgraph partitioning for low-degree vertices using the constrained METIS method, with each partition corresponding to a computing node in the heterogeneous environment; performing initial subgraph partitioning for high-degree vertices, calculating locality scores and load balancing scores for high-degree vertices assigned to different partitions, weighted summing the locality scores and load balancing scores to obtain a comprehensive score for each partition, and selecting the partition with the highest comprehensive score as the partitioning result, with each partition corresponding to a computing node in the heterogeneous environment.

[0006] Optionally, determining the classification thresholds for high-degree vertices and low-degree vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network includes: fitting a power-law probability density function to the vertex degree distribution in the input graph neural network; determining whether the fitting result of the power-law probability density function conforms to the power-law distribution; if it conforms to the power-law distribution, calculating the classification thresholds for high-degree vertices and low-degree vertices based on the significance of the power-law distribution; if it does not conform to the power-law distribution, setting the classification thresholds for high-degree vertices and low-degree vertices to 100%, so that all vertices in the graph neural network are classified as low-degree vertices.

[0007] Optionally, the probability density function of the power-law distribution is expressed as: ,in Indicates degree The probability density of the vertex degree distribution; This is a scaling constant used to adjust the scaling of the function; The exponential parameter is used to represent the significance of the power-law distribution of the function; when determining whether the distribution conforms to a power-law distribution based on the fitting result of the probability density function of the power-law distribution, this includes using the exponential parameter obtained from the fitting. With proportionality constant Calculate the goodness of fit R of the probability density function of the power-law distribution. 2 If the goodness of fit R 2If the result is greater than a preset threshold, the fitting result of the probability density function of the power-law distribution is determined to conform to the power-law distribution; otherwise, the fitting result of the probability density function of the power-law distribution is determined to not conform to the power-law distribution.

[0008] Optionally, the calculation of the classification thresholds for high and low vertices based on the significance of the power-law distribution refers to using the exponential parameter used to represent the significance of the power-law distribution of the function. Substitute the preset piecewise function to obtain the classification thresholds for high-degree vertices and low-degree vertices.

[0009] Optionally, the function expression of the preset piecewise function is:

[0010] ,

[0011] in, These are the quantiles corresponding to the classification thresholds for high-degree vertices and low-degree vertices. and It is a constant parameter.

[0012] Optionally, the initial subgraph partitioning of low-degree vertices using the constrained METIS method includes: obtaining the performance score of each computing node in the heterogeneous environment, dividing the performance score of each computing node by the total score to obtain the proportion of the ideal size of the partition corresponding to each computing node, and calling the METIS method to inject the proportion of the ideal size of the partition corresponding to each computing node as a weight proportion into the constraint, thereby treating each low-degree vertex as having the same weight for partitioning, with each partition corresponding to a computing node in the heterogeneous environment, and making the proportion of the number of vertices in each partition close to the proportion of the ideal size of the partition.

[0013] Optionally, the locality score is the ratio of all neighboring vertices of a height vertex within one hop of the partitioned area to all neighboring vertices of that height vertex; the load balancing score is calculated by first calculating the load ratio between the load of the partition to be partitioned and the target load, and then subtracting the load ratio from 1; the function expression for the weighted sum of the locality score and the load balancing score to the comprehensive score assigned to the corresponding partition is:

[0014] ,

[0015] in, For the height vertex Divide into partitions Overall score For load balancing weights, Scoring for load balancing For local weights, For local scoring, load balancing weights are used. To score load balancing The bias is mapped to the [0,1] interval using the hyperbolic tangent function, and the load balancing weight is obtained. and local weights The sum of the two is 1.

[0016] Furthermore, the present invention also provides a graph partitioning system for degree-based classification of graph neural networks in heterogeneous environments, including an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the graph partitioning method for degree-based classification of graph neural networks in heterogeneous environments.

[0017] Furthermore, the present invention provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute, via a processor, the degree-based classification graph partitioning method for graph neural networks in heterogeneous environments.

[0018] Furthermore, the present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the degree-based classification graph partitioning method for graph neural networks in heterogeneous environments via a processor.

[0019] Compared with existing technologies, the present invention can mainly achieve the following beneficial effects: (1) Existing graph partitioning methods often do not consider the structural characteristics of graph data, and use the number of vertices in the subgraph to represent the load of partitioning, without considering the load difference caused by high-degree vertices and low-degree vertices. The method of the present invention considers the power-law distribution characteristics of graph data, designs a dynamic threshold selection strategy, adaptively classifies high-degree and low-degree vertices, and achieves the significance of power-law distribution and accurate matching of classification through three-level thresholds. (2) The current mainstream GNN processing library defaults to METIS uniform partitioning for distributed GNN tasks, which will cause serious load imbalance in heterogeneous environments and reduce training efficiency. The advantage of the method of the present invention is that: edge size constraints are introduced in the low-degree vertex partitioning module and the high-degree vertex partitioning module to ensure that the number of edges in the subgraph basically matches the performance score of the node, making the training time of each node closer and reducing waiting latency. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the basic principle of the method in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0022] Figure 3 The results are comparative experiments on the Flickr and OGBN-arxiv datasets in this embodiment of the invention.

[0023] Figure 4This presents the comparative experimental results on the Reddit dataset and the ogbn-products dataset in this embodiment of the invention. Detailed Implementation

[0024] This invention aims to address the problems of load imbalance and low communication efficiency caused by device heterogeneity and the power-law characteristics of graph data in distributed GNN training scenarios. Addressing two core shortcomings of existing graph partitioning strategies based on the uniform partitioning assumption (such as METIS) in heterogeneous environments—a lack of awareness of differences in device computing performance leading to load imbalance (the "weakest link" effect), and using vertex count instead of edge count to estimate load resulting in a severe mismatch between subgraph partitioning and load—this invention proposes a degree-based load-balanced graph partitioning method. By introducing vertex degree classification and a dynamic weighted scoring mechanism, it achieves load balancing perceptible to heterogeneous devices while optimizing the edge-cutting rate, effectively improving the overall performance and resource utilization of the distributed GNN training system. To enable those skilled in the art to better understand the technical solution of this invention, the following will provide a more detailed description of the technical solution in conjunction with the accompanying drawings of the embodiments of this invention.

[0025] like Figure 1 and Figure 2 As shown, the degree-based graph partitioning method (LBDH) for heterogeneous environments in this embodiment includes the following steps: determining the classification thresholds for high-degree vertices and low-degree vertices based on the power-law characteristics of the degree distribution of vertices in the input graph neural network; classifying vertices in the graph neural network into low-degree vertices and high-degree vertices based on the classification thresholds; performing initial subgraph partitioning for low-degree vertices using the constrained METIS method, with each partition corresponding to a computing node in the heterogeneous environment; performing initial subgraph partitioning for high-degree vertices, calculating locality scores and load balancing scores for high-degree vertices assigned to different partitions, weighted summing the locality scores and load balancing scores to obtain a comprehensive score for the corresponding partition, and selecting the partition with the highest comprehensive score as the partitioning result, with each partition corresponding to a computing node in the heterogeneous environment.

[0026] This embodiment uses the idea of ​​vertex classification to reduce the cutting rate on the basis of load balancing, and designs three modules: (1) Vertex classification module: First, it is necessary to determine the threshold for classifying high and low degree vertices based on the power law characteristics of the vertex degree distribution in the original graph, and then classify low degree vertices and high degree vertices. (2) Low degree vertex partitioning module: The METIS method with constraints is used to partition the initial subgraph for low degree vertices to reduce the cutting rate and ensure that the total training interval is low. (3) High degree vertex partitioning module: High degree vertices are partitioned. At this time, the distribution of neighboring vertices and the edge size of each subgraph are considered comprehensively. Locality score and load balancing score are calculated for high degree vertices to be partitioned into different partitions. Dynamic weighted scoring is performed, and the partition with the highest comprehensive score is selected as the partitioning result. Among them, the distribution of neighboring vertices affects the cutting rate between subgraphs, and the edge size of the subgraph affects the load balancing. Through the weight adjustment during dynamic weighted scoring, the two objectives are balanced. The synergistic effect of the low degree vertex partitioning module and the high degree vertex partitioning module can further enhance the effectiveness of the overall strategy. It should be noted that the order of low-degree vertex partitioning and high-degree vertex partitioning can be chosen as needed. For example, low-degree vertex partitioning can be performed first, followed by high-degree vertex partitioning. Figure 2 As shown; alternatively, the high-degree vertices can be divided first, and then the low-degree vertices can be divided.

[0027] This embodiment uses the idea of ​​vertex classification. Traditional graph partitioning methods directly apply the same partitioning to all vertices, ignoring the impact of vertex degree on load and communication overhead. High-degree vertices have a large number of neighboring edges; if traditional vertex partitioning is used, some subgraph partitions may become overloaded due to containing multiple high-degree vertices, creating performance bottlenecks. Low-degree vertices have fewer neighboring edges; if edge partitioning is used, creating multiple replica vertices may introduce additional storage and synchronization overhead. Partitioning vertices based on their degree avoids the shortcomings of single partitioning.

[0028] Determining the classification thresholds for high and low vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network involves two sub-steps: power-law distribution fitting evaluation and dynamic threshold optimization. The former calculates the ideal power-law function and then evaluates the power-law characteristics of the vertex degree distribution through a goodness-of-fit test. When a significant power-law distribution characteristic is detected, it indicates a significant difference between high and low vertices. In this embodiment, determining the classification thresholds for high and low vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network includes: fitting the probability density function of the power-law distribution to the vertex degree distribution in the input graph neural network; determining whether the fitting result conforms to the power-law distribution; if it conforms, calculating the classification thresholds for high and low vertices based on the significance of the power-law distribution; if it does not conform, setting the classification thresholds for high and low vertices to 100%, so that all vertices in the graph neural network are classified as low-degree vertices.

[0029] In this embodiment, the probability density function of the power-law distribution is expressed as follows: ,in Indicates degree The probability density of the vertex degree distribution; This is a scaling constant used to adjust the scaling of the function; is the exponential parameter, used to represent the significance of the power-law distribution of the function. Taking the natural logarithm of both sides of the probability density function of the power-law distribution yields:

[0030] ,

[0031] This indicates that in a log-log coordinate system, the probability density function graph of a power-law distribution will appear as a straight line with a slope of The intercept is By analyzing the degree distribution characteristics of vertices, a corresponding probability density function can be fitted. In this embodiment, when determining whether a power-law distribution conforms to the fitted probability density function, the method includes using the fitted exponential parameter... With proportionality constant Calculate the goodness of fit R of the probability density function of the power-law distribution. 2 If the goodness of fit R 2 If the fit is greater than a preset threshold, the probability density function of the power-law distribution is considered to fit the power-law distribution; otherwise, the fit is considered not to fit the power-law distribution. In linear regression, the goodness of fit R0 is... 2 This represents the proportion of the variation in the dependent variable that can be explained by the independent variable. In relevant fields, experience suggests that a double log-linear relationship based on a power-law distribution typically requires a high goodness of fit R0. 2(Generally considered to be greater than 0.8) to ensure the reliability of the fit. Then, based on the goodness-of-fit R... 2 Determine whether to adopt a vertex classification strategy. If the determination is successful, proceed to the dynamic threshold selection step. When the goodness of fit R... 2 When the significance criterion is not met, the vertex degree distribution usually exhibits a uniform characteristic, with no vertices showing significant height. In this case, the optimization effect of degree-based classification strategies is limited, and the threshold can be directly set to 100%, without vertex classification. By uniformly applying a low-degree vertex partitioning strategy, the additional overhead of dynamic threshold calculation is avoided, thus optimizing computational efficiency while ensuring the rationality of the partitioning.

[0032] In this embodiment, calculating the classification thresholds for high and low vertices based on the significance of the power-law distribution refers to using the exponential parameter used to represent the significance of the power-law distribution of the function. Substitute the preset piecewise function to obtain the classification thresholds for high-degree vertices and low-degree vertices.

[0033] In this embodiment, regarding the decay rate of the power-law distribution of degree in the graph, this step uses a three-level dynamic threshold selection strategy to provide suitable thresholds for vertex classification. After confirming vertex classification, it is necessary to determine the thresholds for classifying high-degree vertices and low-degree vertices. The exponential parameters generated in the fitting step can be used. To design a heuristic function for the threshold. When As the threshold increases, the steepness of the power-law distribution curve intensifies, manifested as a sharp increase in the degree of high-degree vertices and an exponential decrease in their number. The time cost incurred by high-degree vertices during the partitioning process is greater; therefore, increasing the threshold can improve the distinction between high-degree and low-degree vertices, reducing the number of high-degree vertices and lowering the time cost. Based on the Pareto principle (80% of the results are often determined by 20% of the causes or inputs), we find that in a power-law distribution… This also determines the proportion of the sum of the degrees of the vertices to the sum of the degrees of all vertices. For example, as an optional implementation, the preset function expression of the piecewise function in this embodiment is:

[0034] ,

[0035] in, These are the quantiles corresponding to the classification thresholds for high-degree vertices and low-degree vertices. and This is a constant parameter. Depending on the significance of the power-law distribution, the dynamic threshold optimization step adaptively increases the classification boundary threshold from 80% to 99%, improving partitioning efficiency by reducing the proportion of high-resolution vertices. The first-level threshold of 80% is inspired by the Pareto principle (80 / 20 rule), illustrating the extreme imbalance of the power-law distribution. The third-level threshold of 99% is inspired by the P99 latency, often defined as the time it takes for 99% of data packets to complete transmission when evaluating network performance. This is also the gold standard in engineering practice, and experimental results have demonstrated the effectiveness of this threshold setting.

[0036] This strategy analyzes exponential parameters. Regarding the influence of power-law distribution, a three-level dynamic threshold selection strategy was determined: (1) The power-law distribution decays slowly, and the difference between high and low degree vertices is not particularly obvious. The sum of the degrees of the top 20% of the high degree vertices accounts for about 80% of the total degree sum, which basically reflects the characteristic of the high degree vertices occupying most of the degree in the power-law distribution. This threshold was inspired by the Pareto principle and determined by parametric experiments. (2) The power-law distribution has moderate decay and needs to be passed through... The value is used to dynamically determine the quantiles of the threshold. Here Set it to 75.54. Set it to 3.316 to keep the piecewise function of this dynamic threshold selection strategy continuous. Use Dynamically mapped quantiles preserve the mathematical properties of power-law distributions, i.e. The effect on the threshold is exponential. (3) The power-law distribution decays rapidly, resulting in very few vertices with high height. At this point, the sum of the degrees of vertices representing only 1% of the total height accounts for 99% of the total degree. This threshold, inspired by the P99 delay, was confirmed through parameter experiments. The three-level dynamic threshold selection strategy significantly improves the flexibility and scene adaptability of threshold setting through a multi-level adaptive adjustment mechanism. It effectively overcomes the limitations of static threshold division in traditional methods such as PowerLyra (e.g., reliance on manual experience and difficulty adapting to complex graph data), thereby achieving more accurate node classification.

[0037] The partitioning of low-degree vertices is a fundamental step in the overall process. This strategy uses a DGL-improved METIS to construct a balanced framework for low-degree vertices, allowing subsequent allocation of high-degree vertices to be optimized based on the existing subgraph structure, ultimately achieving the dual goals of computational efficiency and load balancing. The scientific basis of this strategy stems from the characteristics of low-degree vertices: they constitute a large proportion of the total number of vertices but have low degrees, and the degree differences between vertices are small, making them less prone to load imbalance and suitable for vertex partitioning. The limitation of the METIS method in power-law graph scenarios mainly stems from high-degree vertices: their numerous connections can create computational bottlenecks and affect partitioning quality. However, in subgraphs composed of low-degree vertices, the advantages of METIS are retained: 1) it still maintains a low number of cross-partition edges; 2) due to the small differences in the number of vertex connections, the load balancing strategy based on the number of vertices is close to that based on the number of edges; 3) the total processing volume is reduced by more than 80%, resulting in a significant improvement in method efficiency.

[0038] In this embodiment, the initial subgraph partitioning of low-degree vertices using the constrained METIS method includes: obtaining the performance scores of each computing node in the heterogeneous environment; dividing the performance score of each computing node by the total score to obtain the proportion of the ideal partition size corresponding to each computing node; and calling the METIS method to inject the proportion of the ideal partition size corresponding to each computing node as a weight proportion into the constraints, thereby treating each low-degree vertex as having the same weight for partitioning. Each partition corresponds to a computing node in the heterogeneous environment, and the proportion of the number of vertices in each partition is close to the proportion of the ideal partition size. For example, when the node performance score is [30, 30, 40], by dividing the performance score of each node by the total score (which is 100 in this case), the proportion of the ideal partition size is set to [0.3, 0.3, 0.4]. Theoretically, each node will complete training simultaneously. Subsequently, the improved METIS method in the DGL library is called to inject the proportion of the ideal partition size as a weight proportion into the constraints. The improved METIS treats each low-degree vertex as having the same weight. To balance the total weight of vertices in each partition to meet the target proportion as much as possible, the proportion of the number of vertices in each partition will be close to the proportion of the ideal partition size. Finally, the total number of edges within each partition is recorded. This data reflects the load information of each partition after the low-degree vertex partitioning. This statistical method quickly filters edges with both ends being low-degree vertices through vectorization operations, and counts the number of directed edges within each partition whose source vertex belongs to that partition (undirected edges are decomposed into two directed edges with different source vertices). The initial load of each subgraph after low-degree vertex partitioning is recorded. After the low-degree vertex partitioning is completed, the high-degree vertex subgraphs will be partitioned based on the low-degree vertex partitioning results.

[0039] This embodiment's method (LBDH) determines the final partition of a high-degree vertex by comparing the scores of its partitioning into different partitions. The scoring fits the two final objectives of graph partitioning, obtained by multiplying the locality score and the load balancing score by their respective weights and then summing the results. Initially, the first high-degree vertex is partitioned using the result of the low-degree vertex partitioning, and this partitioning result is continuously updated during the partitioning process. Simultaneously, the load balancing score is calculated using the number of edges, avoiding the load imbalance problem in GNNs caused by the traditional method of using vertex count to represent load. Instead, the total number of edges within each partition after low-degree vertex partitioning represents the initial load of each partition. The goal of this step is to assign high-degree vertices to partitions that maximize locality gain and minimize load imbalance. This goal stems from the topological characteristics of high-degree vertices: their large degree amplifies the impact of allocation decisions on the number of edges across partitions and the distribution of computational load, requiring a refined evaluation strategy to achieve a dynamic balance between locality and load balancing. The high-degree vertex partitioning strategy includes a key mechanism: a dynamic weight adjustment mechanism. This mechanism adjusts the load balancing score's weight by sensing the gap between the current subgraph's edge size and the final target load, ensuring load balancing while fully considering locality scores. Specifically, for each vertex to be assigned, it iterates through all possible target partitions, calculating a dynamically weighted composite score of its locality score and load balancing score.

[0040] In this embodiment, the locality score is the ratio of all neighboring vertices of a height vertex within one hop of the partitioned region to all neighboring vertices of that height vertex. Assigned to partition When locality score is reached, it can be expressed as: Locality score Reflecting the vertices to be divided Assigned to partition The potential gain on the cutting rate. This is determined by all neighboring vertices within the l-hop of the current vertex to be partitioned in the target partition. Total neighbor vertices The locality score is calculated based on the proportion of locality. A higher locality score indicates fewer cross-node data accesses and shorter data loading time during distributed GNN training. The expression for the computation function is:

[0041] ,

[0042] Among them, its Represents the node number. , Represents the number of partitions.

[0043] In this embodiment, the load balancing score is calculated by first calculating the load ratio between the load of the partition to be divided and the target load, and then subtracting this load ratio from 1. The height vertex is then used to determine the score. Assigned to partition When the load balancing score is reached, it can be expressed as: Load balancing score Then, by comparing the load of the current partition with the final target load, the normalized deviation value is calculated, guiding the ratio of the number of edges in each partition to converge to the preset ratio. The higher the score, the closer the current partition's load share is to the final target load share. Load Balancing Score The expression for the computation function is:

[0044] ,

[0045] in, Represents the current partition load, calculated by partition The ratio of the number of edges in a partition to the total number of edges in all partitions is used to obtain the result. The target load for the partition is represented by the compute node. The performance score of a node is determined by its proportion of the total score of all nodes. These two scores guide the allocation of height vertices under different objectives, and their importance varies in different environments. For example, in load balancing, the load balancing score should have a smaller weight to promote locality. Therefore, height vertex partitioning relies on a dynamic weight adjustment mechanism to synthesize the total score. This mechanism dynamically adjusts the weight coefficients of the two scores by monitoring the deviation of the edge ratio of each partition from the target value. When the system detects that the edge ratio of the current partition is close to the preset target ratio, the method automatically reduces the load balancing control weight, prioritizing the locality characteristics of height vertices, thereby reducing unnecessary damage to locality caused by load balancing. Conversely, when the edge ratio of a partition deviates significantly from the target value, the system quickly suppresses further accumulation of deviation by increasing the load balancing weight coefficient, preventing partition size imbalance.

[0046] To achieve this goal, the method in this embodiment (LBDH) uses the following function expression to weightedly sum the locality score and the load balancing score to obtain the comprehensive score assigned to the corresponding partition:

[0047] ,

[0048] in, For the height vertex Divide into partitions Overall score For load balancing weights, Scoring for load balancing For local weights, For local scoring, load balancing weights are used. To score load balancing The bias is mapped to the [0,1] interval using the hyperbolic tangent function, and the load balancing weight is obtained. and local weights The sum of the two is 1. First, calculate the deviation between the current edge ratio and the target value, which can be obtained from the load balancing score. This is represented by the hyperbolic tangent function. The bias is then mapped to the [0,1] interval using the hyperbolic tangent function to obtain the load balancing weight. :

[0049] ,

[0050] Among them, local weights Then with load balancing weight Complementarity ensures the total weight sums to 1. The scaling factor in this function... This is used to control the sensitivity of weight changes, ensuring that a high local weight is maintained with slight deviations, while triggering strong equilibrium regulation with significant deviations. (Modified) Different load balancing targets can be set, and the effect is that the partition edge size deviation usually does not exceed [a certain value]. This is used to improve the applicability of the method. In this experiment, we set up... This indicates that the load balancing objective in this experiment is to ensure that the partition edge size deviation does not exceed 10%, which is the optimal parameter obtained from the experiment. Final node Assigned to partition Overall rating It is composed of a linear combination of locality score and load balancing score with dynamic weights:

[0051] ,

[0052] By comparing the overall scores in different zones Select the section with the highest overall score. As the final partition, the height vertices are assigned to that partition. After each height vertex allocation, the partition state is incrementally updated, allowing subsequent vertex allocations to be aware of the cumulative effect of historical decisions and preventing early allocation load imbalances from being amplified by later decisions. For example, when partitioning... The current partition load is caused by historical allocation. Greater than the partition target load At that time, its load balancing score The value will become negative, and subsequent vertices will tend to be allocated to other low-load partitions, forming a negative feedback adjustment. By alternating between high-degree vertex partitioning and partition state updates, the partitioning of all high-degree vertices is finally completed, effectively achieving the effect of edge partitioning. The innovation of this strategy is reflected in two aspects: First, by using a dynamic weight adjustment mechanism to decouple the local adjacency characteristics of high-degree vertices from the global load state evaluation, it avoids the load imbalance caused by excessive locality bias in traditional greedy methods such as LDG, and also overcomes the problem of random allocation completely ignoring the topological structure. Second, by using incremental updates to achieve real-time perception of partition state, the method of this embodiment (LBDH) can adaptively adjust the allocation strategy and gradually approach the target ratio. In summary, the advantages of the method of this embodiment (LBDH) are as follows: (1) Existing graph partitioning methods often do not consider the structural characteristics of graph data, use the number of vertices in the subgraph to represent the load situation of the partition, and do not consider the load difference brought by high-degree vertices and low-degree vertices. The method in this embodiment (LBDH) considers the power-law distribution characteristics of graph data and designs a dynamic threshold selection strategy to adaptively classify high and low degree vertices. It achieves the significance of the power-law distribution and accurate matching of classification through a three-level threshold. (2) The current mainstream GNN processing library defaults to METIS uniform partitioning for distributed GNN tasks, which will cause serious load imbalance and reduce training efficiency in heterogeneous environments. The advantage of the method in this embodiment (LBDH) is that it introduces edge size constraints in the low-degree vertex partitioning module and the high-degree vertex partitioning module to ensure that the number of edges in the subgraph basically matches the performance score of the node, making the training time of each node closer and reducing waiting latency.

[0053] To verify the effectiveness and performance of the method in this embodiment, a distributed training cluster consisting of four training nodes was set up, with each node connected via gigabit Ethernet. Four graph datasets with different characteristics and sizes were selected to construct a comparative experimental system, allowing for a systematic analysis of the different impacts of node feature dimensions, connection density, and scale on graph partitioning methods, as shown in Table 1.

[0054] Table 1: Information Table for Four Types of Graph Datasets

[0055]

[0056] To test the impact of model aggregation strategies on different partitioning methods, this embodiment selects two widely applicable graph neural network models: GCN and GraphSAG. GCN's aggregation strategy requires processing all neighbor nodes in the graph to update features. Unlike GCN, GraphSAG randomly samples a fixed number of nodes from each node's neighbors. This aggregation strategy reduces the number of neighbors processed in each convolution operation, lowering computational and storage requirements. To ensure the fairness of model training evaluation, the experiments refer to best practices recognized in existing literature, uniformly configuring model parameters to eliminate the impact of parameter selection on model performance evaluation. During distributed model training, a mini-batch training strategy is adopted. The batch size is set to 10% of the training set size to ensure that the total batch size processed by multiple training nodes in each experiment is similar. Simultaneously, to prevent excessively large batch sizes from causing memory overflow, a maximum batch size is set for each node. As a comparison with the method (LBDH) in this embodiment, the RH (RandomHash) method, METIS method, and SUGAR method are used. The RH (RandomHash) method achieves fast graph partitioning by randomly assigning nodes to different subgraphs. The METIS method is the default method used in DGL 2.4.0 and is also the most commonly used method in industry. The SUGAR method is a resource-aware graph partitioning method specifically designed for training large-scale graph neural networks. Its core objective is to achieve efficient subgraph-level optimization in distributed training through dynamic subgraph expansion and explicit memory control. The final results are as follows... Figure 3 and Figure 4 As shown, where Figure 3 The results of the comparative experiments on the Flickr dataset and the OGBN-arxiv dataset are as follows. Figure 4 The comparison results are shown on the Reddit and OGBN-products datasets. The partitioning time is the arithmetic mean of the partitioning times for each graph partitioning method, run five times on each dataset. Observation Figure 3 and Figure 4As can be seen, on the large-scale Reddit and OGBN-Products datasets, the partitioning time of the proposed method (LBDH) is significantly lower than that of the METIS and SUGAR methods, with reductions of up to 42.2% and 34.6%, respectively. However, on smaller datasets, such as the smaller Flickr and OGBN-Arxiv datasets, the partitioning time of the proposed method (LBDH) is slightly higher than that of the METIS and SUGAR methods. This is because although the classification of the proposed method (LBDH) introduces preprocessing overhead (such as threshold calculation), the classification strategy reduces the actual amount of data processed by decomposing the high-dimensional problem into low-dimensional sub-problems. Compared to the additional cost, its benefits are lower on small- to medium-sized datasets but higher on large-scale power-law distribution graphs. The training objective of distributed GNNs is to process large-scale graph data; therefore, the proposed method (LBDH) has more advantages in practical applications. Regardless of the dataset size, the RH method uses random partitioning, resulting in the shortest partitioning time. This is because the RH method ignores the structural features of the graph, reducing a significant amount of computational overhead. However, this approach of ignoring the structural features of the graph is detrimental to subsequent classification accuracy, making it difficult to achieve a balance between overhead and accuracy.

[0057] To quantify the unevenness of training time among heterogeneous nodes in the distributed training process of a graph neural network, the average time of 10 epochs for each training node was used to calculate the coefficient of variation (CV) for the four training nodes. The core idea is to standardize the standard deviation as a percentage of the mean, thereby eliminating the influence of dimensions. The coefficient of variation is a relative indicator that measures the dispersion of data and is suitable for comparing the volatility of data with different means. The final results are shown in Tables 2 and 3.

[0058] Table 2: Experimental results of final training time and coefficient of variation on GCN

[0059]

[0060] Table 3: Experimental results of final training time and coefficient of variation on GraphSAGE

[0061]

[0062] Referring to the experimental results in Tables 2 and 3, the following conclusions can be drawn: 1) When using the method of this embodiment (LBDH) for partitioning on the GCN and GraphSAGE models, the final training time and coefficient of variation are both less than those of the METIS method. Compared to the METIS method, the training time is reduced by up to 35.6%, and the coefficient of variation is reduced by up to 80.3%. Compared to the SUGAR method, the coefficient of variation of the method of this embodiment (LBDH) is lower in all aspects, and the final training time is higher only on the combination of the GCN model and the ogbn-arxiv dataset. This is because the dataset size is smaller, and the optimization of the edge cutting rate of the SUGAR method can more easily offset its disadvantage of uneven load. However, distributed GNN training mainly processes large-scale graph data in heterogeneous environments. In real-world environments, the overall optimization of the method of this embodiment (LBDH) is better than that of the METIS and SUGAR methods. The reduction in training time and coefficient of variation in the method of this embodiment (LBDH) is mainly due to its heuristic approach of evaluating the performance of training nodes. It assigns a load suitable for the training performance of each training node based on the number of edges in the subgraph, achieving load balancing among nodes and enabling them to complete training tasks almost simultaneously, thus reducing waiting latency between nodes. In contrast, the METIS and SUGAR methods only uniformly partition the nodes of the subgraph, failing to consider that the number of edges in a GNN is the key factor affecting the computational load of each subgraph, resulting in a first-order imbalance. The METIS method also ignores the performance differences of training nodes in heterogeneous environments, causing a second-order imbalance. The SUGAR method only uses GPU memory size as the basis for graph partitioning optimization, without considering the complex environment of GNN training, so its performance is only slightly better than the METIS method. 2) Simultaneously observing the impact of model and average degree on the training time of the method of this embodiment (LBDH), it was found that the more neighbor samples the model uses (compared to partial neighbor sampling in GraphSAGE, full neighbor sampling in the GCN model aggregates more nodes), and the higher the average degree, the more significant the optimization of training time. The reason is that the larger the neighbor sampling and the average degree of the vertices in the model, the better the information is generated when each height vertex is aggregated in the GNN. l-hopThe larger the neighbor subgraph, the greater the load difference with low-degree vertices. The RH, METIS, and SUGAR methods assume that each vertex contributes the same load, thus their load balancing strategies are less effective. 3) The RH method has the longest training time in all cases. Compared to the METIS method, the training time increased by up to 163.9%. This is because the random partitioning used in the RH method results in weak adjacency among nodes within each subgraph, disrupting the topology of the original graph data. Therefore, almost all of the original graph data is needed during aggregation, causing extensive cross-node communication and significantly reducing training speed. While the coefficient of variation in training time is optimized because each training node needs the same original graph data, this optimization is less significant in practical training compared to its high GPU memory usage and long training time. 4) During the experiment, it was found that training could not be completed when using the GCN model and the Reddit dataset. This is because the average degree of the Reddit dataset is high, while the GCN model samples all neighbors, resulting in a large number of nodes sampled in each batch during training. Furthermore, the large feature dimensions of the nodes result in a huge GPU memory overhead when training each batch. Taking training node 1 with 6GB of GPU memory as an example, its maximum batch size is only about 30, its training set size is greater than 30,000, and a single training round takes more than 3 hours, and it is prone to GPU memory shortages. Nevertheless, the method in this embodiment (LBDH) is also representative and convincing on other datasets.

[0063] In summary, this embodiment of the graph partitioning method based on degree classification for heterogeneous environment graph neural networks aims to solve the problems of low resource utilization and low training efficiency caused by uneven load in the training of distributed graph neural networks. The main work completed is as follows: (1) Considering the power-law distribution characteristics of graph data, a dynamic threshold selection strategy is designed to adaptively classify high and low degree vertices. The significance of the power-law distribution and the accurate adaptation of classification are achieved through three levels of thresholds. (2) A hierarchical partitioning mechanism is proposed. First, METIS partitioning with weight constraints is used for low degree vertices to reduce the edge cutting rate and build a balanced subgraph structure. (3) For the partitioning of high degree vertices, dynamic weight adjustment is introduced. By sensing the current status of each partition, the weights of locality score and load balance score are dynamically adjusted to optimize the balance between locality and the current partition load. By the mixed weighting of locality score and load balance score, the comprehensive score of high degree vertices partitioned into different partitions is obtained, thereby determining the optimal partitioning of high degree vertices. Experiments show that the proposed method (LBDH) outperforms traditional methods in heterogeneous environments: compared to METIS and SUGAR methods, its graph partitioning time is reduced by up to 42.2% and 34.6% on large-scale datasets, respectively. During the distributed training phase, the proposed method (LBDH) reduces the final training time by up to 35.6% and the coefficient of variation by up to 80.3%, achieving dual optimization of total training time and load balancing.

[0064] Furthermore, this embodiment also provides a graph partitioning system for degree-based classification of graph neural networks in heterogeneous environments, including an interconnected microprocessor and a memory. The microprocessor is programmed or configured to execute the graph partitioning method for degree-based classification of graph neural networks in heterogeneous environments. Additionally, this embodiment provides a graph partitioning system for degree-based classification of graph neural networks in heterogeneous environments, including a vertex classification program unit, a low-degree vertex partitioning program unit, and a height vertex partitioning program unit. The vertex classification program unit is used to confirm whether the degree distribution of vertices in the original graph conforms to a power-law distribution through power-law distribution fitting evaluation, and then decides whether to perform height and low-degree vertex classification. The threshold for vertex classification is determined by dynamic threshold selection, breaking through the limitations of traditional static thresholds. The classification granularity is dynamically adjusted. The high-degree and low-degree vertices in the classification graph lay the foundation for differentiated partitioning. The low-degree vertex partitioning unit first calculates the target load ratio based on the node's computing power. Then, it assigns a unit weight to each low-degree vertex and calls the improved METIS method to ensure the sum of the vertex weights in the subgraph conforms to the target ratio, achieving initial load balancing. Finally, it counts the actual number of edges in each partition as a baseline load, providing data support for subsequent high-degree vertex partitioning. This unit utilizes the small degree difference of low-degree vertices to avoid the shortcomings of traditional METIS in power-law graphs, while significantly reducing the edge-cutting rate. The high-degree vertex partitioning unit introduces a dynamic weight adjustment mechanism. By sensing the current state of each partition, it dynamically adjusts the weights of locality scores and load balancing scores to optimize the balance between locality and the current partition load. Through a weighted average of locality and load balancing scores, a comprehensive score for high-degree vertices partitioned into different partitions is obtained, thus determining the optimal partitioning of high-degree vertices. Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the degree-based classification graph partitioning method for heterogeneous environment graph neural networks via a processor. Additionally, this embodiment also provides a computer program product including a computer program or instructions programmed or configured to execute the degree-based classification graph partitioning method for heterogeneous environment graph neural networks via a processor.

[0065] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A graph partitioning method based on degree classification for graph neural networks in heterogeneous environments, characterized in that, The process includes the following steps: determining classification thresholds for high-degree and low-degree vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network; classifying vertices in the graph neural network into low-degree and high-degree vertices based on the classification thresholds; performing initial subgraph partitioning for low-degree vertices using the constrained METIS method, with each partition corresponding to a computing node in a heterogeneous environment; performing initial subgraph partitioning for high-degree vertices, calculating locality scores and load balancing scores for different partitions assigned to high-degree vertices, weighted summing the locality scores and load balancing scores to obtain a comprehensive score for each partition, and selecting the partition with the highest comprehensive score as the partitioning result, with each partition corresponding to a computing node in a heterogeneous environment. The process of determining the classification thresholds for high and low vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network includes: fitting a power-law probability density function to the vertex degree distribution in the input graph neural network; determining whether the fitting result of the power-law probability density function conforms to a power-law distribution; and if it conforms to a power-law distribution, calculating the classification thresholds for high and low vertices based on the significance of the power-law distribution. Calculating the classification thresholds for high and low vertices based on the significance of the power-law distribution refers to using the exponential parameter used to represent the significance of the power-law distribution function. Substitute the preset piecewise function to obtain the classification thresholds for high-degree vertices and low-degree vertices; if they do not conform to the power-law distribution, set the classification thresholds for high-degree vertices and low-degree vertices to 100%, so that all vertices in the graph neural network are classified as low-degree vertices.

2. The graph partitioning method based on degree classification for graph neural networks in heterogeneous environments according to claim 1, characterized in that, The probability density function of the power-law distribution is expressed as follows: ,in Indicates degree The probability density of the vertex degree distribution; This is a scaling constant used to adjust the scaling of the function; The exponential parameter is used to represent the significance of the power-law distribution of the function; when determining whether the distribution conforms to a power-law distribution based on the fitting result of the probability density function of the power-law distribution, this includes using the exponential parameter obtained from the fitting. With proportionality constant Calculate the goodness of fit R of the probability density function of the power-law distribution. 2 If the goodness of fit R 2 If the result is greater than a preset threshold, the fitting result of the probability density function of the power-law distribution is determined to conform to the power-law distribution; otherwise, the fitting result of the probability density function of the power-law distribution is determined to not conform to the power-law distribution.

3. The graph partitioning method based on degree classification for graph neural networks in heterogeneous environments according to claim 1, characterized in that, The function expression of the preset piecewise function is: ， in, These are the quantiles corresponding to the classification thresholds for high-degree vertices and low-degree vertices. and It is a constant parameter.

4. The graph partitioning method based on degree classification for graph neural networks in heterogeneous environments according to claim 1, characterized in that, The initial subgraph partitioning of low-degree vertices using the constrained METIS method includes: obtaining the performance score of each computing node in the heterogeneous environment; dividing the performance score of each computing node by the total score to obtain the proportion of the ideal size of the partition corresponding to each computing node; calling the METIS method to inject the proportion of the ideal size of the partition corresponding to each computing node as a weight proportion into the constraint, thereby treating each low-degree vertex as having the same weight for partitioning, with each partition corresponding to a computing node in the heterogeneous environment, and making the proportion of the number of vertices in each partition close to the proportion of the ideal size of the partition.

5. The graph partitioning method based on degree classification for graph neural networks in heterogeneous environments according to claim 1, characterized in that, The locality score is the proportion of all neighboring vertices of a given height vertex within one hop of the partitioned area to the total number of neighboring vertices of that height vertex; the load balancing score is calculated by first calculating the load ratio between the load of the partition to be partitioned and the target load, and then subtracting this load ratio from 1; the function expression for the weighted sum of the locality score and the load balancing score to the comprehensive score assigned to the corresponding partition is as follows: ， in, For the height vertex Divide into partitions Overall score For load balancing weights, Scoring for load balancing For local weights, For local scoring, load balancing weights are used. To score load balancing The bias is mapped to the [0,1] interval using the hyperbolic tangent function, and the load balancing weight is obtained. and local weights The sum of the two is 1.

6. A graph partitioning system based on degree classification for graph neural networks in heterogeneous environments, comprising interconnected microprocessors and memory, characterized in that, The microprocessor is programmed or configured to execute the degree-based graph partitioning method for graph neural networks in heterogeneous environments as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the degree-based graph partitioning method for graph neural networks in heterogeneous environments as described in any one of claims 1 to 5.

8. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the degree-based graph partitioning method for graph neural networks in heterogeneous environments as described in any one of claims 1 to 5.

Images