Graph division method and system oriented to heterogeneous environment graph neural network and based on degree classification

Abstract

The invention discloses a degree classification-based graph division method and system for a heterogeneous environment graph neural network, and the method comprises the steps: determining the classification threshold values of a high vertex and a low vertex according to the power law characteristics of the vertex degree distribution in an input graph neural network, and classifying the vertexes into the low vertex and the high vertex; performing initial sub-graph division on the low-degree vertex by using a constrained METIS method; performing initial subgraph division on the height vertex, calculating a locality score and a load balance score for dividing the height vertex into different partitions, performing weighted summation on the locality score and the load balance score to obtain a comprehensive score divided into the corresponding partition, and selecting the partition with the highest comprehensive score as a division result; each partition corresponds to one computing node in the heterogeneous environment. The invention aims to realize load balancing sensed by heterogeneous equipment while optimizing the edge cutting rate, and improve the overall performance and the resource utilization rate of a distributed graph neural network training system.

Description

Technical Field

[0001] The present invention relates to the technical field of computer resource scheduling in heterogeneous environments, and in particular to a graph partitioning method and system based on degree classification for a graph neural network in a heterogeneous environment. Background Art

[0002] Graph Neural Networks (GNNs), a key technology supporting data-intensive applications such as social network analysis and intelligent recommendation systems, are facing computational and storage resource bottlenecks caused by the exponential growth of graph data structures. Traditional centralized GNN architectures suffer from drawbacks such as excessive GPU memory usage during model training, making them incapable of meeting the demands of large-scale graph data processing. Distributed GNN architectures based on data parallelism improve GNN training efficiency by partitioning the original graph data into multiple subgraphs, enabling multi-node collaborative computing. These architectures offer scalability for processing extremely large graph datasets in the cloud. However, in heterogeneous computing environments, the heterogeneity between devices and the power-law nature of the degree distribution of graph data leads to significant load skew in subgraph partitioning tasks using traditional load balancing strategies based on the uniform partitioning assumption. This leads to systemic bottlenecks such as decreased computational resource utilization and increased communication overhead, severely hampering the practical performance of distributed GNN training systems. To reduce training time in distributed training scenarios, partitioning methods for large-scale graphs have made significant progress in recent years. Traditional methods such as METIS and LDG aim to reduce cross-node communication costs by reducing the number of edges cut between subgraphs (the edge cutting rate). This means reducing the total training time by reducing the total load on each subgraph. However, current mainstream methods still face two problems in heterogeneous environments. First, they lack the flexibility to distribute subgraph load based on device performance differences, which can easily lead to load imbalance. For example, METIS assumes that all devices have equal computing performance, resulting in subgraphs with similar numbers of vertices. However, for nodes with different performance, nearly identical loads can easily cause high-performance training nodes to wait for low-performance nodes to complete training tasks, creating a "barrel effect." Second, load estimation uses the number of vertices instead of the number of edges, resulting in a mismatch between load distribution and actual performance. For example, SUGAR's core assumption is flawed: it measures the load of each subgraph by counting the number of vertices. In reality, the degrees of vertices in a graph often follow a power-law distribution, meaning that most vertices have relatively few neighbors, while a few have a large number of neighbors. High-degree vertices, which aggregate extensive neighbor information, carry a much higher load than low-degree vertices. Measuring the number of vertices alone cannot fully reflect the true load of each subgraph. In fact, the computational load and communication load of each partition are linearly related to the number of edges, because the training of graph neural networks aggregates information through edges. Summary of the Invention

[0003] The technical problem to be solved by the present invention: In response to the above-mentioned problems of the prior art, a graph partitioning method and system based on degree classification for heterogeneous environment graph neural networks are provided. The present invention aims to achieve load balancing of heterogeneous device perception while optimizing the cutting rate, and improve the overall performance and resource utilization of the distributed graph neural network training system.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: A graph partitioning method based on degree classification for heterogeneous environment graph neural networks includes the following steps: determining the classification threshold of high-degree vertices and low-degree vertices according to the power-law characteristics of the vertex degree distribution in the input graph neural network, and classifying the vertices in the graph neural network into low-degree vertices and high-degree vertices according to the classification threshold; performing initial subgraph partitioning on the low-degree vertices using the constrained METIS method, and each partition obtained by the partition corresponds to a computing node in the heterogeneous environment; performing initial subgraph partitioning on the high-degree vertices, calculating locality scores and load balancing scores for the high-degree vertices divided into different partitions, weightedly summing the locality scores and load balancing scores to obtain a comprehensive score for the partitions, and selecting the partition with the highest comprehensive score as the partitioning result, and each partition corresponds to a computing node in the heterogeneous environment.

[0005] Optionally, the method of determining the classification thresholds of high-degree vertices and low-degree vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network includes: using the vertex degree distribution in the input graph neural network to fit the probability density function of the power-law distribution, judging whether it conforms to the power-law distribution based on the fitting result of the probability density function of the power-law distribution, and if it conforms to the power-law distribution, calculating the classification thresholds of high-degree vertices and low-degree vertices based on the significance of the power-law distribution; if it does not conform to the power-law distribution, setting the classification thresholds of high-degree vertices and low-degree vertices to 100%, so that all vertices in the graph neural network are classified as low-degree vertices.

[0006] Optionally, the function expression of the probability density function of the power-law distribution is: ,in Indicates degree The probability density of the vertex degree distribution; is a proportional constant used to adjust the scaling of the function; is an exponential parameter used to indicate the significance of the power law distribution of the function; when judging whether the fitting result of the probability density function of the power law distribution conforms to the power law distribution, the exponential parameter obtained by the fitting is included. and the proportional constant Calculate the goodness of fit R of the probability density function of the power law distribution 2 If the goodness of fit R 2If the value is greater than a preset threshold, it is determined that the fitting result of the probability density function of the power-law distribution conforms to the power-law distribution; otherwise, it is determined that the fitting result of the probability density function of the power-law distribution does not conform to the power-law distribution.

[0007] Optionally, the classification thresholds for calculating high-degree vertices and low-degree vertices according to the significance of the power law distribution refer to the exponential parameter used to represent the significance of the power law distribution function. Substitute the preset piecewise function to obtain the classification thresholds for high-degree vertices and low-degree vertices.

[0008] Optionally, the function expression of the preset piecewise function is: , in, is the quantile corresponding to the classification threshold of high-degree vertices and low-degree vertices, and is a constant parameter.

[0009] Optionally, the initial subgraph partitioning of low-degree vertices using the constrained METIS method includes: obtaining the performance score of each computing node in the heterogeneous environment, dividing the performance score of each computing node by the total score to obtain the ratio of the ideal size of the partition corresponding to each computing node, calling the METIS method to inject the ratio of the ideal size of the partition corresponding to each computing node as the weight ratio into the constraint, so that each low-degree vertex is regarded as having the same weight for partitioning, each partition corresponds to a computing node in the heterogeneous environment, and the ratio of the number of vertices in each partition is close to the ratio of the ideal size of the partition.

[0010] Optionally, the locality score is the ratio of all neighbor vertices of a high-degree vertex within one hop of the partition to be divided to all neighbor vertices of the high-degree vertex; the load balancing score is calculated by first calculating the load ratio between the load of the partition to be divided and the target load, and then subtracting the load ratio from 1 to obtain the result; the function expression for weighted summation of the locality score and the load balancing score to obtain the comprehensive score of the partition to be divided is: , in, is the height vertex Divide into partitions The comprehensive rating of is the load balancing weight, Score the load balancing score. is the locality weight, is the locality score, where the load balancing weight Score the load balancing The hyperbolic tangent function is used to map the deviation to the interval [0,1], and the load balancing weight and locality weight The sum of the two is 1.

[0011] In addition, the present invention also provides a graph partitioning system based on degree classification for heterogeneous environment graph neural networks, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural networks.

[0012] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural networks through a processor.

[0013] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural networks through a processor.

[0014] Compared with the existing technology, the present invention can mainly achieve the following beneficial effects: (1) Existing graph partitioning methods often do not consider the structural characteristics of graph data, and use the number of vertices in the subgraph to represent the load of the partition, without considering the load difference caused by high-degree vertices and low-degree vertices. The method of the present invention takes into account the power-law distribution characteristics of graph data, designs a dynamic threshold selection strategy, adaptively classifies high-degree and low-degree vertices, and achieves the significance of power-law distribution and accurate adaptation of classification through three-level thresholds. (2) The current mainstream GNN processing library defaults to METIS uniform partitioning for distributed GNN tasks, which will cause serious load imbalance in heterogeneous environments and reduce training efficiency. The advantage of the method of the present invention is that it introduces edge scale constraints in the low-degree vertex partitioning module and the high-degree vertex partitioning module to ensure that the number of edges in the subgraph basically meets the performance score of the node, making the training time of each node closer and reducing waiting delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the basic principle of the method of the embodiment of the present invention.

[0016] Figure 2 Schematic diagram of the basic process of the method of the embodiment of the present invention.

[0017] Figure 3 These are comparative experimental results on the Flickr dataset and the ogbn-arxiv dataset in an embodiment of the present invention.

[0018] Figure 4 These are comparative experimental results on the Reddit dataset and the ogbn-products dataset in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention aims to solve the problems of load imbalance and low communication efficiency caused by device heterogeneity and the power-law characteristics of graph data in distributed GNN training scenarios. In response to the two core defects of existing graph partitioning strategies based on the uniform partitioning assumption (such as METIS) in heterogeneous environments - the lack of awareness of differences in device computing performance leads to unbalanced load distribution (the "barrel effect"), and the use of the number of vertices instead of the number of edges to estimate the load, resulting in a serious mismatch between subgraph partitioning and load. The present invention proposes a load-balanced graph partitioning method based on degree classification. By introducing vertex degree classification and a dynamic weighted scoring mechanism, it achieves heterogeneous device-aware load balancing while optimizing the edge cutting rate, effectively improving the overall performance and resource utilization of the distributed GNN training system. In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0020] like Figure 1 and Figure 2 As shown, the graph partitioning method based on degree classification for heterogeneous environment graph neural networks (LBDH for short) of this embodiment includes the following steps: determining the classification thresholds of high-degree vertices and low-degree vertices according to the power-law characteristics of the vertex degree distribution in the input graph neural network, and classifying the vertices in the graph neural network into low-degree vertices and high-degree vertices according to the classification thresholds; performing initial subgraph partitioning on low-degree vertices using the constrained METIS method, and each partition obtained by the partition corresponds to a computing node in the heterogeneous environment; performing initial subgraph partitioning on high-degree vertices, calculating locality scores and load balancing scores for the high-degree vertices divided into different partitions, weighting the locality scores and load balancing scores to obtain a comprehensive score for the corresponding partitions, and selecting the partition with the highest comprehensive score as the partitioning result, and each partition corresponds to a computing node in the heterogeneous environment.

[0021] The method of this embodiment uses the idea of ​​vertex classification to reduce the edge cutting rate on the basis of load balancing, and designs three modules: (1) Vertex classification module: First, it is necessary to determine the threshold for high-degree and low-degree vertex classification based on the power law characteristics of the vertex degree distribution in the original graph, and then classify low-degree and high-degree vertices. (2) Low-degree vertex partitioning module: Use the constrained METIS method to perform initial subgraph partitioning on low-degree vertices to reduce the edge cutting rate and ensure that the total training space is low. (3) High-degree vertex partitioning module: Divide high-degree vertices. At this time, the distribution of neighbor vertices and the edge scale of each subgraph are comprehensively considered. The locality score and load balancing score are calculated for the high-degree vertices to be divided into different partitions, and dynamic weighted scoring is performed. The partition with the highest comprehensive score is selected as the partition result. The distribution of neighbor vertices affects the edge cutting rate between subgraphs, and the edge scale of the subgraph affects load balancing. By adjusting the weight during dynamic weighted scoring, the two goals are balanced. The synergy between the low-degree vertex partitioning module and the high-degree vertex partitioning module can further enhance the effectiveness of the overall strategy. It should be noted that the order of low-degree vertex division and high-degree vertex division can be selected as needed. For example, the low-degree vertex division can be performed first, and then the high-degree vertex division can be performed. Figure 2 As shown; in addition, you can first perform high-degree vertex division and then perform low-degree vertex division.

[0022] The method of this embodiment uses the idea of ​​vertex classification. Traditional graph partitioning methods directly perform the same partitioning on all vertices, ignoring the impact of vertex degree on load and communication overhead. High-degree vertices have a large number of adjacent edges. If traditional vertex partitioning is used, some subgraph partitions may be overloaded due to the presence of multiple high-degree vertices, forming a performance bottleneck. Low-degree vertices have fewer adjacent edges. If edge partitioning is used, additional storage and synchronization overhead may be introduced due to the creation of multiple copy vertices. Dividing vertices differently according to their degree can avoid the defects of a single partition.

[0023] Determining the classification thresholds of high-degree vertices and low-degree vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network includes two sub-steps: power-law distribution fitting evaluation and dynamic threshold optimization. The former is used to calculate the ideal power-law function, and then evaluate the power-law characteristics of the vertex degree distribution through a goodness-of-fit test. When a significant power-law distribution feature is detected, it indicates that there is a significant difference between high-degree vertices and low-degree vertices. In this embodiment, determining the classification thresholds of high-degree vertices and low-degree vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network includes: using the vertex degree distribution in the input graph neural network to fit the probability density function of the power-law distribution, judging whether it conforms to the power-law distribution based on the fitting result of the probability density function of the power-law distribution, and if it conforms to the power-law distribution, calculating the classification thresholds of high-degree vertices and low-degree vertices based on the significance of the power-law distribution; if it does not conform to the power-law distribution, setting the classification thresholds of high-degree vertices and low-degree vertices to 100% so that all vertices in the graph neural network are classified as low-degree vertices.

[0024] In this embodiment, the function expression of the probability density function of the power law distribution is: ,in Indicates degree The probability density of the vertex degree distribution; is a proportional constant used to adjust the scaling of the function; is an exponential parameter used to indicate the significance of the power law distribution function. Taking the natural logarithm of both sides of the probability density function of the power law distribution yields: , This shows that in the double logarithmic coordinate system, the probability density function graph of the power law distribution will appear as a straight line with a slope of , the intercept is By analyzing the distribution characteristics of the vertex degree, the corresponding probability density function can be fitted. In this embodiment, when judging whether it conforms to the power law distribution according to the fitting result of the probability density function of the power law distribution, the exponential parameter obtained according to the fitting is included. and the proportional constant Calculate the goodness of fit R of the probability density function of the power law distribution 2 If the goodness of fit R 2 If it is greater than the preset threshold, it is determined that the fitting result of the probability density function of the power law distribution conforms to the power law distribution; otherwise, it is determined that the fitting result of the probability density function of the power law distribution does not conform to the power law distribution. In linear regression, the goodness of fit R 2 Indicates the proportion of the dependent variable's variation that can be explained by the independent variable. In related field experience, the double logarithmic linear relationship of the power law distribution usually requires a higher goodness of fit R 2 (usually considered to be greater than 0.8) to ensure the reliability of the fit. 2Determine whether to adopt the vertex classification strategy. If it passes the judgment, enter the dynamic threshold selection step. 2 When the significance criterion is not met, the vertex degree distribution is typically uniform, with no vertices of significant height. In this case, the optimization effect of a degree-based partitioning strategy is limited. Therefore, the threshold can be set to 100% without performing any vertex classification. By uniformly applying a low-degree vertex partitioning strategy, the overhead of dynamic threshold calculations can be avoided, optimizing computational efficiency while ensuring a reasonable partitioning.

[0025] In this embodiment, the classification thresholds of high-degree vertices and low-degree vertices are calculated based on the significance of the power law distribution. Substitute the preset piecewise function to obtain the classification thresholds for high-degree vertices and low-degree vertices.

[0026] In this embodiment, the decay rate of the power law distribution of the degree in the graph is considered. This step provides a suitable threshold for vertex classification through a three-level dynamic threshold selection strategy. After confirming the vertex classification, it is necessary to confirm the threshold for classifying high-degree vertices and low-degree vertices. The exponential parameter generated in the fitting step can be used. To design a heuristic function for the threshold. As the value increases, the power law distribution curve becomes steeper, which is manifested as a sharp increase in the degree of high-degree vertices while the number decreases exponentially. During the division process, high-degree vertices bring greater time overhead, so by increasing the threshold, the distinction between high-degree vertices and low-degree vertices can be increased, the number of high-degree vertices can be reduced, and the time overhead can be reduced. According to the 80 / 20 rule (80% of the results are often determined by 20% of the reasons or inputs), we find that in the power law distribution, It also determines the ratio of the sum of the degrees of the high-degree vertices to the sum of the degrees of all vertices. For example, as an optional implementation, the function expression of the piecewise function preset in this embodiment is: , in, is the quantile corresponding to the classification threshold of high-degree vertices and low-degree vertices, and is a constant parameter. As the power-law distribution becomes more pronounced, the dynamic threshold optimization step adaptively increases the classification boundary threshold from 80% to 99%, improving segmentation efficiency by reducing the proportion of high-degree vertices. The 80% threshold for the first level is inspired by the 80 / 20 rule, which accounts for the extreme imbalance of power-law distributions. The 99% threshold for the third level is inspired by the fact that when evaluating network performance, the latency required for 99% of packets to complete transmission (P99 latency) is often defined as a quality of service guarantee. This is also the gold standard in engineering practice, and the effectiveness of this threshold setting has been demonstrated in experimental results.

[0027] The strategy analyzes the index parameters For the influence of power-law distribution, a three-level dynamic threshold selection strategy was determined: (1) :The power law distribution decays slowly, and the difference between high and low degree vertices is not particularly obvious. The sum of the degrees of the top 20% of high degree vertices accounts for about 80% of the total degree sum, which basically reflects the characteristic that high degree vertices occupy most of the degrees in the power law distribution. This threshold is inspired by the 80 / 20 rule and determined by parameter experiments. (2) : The power law distribution has moderate attenuation and needs to be passed The value of is used to dynamically determine the threshold quantile. Set to 75.54, Set to 3.316 to keep the piecewise function of the dynamic threshold selection strategy continuous. Dynamically mapping quantiles preserves the mathematical properties of the power-law distribution, i.e. The impact on the threshold is exponential. (3) The power-law distribution decays quickly, and high-degree vertices are extremely rare. At this point, the sum of the degrees of vertices with only 1% of the total degree accounts for 99% of the total. This threshold is inspired by the P99 delay and confirmed through parameter experiments. The three-level dynamic threshold selection strategy significantly improves the flexibility and adaptability of threshold settings through a multi-level adaptive adjustment mechanism. This effectively overcomes the limitations of static threshold division in traditional methods such as PowerLyra (such as reliance on manual experience and difficulty adapting to complex graph data), thereby achieving more accurate node classification.

[0028] The partitioning of low-degree vertices is a fundamental part of the overall process. This strategy uses the DGL-improved METIS to process low-degree vertices and build a balancing framework, so that the subsequent allocation of high-degree vertices can be optimized based on the existing subgraph structure, ultimately achieving the dual goals of computational efficiency and load balancing. The scientific nature of this strategy stems from the characteristics of low-degree vertices: the number of low-degree vertices is large but the degree is low, and the degree difference between vertices is small, which is not easy to cause load imbalance and is suitable for point partitioning. The limitations of the METIS method in power-law graph scenarios mainly come from high-degree vertices: their large number of connections will create computational bottlenecks and affect the partitioning quality. However, in subgraphs composed of low-degree vertices, the advantages of METIS are retained: 1) the number of cross-partition edges can still be kept low; 2) due to the small difference in the number of vertex connections, the load balancing strategy based on the number of vertices is close to the load balancing strategy based on the number of edges; 3) the total processing volume is reduced by more than 80%, which significantly improves the efficiency of the method.

[0029] In this embodiment, the constrained METIS method is used to perform initial subgraph partitioning on low-degree vertices. The method includes obtaining the performance score of each compute node in the heterogeneous environment, dividing each node's performance score by the total score to obtain the ideal partition size ratio corresponding to each compute node, and then invoking the METIS method to inject the ideal partition size ratio corresponding to each compute node as a weight ratio constraint. This allows each low-degree vertex to be treated as having equal weight for partitioning. Each partition corresponds to a compute node in the heterogeneous environment, and the ratio of the number of vertices in each partition is close to the ideal partition size ratio. For example, when the node performance scores are [30, 30, 40], by dividing each node's performance score by the total score (which is 100 in this case), the ideal partition size ratio is set to [0.3, 0.3, 0.4]. In theory, all nodes will complete training simultaneously. Subsequently, the improved METIS method in the DGL library is invoked, injecting the ideal partition size ratio as a weight ratio constraint. The improved METIS method treats each low-degree vertex as having equal weight. To minimize the total weight of each partition's vertices and ensure that it meets the target ratio, the ratio of the number of vertices in each partition is close to the ideal partition size ratio. Finally, the total number of edges within each partition is recorded. This data reflects the load of each partition after the low-degree vertex partitioning. This statistical method uses vectorized operations to quickly filter edges whose ends are both low-degree vertices. It also counts the number of directed edges whose source vertices belong to each partition (undirected edges are split into two directed edges with different source vertices). This records the initial load of each subgraph after the low-degree vertex partitioning. After the low-degree vertex partitioning is completed, the high-degree vertex subgraph is partitioned based on the low-degree vertex partitioning results.

[0030] This embodiment's method (LBDH) determines the final partitioning of high-degree vertices by comparing their scores when assigned to different partitions. The scores align with the two ultimate goals of graph partitioning and are obtained by multiplying the locality score and the load balance score by their respective weights and then summing them. At the beginning of a high-degree vertex partition, the result of the first high-degree vertex partitioned using low-degree vertices represents the current subgraph partitioning. The partitioning results are continuously updated throughout the partitioning process. The load balance score is calculated using the number of edges, avoiding the load imbalance problem often associated with traditional methods that use vertex count to represent load in GNNs. The total number of edges within each partition after the low-degree vertex partitioning is completed represents the initial load of each partition. The goal of this step is to assign high-degree vertices to partitions that maximize locality gains and minimize load imbalance. This goal stems from the topological properties of high-degree vertices: their large degree causes assignment decisions to amplify the number of edges and computational load distribution across partitions. A refined evaluation strategy is required to achieve a dynamic balance between locality and load balance. The high-degree vertex partitioning strategy includes a key mechanism: a dynamic weight adjustment mechanism. This mechanism detects the gap between the current subgraph's edge size and the final target load and adjusts the weight of the load balancing score to ensure load balancing while also taking locality into account. Specifically, for each high-degree vertex to be assigned, it traverses all possible target partitions and calculates a dynamically weighted composite score of its locality score and load balancing score.

[0031] In this embodiment, the locality score is the ratio of all neighbor vertices of the high-degree vertex within one hop of the partition to all neighbor vertices of the high-degree vertex. Assign to partition When , the locality score can be expressed as , locality score Reflects the vertex to be divided Assign to partition The potential gain of the posterior cutting rate. All neighboring vertices of the current vertex to be divided within the l-hop of the target partition Total neighbor vertices The higher the locality score, the less cross-node data access is required in distributed GNN training, and the shorter the data loading time is. The calculation function expression is: , Among them, represents the node number, , Indicates the number of partitions.

[0032] In this embodiment, the load balancing score is calculated by first calculating the load ratio between the load of the partition to be divided and the target load, and then subtracting the load ratio from 1 to obtain the value. Assign to partition When , the load balancing score can be expressed as , load balancing score By comparing the load of the current partition with the final target load, the normalized deviation value is calculated to guide the edge ratio between each partition to converge to the preset ratio. The higher the value, the closer the load ratio of the current partition is to the final target load ratio. The calculation function expression is: , in, Represents the current partition load, by calculating the partition The ratio of the number of edges in to the total number of edges in all partitions is obtained. Represents the partition target load, then compute nodes The performance score of a node is calculated as the ratio of the node performance score to the total score of all nodes. These two scores guide the allocation of high-level vertices based on different objectives, and their importance varies in different environments. For example, in the case of load balancing, the load balancing score should account for a smaller proportion to promote locality. Therefore, the division of high-level vertices relies on a dynamic weight adjustment mechanism to integrate the total score. This mechanism dynamically adjusts the weight coefficients of the two scores by monitoring the deviation between the edge ratio of each partition and the target value. When the system detects that the edge ratio of the current partition is close to the preset target ratio, the method will automatically reduce the load balancing control weight, prioritize the local characteristics of high-level vertices, and thus reduce the damage to local characteristics caused by unnecessary load balancing. Conversely, when the partition edge ratio deviates significantly from the target value, the system quickly suppresses the further accumulation of deviations by increasing the load balancing weight coefficient to prevent imbalance in partition size.

[0033] To achieve this goal, the method (LBDH) of this embodiment adds the weighted sum of the locality score and the load balancing score to obtain the function expression of the comprehensive score divided into the corresponding partitions: , in, is the height vertex Divide into partitions The comprehensive rating of is the load balancing weight, Score the load balancing score. is the locality weight, is the locality score, where the load balancing weight Score the load balancing The hyperbolic tangent function is used to map the deviation to the interval [0,1], and the load balancing weight and locality weight The sum of the two is 1. First, calculate the deviation between the current edge ratio and the target value, which can be obtained by the load balancing score Then the hyperbolic tangent function is used to map the deviation to the [0,1] interval to obtain the load balancing weight : , Among them, the local weight The load balancing weight Complementary, ensuring that the total weight sum is 1. The scale factor in this function Used to control the sensitivity of weight changes, ensuring that high local weights are maintained in the event of slight deviations, while triggering strong balancing controls in the event of significant deviations. You can set different load balancing targets, the effect is that the size deviation of the partition edge usually does not exceed , used to improve the applicability of the method. In this experiment, , which means that the load balancing target in this experiment is that the deviation of the partition edge scale does not exceed 10%. This is the optimal parameter obtained in the experiment. Assign to partition Overall rating It is composed of a linear combination of the locality score and the load balancing score according to dynamic weights: , By comparing the comprehensive scores in different partitions , select the partition with the highest comprehensive score As the final partition, the high-degree vertices are assigned to this partition. After each high-degree vertex is assigned, the partition state is incrementally updated so that the subsequent vertex assignments can perceive the cumulative effect of historical decisions and avoid the load imbalance of early assignments being amplified by subsequent decisions. For example, when the partition Partition load on the current partition due to historical allocations Greater than the partition target load When the load balancing score It will become a negative number, and subsequent vertices will be more inclined to be allocated to other low-load partitions, forming a negative feedback regulation. By cross-processing the high-degree vertex partitioning and partition state updating, the partitioning of all high-degree vertices is finally completed, and the effect of edge partitioning is actually achieved. The innovation of this strategy is reflected in two aspects: First, the local adjacency characteristics of high-degree vertices are decoupled from the global load state through the dynamic weight adjustment mechanism, which not only avoids the load imbalance caused by the excessive bias towards locality of traditional greedy methods such as LDG, but also overcomes the problem of random allocation completely ignoring the topological structure. Second, the real-time perception of the partition state is achieved through incremental updates, so that the method of this embodiment (LBDH) can adaptively adjust the allocation strategy and gradually approach the target ratio. In summary, the advantages of the method of this embodiment (LBDH) are in the following two aspects: (1) Existing graph partitioning methods often do not consider the structural characteristics of the graph data, and use the number of vertices in the subgraph to represent the load of the partition, without considering the load difference brought by high-degree vertices and low-degree vertices. The method of this embodiment (LBDH) takes into account the power-law distribution characteristics of graph data, designs a dynamic threshold selection strategy, adaptively classifies high-degree and low-degree vertices, and achieves accurate adaptation of the power-law distribution significance and classification through a three-level threshold. (2) The current mainstream GNN processing library defaults to METIS uniform partitioning for distributed GNN tasks, which can cause serious load imbalance in heterogeneous environments and reduce training efficiency. The advantage of the method of this embodiment (LBDH) is that it introduces edge scale constraints in the low-degree vertex partitioning module and the high-degree vertex partitioning module to ensure that the number of edges in the subgraph basically meets the performance score of the node, making the training time of each node closer and reducing waiting delay.

[0034] To verify the effectiveness and performance of this embodiment's method, a distributed training cluster consisting of four training nodes was set up, connected by a gigabit Ethernet cable. Four graph datasets with different characteristics and sizes were selected to construct a comparative experimental system. This allowed for a systematic analysis of the impact of node feature dimensions, connection density, and scale on the graph partitioning method, as shown in Table 1.

[0035] Table 1: Information table of 4 graph datasets

[0036] To test the impact of model aggregation strategies on different partitioning methods, the experiments in this example selected two widely applicable graph neural network models: GCN and GraphSAGE. GCN's aggregation strategy processes all neighboring nodes in the graph to achieve feature updates. Unlike GCN, GraphSAGE randomly samples a fixed number of nodes from each node's neighbors. This aggregation strategy reduces the number of neighbors processed for each convolution operation, lowering computational and storage requirements. To ensure fairness in model training and evaluation, the experiments referenced best practices recognized in existing literature and standardized model parameter configuration, eliminating the impact of parameter selection on model performance evaluation. During distributed model training, a mini-batch training strategy was employed. The batch size was set to 10% of the training set size to ensure that the total number of batches processed by multiple training nodes in each experiment was similar. Furthermore, to prevent excessively large batch sizes from causing memory overflow, a maximum batch size was set for each node. For comparison with the method in this example (LBDH), the RH (RandomHash) method, the METIS method, and the SUGAR method were used. The RH (RandomHash) method achieves rapid graph partitioning by randomly assigning nodes to different subgraphs. The METIS method is the default method used by DGL2.4.0 and is also the most commonly used method in the industry. The SUGAR method is a resource-aware graph partitioning method designed for large-scale graph neural network training. Its core goal is to achieve efficient subgraph-level optimization in distributed training through dynamic subgraph expansion and explicit memory control. The final result is as follows Figure 3 and Figure 4 As shown, among which Figure 3 For the comparative experimental results on the Flickr dataset and the ogbn-arxiv dataset, Figure 4 The comparative experimental results on the Reddit dataset and the ogbn-products dataset are shown in Figure 2. The partitioning time is the arithmetic mean of the partitioning time of each graph partitioning method running five times on each dataset. Figure 3 and Figure 4As can be seen, on the larger Reddit and ogbn-products datasets, the partitioning time of the method (LBDH) of this embodiment is significantly lower than that of the METIS and SUGAR methods, with reductions of up to 42.2% and 34.6% respectively. However, on smaller datasets, such as the Flickr and ogbn-arxiv datasets, the partitioning time of the method (LBDH) of this embodiment is slightly higher than that of the METIS and SUGAR methods. This is because, although the classification method (LBDH) of this embodiment introduces preprocessing overhead (such as threshold calculation), the classification strategy reduces the actual amount of data processed by decomposing the high-dimensional problem into low-dimensional subproblems. Compared to the additional cost, the benefits are lower on small and medium-sized datasets and higher on large-scale power-law distribution graphs. The goal of distributed GNN training is to process large-scale graph data, so the method (LBDH) of this embodiment has advantages in practical applications. Regardless of the scale of the data set, the RH method directly uses random partitioning, which takes the shortest partitioning time. The reason is that the RH method ignores the structural characteristics of the graph and reduces a lot of computational overhead. However, this method of ignoring the structural characteristics of the graph is not conducive to subsequent classification accuracy, and it is difficult to achieve a balance between overhead and accuracy.

[0037] To quantify the degree of training time imbalance among heterogeneous nodes during distributed training of graph neural networks, we calculated the coefficient of variation (CV) for the four training nodes using the average of 10 epochs of training time on each node. The core concept behind this calculation is to normalize the standard deviation as a percentage of the mean, thereby eliminating the effects of dimension. The CV is a relative measure of data dispersion and is suitable for comparing the volatility of data with different means. The final results are shown in Tables 2 and 3.

[0038] Table 2: Experimental results of final training time and coefficient of variation on GCN

[0039] Table 3: Experimental results of final training time and coefficient of variation on GraphSAGE

[0040] The experimental results in Tables 2 and 3 provide the following conclusions: 1) When using the partitioning method of this embodiment (LBDH) on the GCN and GraphSAGE models, the final training time and coefficient of variation are both lower than those of the METIS method. Compared to the METIS method, the training time is reduced by up to 35.6%, and the coefficient of variation is reduced by up to 80.3%. Compared to the SUGAR method, the method of this embodiment (LBDH) has a lower coefficient of variation in all cases, and the final training time is higher only on the combination of the GCN model and the ogbn-arxiv dataset. This is due to the smaller dataset size, and the SUGAR method's optimization of the edge pruning rate more easily offsets its disadvantage of uneven load. However, distributed GNN training primarily processes large-scale graph data in heterogeneous environments. In practical settings, the comprehensive optimization of the method of this embodiment (LBDH) outperforms both the METIS and SUGAR methods. The reduction in training time and coefficient of variation of the method (LBDH) in this embodiment is primarily due to its use of node performance evaluation as a heuristic. It assigns each training node a load appropriate to its training performance based on the number of edges within the subgraph, achieving load balancing across nodes, allowing training tasks to complete almost simultaneously and reducing inter-node latency. The METIS and SUGAR methods, on the other hand, only evenly partition the nodes within the subgraph, failing to consider that the number of edges is the key factor affecting the computational load of each subgraph in GNNs, resulting in a primary imbalance. The METIS method also ignores the performance differences of training nodes in heterogeneous environments, causing a secondary imbalance. The SUGAR method only uses GPU memory size as a basis for graph partitioning optimization, failing to consider the complex environment of GNN training. Therefore, its performance is only slightly better than the METIS method. Second, examining the impact of both model and average degree on the training time of the method (LBDH) in this embodiment, we found that the greater the number of neighbor sampling in the model (compared to GraphSAGE's partial neighbor sampling, the GCN model's full neighbor sampling aggregates more nodes) and the larger the average degree, the more significant the improvement in training time. The reason is that the larger the neighbor sampling and the average degree of the vertex of the model, the more each high-degree vertex is formed when aggregating information in GNN. l-hopThe larger the neighbor subgraph, the greater the load disparity with low-degree vertices. However, the RH, METIS, and SUGAR methods assume that each vertex carries the same load, resulting in less effective load balancing strategies. 3) The RH method takes the longest to train in all cases. Compared to the METIS method, training time increases by up to 163.9%. This is because the random partitioning employed by the RH method results in weak node adjacency within each subgraph, disrupting the topological structure of the original graph data. Therefore, the aggregation process requires almost the entire original graph data, resulting in extensive cross-node communication and a significant reduction in training speed. Furthermore, since each training node requires access to the same original graph data, the RH method improves the coefficient of variation in training time. However, this optimization is less significant in practical training compared to the high GPU memory usage and long training time. 4) During experiments, it was found that training failed to complete when using the GCN model with the Reddit dataset. This is because the average degree of the Reddit dataset is relatively high, and the GCN model samples all neighbors, resulting in a large number of nodes sampled in each training batch. Furthermore, the large feature dimensions of the nodes result in significant GPU memory overhead for training each batch. For example, for training node 1 with 6GB of GPU memory, its maximum batch size is only approximately 30, while its training set size exceeds 30,000. A single training epoch takes over three hours, and GPU memory can easily run out. Despite this, the performance of this embodiment's method (LBDH) on other datasets is also representative and convincing.

[0041] In summary, the graph partitioning method based on degree classification for heterogeneous environment graph neural network in this embodiment mainly completes the following work in order to solve the problem of low resource utilization and low training efficiency caused by uneven load in distributed graph neural network training: (1) Considering the power-law distribution characteristics of graph data, a dynamic threshold selection strategy is designed to adaptively classify high-degree and low-degree vertices, and the significance of power-law distribution and accurate adaptation of classification are achieved through three-level thresholds. (2) A hierarchical partitioning mechanism is proposed, which first uses weight-constrained METIS partitioning for low-degree vertices to reduce the edge cutting rate while building a balanced subgraph structure. (3) For the partitioning of high-degree vertices, dynamic weight adjustment is introduced. By sensing the current status of each partition, the weights of the locality score and the load balancing score are dynamically adjusted to optimize the balance between locality and the current partition load. By mixed weighting of the locality score and the load balancing score, the comprehensive score of the high-degree vertices divided into different partitions is obtained, thereby determining the optimal partitioning of the high-degree vertices. Experiments show that the method in this embodiment (LBDH) outperforms traditional methods in heterogeneous environments: compared with the METIS and SUGAR methods, its graph partitioning time on large-scale datasets is reduced by up to 42.2% and 34.6%. During the distributed training phase, the method in this embodiment (LBDH) reduces the final training time by up to 35.6%, and the coefficient of variation by up to 80.3%, achieving dual optimization of total training time and load balancing.

[0042] In addition, the present embodiment also provides a graph partitioning system based on degree classification for heterogeneous environment graph neural networks, including interconnected microprocessors and memories, and the microprocessors are programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural networks. In addition, the present embodiment also provides a graph partitioning system based on degree classification for heterogeneous environment graph neural networks, including a vertex classification program unit, a low-degree vertex partitioning program unit, and a high-degree vertex partitioning program unit, wherein the vertex classification program unit is used to confirm whether the degree distribution of vertices in the original graph conforms to the power-law distribution through power-law distribution fitting evaluation, and then decide whether to perform high-degree and low-degree vertex classification. The threshold for vertex classification is determined by dynamic threshold selection, breaking through the traditional static threshold limitation, through Dynamically adjust the classification granularity. By classifying high-degree and low-degree vertices in the graph, the foundation for differentiated partitioning is laid. The low-degree vertex partitioning program unit first calculates the target load ratio based on the node's computing power. Each low-degree vertex is then assigned a unit weight and the improved METIS method is used to control the sum of the subgraph vertex weights to meet the target ratio, achieving preliminary load balancing. Finally, the actual number of edges in each partition is calculated as a baseline load, providing data support for subsequent high-degree vertex partitioning. This program unit utilizes the small degree variance of low-degree vertices to circumvent the shortcomings of traditional METIS in power-law graphs while significantly reducing the edge pruning rate. The high-degree vertex partitioning program unit introduces a dynamic weight adjustment mechanism. By sensing the current state of each partition, the weights of the locality score and the load balancing score are dynamically adjusted to optimize the balance between locality and the current partition's load. By combining the locality score and the load balancing score, a comprehensive score for the partitioning of high-degree vertices into different partitions is obtained, which is used to determine the optimal partitioning of high-degree vertices. In addition, this embodiment also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural networks through a processor. In addition, this embodiment also provides a computer program product, including a computer program or instruction, which is programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural networks through a processor.

[0043] Those skilled in the art should understand that the technical solution provided by the present invention may be in the form of a method, a system, or a computer program product. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions described in the process. Figure 1 a process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0044] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A graph partitioning method based on degree classification for heterogeneous environment graph neural network, characterized by: The method includes the following steps: determining the classification threshold of high-degree vertices and low-degree vertices according to the power-law characteristics of the vertex degree distribution in the input graph neural network, and classifying the vertices in the graph neural network into low-degree vertices and high-degree vertices according to the classification threshold; performing initial subgraph partitioning on the low-degree vertices using the constrained METIS method, and each partition obtained by the partition corresponds to a computing node in the heterogeneous environment; performing initial subgraph partitioning on the high-degree vertices, calculating the locality score and load balancing score for the high-degree vertices divided into different partitions, weighting the locality score and the load balancing score to obtain a comprehensive score for the partition, and selecting the partition with the highest comprehensive score as the partition result, and each partition corresponds to a computing node in the heterogeneous environment.

2. The graph partitioning method based on degree classification for heterogeneous environment graph neural network according to claim 1 is characterized in that: The method of determining the classification thresholds of high-degree vertices and low-degree vertices based on the power-law characteristics of the vertex degree distribution in the input graph neural network includes: using the vertex degree distribution in the input graph neural network to fit the probability density function of the power-law distribution, judging whether it conforms to the power-law distribution based on the fitting result of the probability density function of the power-law distribution, and if it conforms to the power-law distribution, calculating the classification thresholds of high-degree vertices and low-degree vertices based on the significance of the power-law distribution; if it does not conform to the power-law distribution, setting the classification thresholds of high-degree vertices and low-degree vertices to 100%, so that all vertices in the graph neural network are classified as low-degree vertices.

3. The graph partitioning method based on degree classification for heterogeneous environment graph neural network according to claim 2 is characterized in that: The functional expression of the probability density function of the power law distribution is: ,in Indicates degree The probability density of the vertex degree distribution; is a proportional constant used to adjust the scaling of the function; is an exponential parameter used to indicate the significance of the power law distribution of the function; when judging whether the fitting result of the probability density function of the power law distribution conforms to the power law distribution, the exponential parameter obtained by the fitting is included. and the proportional constant Calculate the goodness of fit R of the probability density function of the power law distribution 2 , if the goodness of fit R 2 If the value is greater than a preset threshold, it is determined that the fitting result of the probability density function of the power-law distribution conforms to the power-law distribution; otherwise, it is determined that the fitting result of the probability density function of the power-law distribution does not conform to the power-law distribution.

4. The graph partitioning method based on degree classification for heterogeneous environment graph neural network according to claim 3 is characterized in that: The classification threshold for calculating high-degree vertices and low-degree vertices according to the power law distribution significance refers to the exponential parameter used to represent the significance of the power law distribution function. Substitute the preset piecewise function to obtain the classification thresholds for high-degree vertices and low-degree vertices.

5. The graph partitioning method based on degree classification for heterogeneous environment graph neural network according to claim 4 is characterized in that: The function expression of the preset piecewise function is: ， in, is the quantile corresponding to the classification threshold of high-degree vertices and low-degree vertices, and is a constant parameter.

6. The graph partitioning method based on degree classification for heterogeneous environment graph neural network according to claim 1 is characterized in that: The initial subgraph partitioning of low-degree vertices using the constrained METIS method includes: obtaining the performance score of each computing node in the heterogeneous environment, dividing the performance score of each computing node by the total score to obtain the ratio of the ideal size of the partition corresponding to each computing node, calling the METIS method to inject the ratio of the ideal size of the partition corresponding to each computing node as the weight ratio into the constraint, so that each low-degree vertex is regarded as having the same weight for partitioning, each partition corresponds to a computing node in the heterogeneous environment, and the ratio of the number of vertices in each partition is close to the ratio of the ideal size of the partition.

7. The graph partitioning method based on degree classification for heterogeneous environment graph neural network according to claim 1 is characterized in that: The locality score is the ratio of all neighbor vertices of a high-degree vertex within one hop of the partition to be divided to all neighbor vertices of the high-degree vertex; the load balancing score is calculated by first calculating the load ratio between the load of the partition to be divided and the target load, and then subtracting the load ratio from 1 to obtain the result; the function expression for the weighted sum of the locality score and the load balancing score to obtain the comprehensive score of the partition to be divided is: ， in, is the height vertex Divide into partitions The comprehensive rating of is the load balancing weight, Score the load balancing score. is the locality weight, is the locality score, where the load balancing weight Score the load balancing The hyperbolic tangent function is used to map the deviation to the interval [0,1], and the load balancing weight and locality weight The sum of the two is 1.

8. A graph partitioning system based on degree classification for heterogeneous environment graph neural network, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural network as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural network described in any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the graph partitioning method based on degree classification for heterogeneous environment graph neural network described in any one of claims 1 to 7 through a processor.

Images