A distributed inference communication method and system of a super-large-scale graph neural network
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
- Filing Date
- 2026-04-20
- Publication Date
- 2026-07-24
Smart Images
Figure CN122450656A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of high-performance computing technology, and in particular to a distributed inference communication method and system for ultra-large-scale graph neural networks. Background Technology
[0002] In scientific computing fields such as aerospace and nuclear engineering, high-fidelity numerical simulations heavily rely on ultra-large-scale (billions to tens of billions of elements) unstructured meshes. Mesh smoothing, as a core mesh quality optimization technique, minimizes element distortion by adjusting node coordinates. In recent years, deep learning methods based on graph neural networks (GNNs) have been introduced into mesh optimization to replace computationally expensive traditional iterative solvers. However, applying GNNs to distributed multi-GPU inference on ultra-large-scale three-dimensional volumetric meshes presents technical challenges such as fragmented cross-device communication, low GPU computing resource utilization, and communication latency blocking computation. Summary of the Invention
[0003] To address the problems existing in the prior art, embodiments of this application provide a method, apparatus, computing device, computer storage medium, and product containing a computer program for distributed inference communication of ultra-large-scale graph neural networks, which can eliminate the GPU idle problem caused by communication fragmentation and improve the throughput of distributed inference.
[0004] In a first aspect, embodiments of this application provide a distributed inference communication method for ultra-large-scale graph neural networks, comprising: dividing the global topology to be processed into multiple local subdomains matching the number of GPUs, and determining the local master node allocated to each GPU and the boundary master nodes involved in GPU communication; calculating the coloring priority of each node, performing parallel graph coloring based on the coloring priority to construct a global independent set sequence, wherein the boundary master nodes are clustered in the earliest independent set; mapping the global independent set sequence to the local subdomains of each GPU to generate a local task slice sequence, the local task slice sequence containing boundary node slices and internal node slices, evaluating the communication load and computation load of the slices, using the internal node slices as computational fillers, and interleaving them with the communication waiting period of the boundary node slices; in the distributed inference stage, each GPU executes the inference task according to the local task slice sequence, using an internally constructed multi-stream asynchronous pipeline, so that the cross-device synchronous communication of the boundary node slices and the model computation depth of the internal node slices overlap.
[0005] In some possible implementations, the coloring priority of each node is calculated by setting different priority calculation strategies for boundary master nodes and internal master nodes; wherein the coloring priority of the boundary master node is higher than that of the internal master node, and the coloring priority of nodes within the same category is positively correlated with the degree of the node.
[0006] In some possible implementations, the formula for calculating color priority is as follows:
[0007]
[0008] In the formula, The coloring priority of node v is represented. The communication priority weight coefficient is used to ensure that the priority of the boundary master node is strictly higher than that of the internal master nodes. Characterized by the cost of cross-GPU communication, i.e., the number of mirror nodes of node v on other GPUs. The degree of node v is represented by its characteristic. Represents the boundary master node, Characterizes the random perturbation values used to break symmetry. Represents the internal master node.
[0009] In some possible implementations, parallel graph coloring is performed based on the coloring priority to construct a global sequence of independent sets, including: adopting a maximum degree-first heuristic strategy, where nodes of the same type are processed first, nodes with higher degrees are processed first, in order to minimize the total number of generated independent sets.
[0010] In some possible implementations, the communication and computational loads of a slice are evaluated, and internal node slices are used as computational fillers, interleaving their execution within the communication latency of boundary node slices. This includes: evaluating the communication latency of a boundary node slice based on the number of mirror nodes required for that slice; evaluating the computational execution time of an internal node slice based on the number of nodes it contains; and continuously interleaving one or more internal node slices after a boundary node slice, such that the total computational execution time of the interleaved internal node slices matches the communication latency of the boundary node slice.
[0011] In some possible implementations, the multi-stream asynchronous pipeline includes three asynchronously executed streams maintained within each GPU: a first stream for processing topology tasks on the CPU side, including master node acquisition, neighbor node retrieval, and subgraph construction; a second stream for processing data transmission and synchronization tasks on the GPU side, including node feature acquisition and assembly, and asynchronous synchronization of boundary mirror nodes across devices; and a third stream for processing core model inference computation tasks on the GPU side; wherein the execution of the third stream is decoupled from the first and second streams.
[0012] In some possible implementations, the second stream employs different data retrieval strategies based on the currently executed slice type when acquiring and assembling node features: when executing boundary node slices, neighbor node data is jointly assembled from GPU local memory and local cache, where the local cache is used to store mirror node data received asynchronously over the network; when executing internal node slices, all neighbor node data is directly retrieved from GPU local memory.
[0013] In some possible implementations, the multi-stream asynchronous pipeline is implemented using a producer-consumer model: the first and second streams act as producers, placing data-ready task slices into a queue; the third stream acts as a consumer, retrieving task slices from the queue and executing them; when the queue contains internal node slices, the third stream preferentially retrieves and executes the internal node slices to fill the idle time slots of the second stream while waiting for cross-device communication.
[0014] Secondly, embodiments of this application provide a distributed inference communication system for ultra-large-scale graph neural networks, comprising: a subdomain partitioning module, used to partition the global topology to be processed into multiple local subdomains matching the number of GPUs, and determine the local master node allocated to each GPU and the boundary master node involved in GPU communication; an independent set construction module, used to calculate the coloring priority of each node, and perform parallel graph coloring based on the coloring priority to construct a global independent set sequence, wherein the boundary master nodes are clustered in the earliest independent set; an interleaved scheduling module, used to map the global independent set sequence to the local subdomains of each GPU, generate a local task slice sequence, the local task slice sequence containing boundary node slices and internal node slices, evaluate the communication load and computation load of the slices, use the internal node slices as computation fillers, and interleave them during the communication waiting period of the boundary node slices; and an asynchronous inference module, used in the distributed inference stage, where each GPU executes inference tasks according to the local task slice sequence using an internally constructed multi-stream asynchronous pipeline, so that the cross-device synchronous communication of the boundary node slices and the model computation depth of the internal node slices overlap.
[0015] Thirdly, embodiments of this application provide a computer-readable storage medium including computer-readable instructions that, when read and executed by a computer, cause the computer to perform the method as described in any of the first aspects.
[0016] Fourthly, embodiments of this application provide a computing device, including a processor and a memory, wherein the memory stores computer program instructions, which, when executed by the processor, perform the method as described in any of the first aspects.
[0017] Fifthly, embodiments of this application provide a product comprising a computer program that, when the computer program product is run on a processor, causes the processor to perform the method as described in any of the first aspects. Attached Figure Description
[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating a distributed inference communication method for a large-scale graph neural network provided in an embodiment of this application.
[0020] Figure 2 This is a schematic diagram illustrating the principle of boundary-first independent set construction and interleaved task scheduling provided in an embodiment of this application;
[0021] Figure 3 This is a schematic diagram of timing overlap for multi-stream asynchronous pipelined inference provided in an embodiment of this application;
[0022] Figure 4 This is a schematic diagram of the structure of a distributed inference communication system for a large-scale graph neural network provided in an embodiment of this application. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.
[0025] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.
[0026] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0027] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.
[0028] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.
[0029] When applying Generative Neural Networks (GNNs) to distributed multi-GPU inference on ultra-large-scale 3D volumetric meshes, mesh smoothing, which is essentially based on message passing between spatially connected nodes, strictly requires in-place updates. Specifically, the computation of a vertex depends on the latest state of its neighboring nodes. This strong dependency causes the synchronous parallel update mechanism supported by standard GNNs to fail, leading to data races. To resolve data races, some known techniques typically employ algorithms based on Independent Sets (IS) to divide nodes into multiple ordered sets. Nodes within the same IS set have no connecting edges and can be updated in parallel. However, existing distributed IS algorithms construct sets solely based on graph connectivity, completely ignoring the physical distribution of nodes and communication costs. This directly results in severe fragmentation of cross-GPU communication: the mirror nodes required by the local master node are often scattered across multiple remote GPUs, generating high-frequency, fine-grained data transfer requests. This communication bottleneck forces GPU computing units into long idle periods while waiting for remote data, significantly slowing down overall computational efficiency and limiting the scalability of ultra-large-scale graph neural networks on multi-GPU clusters.
[0030] In view of this, embodiments of this application provide a distributed inference communication method for ultra-large-scale graph neural networks, and in particular an optimization method for inference communication of grid processing algorithms involving spatial topology message passing and in-situ updates in ultra-large-scale graph neural networks.
[0031] For example, Figure 1 This illustration shows a flowchart of a distributed inference communication method for a large-scale graph neural network provided in an embodiment of this application. Figure 1 As shown, the method may include the following steps:
[0032] S11: Divide the global topology to be processed into multiple local subdomains that match the number of GPUs, and determine the local master node assigned to each GPU and the boundary master node involved in GPU communication.
[0033] In this embodiment, the computational task of the graph neural network is spatially partitioned to establish the computational and communication boundaries in a distributed environment. Please refer to... Figure 2 First, the global topology to be processed is obtained, such as the topology graph in CSR format corresponding to a 3D mesh model containing tens of millions of nodes. Using a graph partitioning algorithm, the obtained global topology is divided into N local subdomains, where N matches the number of GPUs in the cluster. After partitioning, each GPU is assigned a local subdomain for independent processing. At this point, the system identifies and classifies the nodes within the subdomain, clearly dividing them into two categories: one is inner nodes, where all their immediate neighbors reside in the local memory of the same GPU, and their feature aggregation process relies entirely on local data, requiring no cross-GPU network communication; the other is boundary nodes, where their immediate neighbors include remote nodes located on other GPUs (i.e., mirror nodes), and these nodes must pull remote data from other GPUs through network interfaces (such as NCCL or MPI) when performing graph convolution or message passing.
[0034] S12: Calculate the coloring priority of each node, and perform parallel graph coloring based on the coloring priority to construct a global independent set sequence, wherein the boundary master nodes are clustered in the earliest independent set.
[0035] In this embodiment, during the inference process of a distributed graph neural network, multiple graphics processors (GPUs) need to process subgraph data allocated to different GPUs in parallel. To support in-situ updates of node features and avoid conflicts caused by multiple GPUs simultaneously writing to the same node, the nodes in the graph need to be divided into several independent sets. Nodes within each independent set are not adjacent to each other, thus allowing for safe parallel computation. Traditional graph coloring algorithms determine the independent set a node belongs to based solely on its connectivity, without distinguishing whether a node needs cross-GPU communication. This results in boundary master nodes that need to obtain neighbor node data from other GPUs being randomly distributed across a large number of independent sets. This distribution can trigger multiple fragmented cross-device network transmissions each time a new independent set is switched, leading to severe communication fragmentation. This forces GPUs to frequently wait for sporadic data arrivals, hindering efficient utilization of computing resources. Therefore, this embodiment of the application forcibly intervenes in the coloring order when constructing independent sets, concentrating high-communication-overhead boundary master nodes in one or more of the earliest generated independent sets. By clustering boundary node slices in the earliest independent sets, most cross-device data synchronization can be completed centrally during the initial communication phase of inference for each GPU, reducing the frequency of communication interruptions in subsequent independent sets. Specifically, a coloring priority is first calculated for each node in the graph. The design principle of this priority is to ensure that the priority of any boundary master node is strictly higher than that of internal master nodes, while also reflecting the computational or communication importance within the same node class. Specifically, for a node belonging to the boundary master node set, its priority is equal to a preset communication priority weight coefficient multiplied by the node's cross-GPU communication cost, plus the node's degree. The cross-GPU communication cost is defined as the number of mirror nodes of this node on other GPUs, and the communication priority weight coefficient is set to a maximum value, for example, greater than the maximum possible degree in the graph, thus ensuring that the priority of boundary master nodes is necessarily higher than that of internal master nodes. For a node belonging to the internal master node set, its priority is equal to the node's degree plus a small random perturbation value used to break symmetry. This random perturbation value is only used to distinguish internal nodes with the same degree and does not affect the dominant role of degree ranking.
[0036] After obtaining the priorities of all nodes, they are colored sequentially in descending order of priority. When processing each node, the color used by its already colored neighbors is prohibited, and the first available color is attempted to be assigned, starting with the smallest color number. Since boundary master nodes have extremely high priority, they are processed first and assigned to the color classes with the smallest color numbers. Each color class corresponds to an independent set; therefore, the independent set with the smallest color number contains the most boundary master nodes. The number of boundary master nodes gradually decreases in subsequent color classes, while internal master nodes, due to their lower priority, mainly appear in later independent sets. This process enforces the aggregation of boundary master nodes to the earliest independent set.
[0037] In some possible implementations, a maximum degree-first heuristic strategy can also be used during the coloring process to control the total number of independent sets. Among nodes of the same type (e.g., all boundary master nodes or all internal master nodes), when multiple nodes have the same communication cost, nodes with higher degrees are processed first, because nodes with higher degrees are more likely to be constrained by their assigned colors; processing them early helps reduce subsequent conflicts, thereby reducing the total number of independent sets and lowering overall synchronization overhead. When multiple nodes have the same communication cost, degree is used as a deterministic winning condition.
[0038] In some possible embodiments, the specific formula for the node coloring priority is as follows:
[0039]
[0040] In the formula, The coloring priority of node v is represented. The communication priority weight coefficient is used to ensure that the priority of the boundary master node is strictly higher than that of the internal master nodes. Characterized by the cost of cross-GPU communication, i.e., the number of mirror nodes of node v on other GPUs. The degree of node v is represented by its characteristic. Represents the boundary master node, Characterizes the random perturbation values used to break symmetry. Represents the internal master node.
[0041] S13: Map the global independent set sequence to the local subdomain of each GPU to generate a local task slice sequence. The local task slice sequence contains boundary node slices and internal node slices. Evaluate the communication load and computation load of the slices, use the internal node slices as computation fillers, and interleave them during the communication waiting period of the boundary node slices.
[0042] In this embodiment, after obtaining the global independent set sequence through a boundary-first parallel graph coloring process, each graphics processor (GPU) needs to map this sequence onto its own local subdomain to generate executable local tasks. Since the global independent set sequence is a set of nodes abstracted from the topology of the entire graph, the nodes in each independent set may be distributed across different GPUs. Therefore, each GPU needs to traverse every independent set in the global independent set sequence, selecting the master node belonging to its own local subdomain, and arranging these nodes according to the order of the independent sets to form a local task slice sequence. Specifically, when a node in an independent set is the local master node of that GPU, that node is encapsulated as a task slice, retaining the original order of its belonging independent set. Thus, each GPU ultimately obtains a local task slice sequence corresponding to the global independent set sequence, where each slice represents a node computation task that needs to be executed on that GPU. These local task slices are divided into two categories based on the node type. The first category is boundary node slices, corresponding to tasks that originally belonged to boundary master nodes. Because at least one of the ring neighbors of a boundary master node is a mirror node located on another graphics processor, the current graphics processor must obtain the latest feature data or coordinate data of these mirror nodes from the other graphics processor via the network before executing this type of slice. Therefore, the execution overhead of boundary node slices consists of two parts: cross-device communication latency and local model computation time. The second type is internal node slices, corresponding to tasks that originally belonged to the internal master node. Since all the ring neighbors of an internal master node are located on the same graphics processor, no cross-graphics processor communication is required when executing this type of slice. All the necessary data can be read directly from the local video memory. Therefore, the execution overhead of internal node slices only includes local model computation time.
[0043] To eliminate the situation where the processor is idle and cannot be effectively utilized while waiting for network transmission in traditional methods, this application adopts a workload-aware interleaved task scheduling method. First, load assessment is performed on different types of task slices. For boundary node slices, the communication latency required to complete cross-device data synchronization is estimated based on the number of mirror nodes required and the current network bandwidth. For internal node slices, the computational execution time required to execute the slice is estimated based on the number of nodes it contains and the computational complexity of the current graph neural network model. After load assessment, the scheduler dynamically adjusts the execution order of task slices. Internal node slices that were originally concentrated at the back of the sequence are retrieved in advance and interleaved into the communication waiting period of boundary node slices as computational fillers. Specifically, when a boundary node slice is encountered, the scheduler does not immediately block and wait for its data return, but first initiates an asynchronous network communication request for that slice, submitting the operation of obtaining remote mirror node data to the underlying communication library. Subsequently, the scheduler selects one or more unexecuted internal node slices from the task slice sequence and executes them continuously on the current graphics processor. The total computational execution time of these internal node slices is arranged to approximately match the communication latency of the aforementioned boundary node slices. In this way, while the graphics processor (GPU) performs intensive matrix calculations on internal node slices, network communication operations are carried out asynchronously in the background, with data being transferred from other GPUs to the local cache. When all the interleaved internal node slices have completed their calculations, the remote data from the boundary node slices has typically arrived locally. At this point, the scheduler switches back to that boundary node slice to execute its local model calculations. In this manner, the network communication latency that would otherwise cause the GPU to idle is completely filled into the effective computation time of the internal node slices. If the communication latency of a boundary node slice is long, the scheduler can interleave more internal node slices; conversely, if the communication latency is short, fewer internal node slices are interleaved, thus achieving adaptive adjustment to different loads.
[0044] S14: During the distributed inference phase, each GPU executes inference tasks according to the local task slice sequence and utilizes the internally built multi-stream asynchronous pipeline. This allows the cross-device synchronous communication of boundary node slices to overlap with the model computation depth of internal node slices, thus hiding communication latency.
[0045] In this embodiment, after obtaining a sequence of local task slices containing alternating boundary node slices and internal node slices through staggered scheduling, the distributed inference phase needs to be executed on each GPU. On each GPU, a multi-stream asynchronous pipeline is constructed to decouple the originally serial data preparation, network communication, and model computation stages, allowing them to proceed independently in parallel.
[0046] For details, please refer to Figure 3The first stream runs on the CPU side, handling lightweight graph topology processing tasks. When a task slice needs to be executed, the first stream retrieves the identifiers of the node's ring neighbors from memory beforehand and constructs the local computational subgraph required for the current computation based on the neighbor relationships. Since these topology operations do not involve large-scale numerical computations and do not rely on cross-device data, executing them on the CPU frees up the GPU's computing resources. The second stream runs on the GPU side, handling data transmission and cross-device synchronization tasks. This stream manages the acquisition and assembly of node features, the updating of coordinate data, and asynchronous synchronization operations of boundary mirror nodes across GPUs. All communication operations in the second stream are initiated asynchronously; that is, after issuing a data request, it does not block and wait for a response but returns immediately, allowing other streams to continue working. The third stream also runs on the GPU side, handling core graph neural network model inference computation tasks, including intensive matrix multiplication, activation function calculation, and feature aggregation operations. The third stream is computationally intensive and needs to be kept running continuously as much as possible to avoid idle computing units.
[0047] The three streams communicate and synchronize via a producer-consumer queue. The first stream places prepared computational subgraph metadata into the queue for consumption by the second and third streams. The second stream executes a differentiated data retrieval strategy based on the type of the current slice. When a boundary node slice is about to be executed, the second stream first reads existing local node features from the graphics processor's local memory and simultaneously initiates a network request to retrieve the required remote mirror node feature data from other graphics processors. After these remote data arrive via network transmission, they are stored in a specially designed local cache. The second stream then combines the local data with the remote data to form a complete input feature tensor. When an internal node slice is about to be executed, since all its ring-near nodes are located within the same graphics processor, the second stream does not need to initiate any network requests and can directly read all the required data from the graphics processor's local memory. The execution of the third stream is decoupled from the first and second streams, ensuring that the core inference computation process of the third stream is not blocked by data preparation or boundary synchronization, thus completely hiding the cross-device network communication latency within the computation time of the local internal node slice.
[0048] The above describes the distributed inference communication method for ultra-large-scale graph neural networks provided in this application. It breaks the limitation of traditional graph coloring relying solely on connectivity by introducing a communication priority weight coefficient, forcing the boundary master node to trigger at the very front of inference. Secondly, it uses internal node slices that do not rely on remote data as computational fillers, precisely interspersed within the network communication waiting period of the boundary node slices. A three-stream asynchronous pipeline, completely decoupled from topology processing, data synchronization, and core inference, is constructed within a single GPU, achieving deep collaboration between the scheduling and execution layers. This invention not only reconstructs the task execution order according to communication overhead at the algorithm layer but also replaces the traditional synchronous blocking wait with asynchronous interleaved execution at the system layer, transforming the originally serial "communication-computation" process into a deep "communication and computation overlap." This invention overcomes the scalability bottleneck of ultra-large-scale graph neural networks on multi-GPU clusters. Through interleaved scheduling and multi-stream asynchronous pipelines, it can perfectly hide most of the cross-GPU network communication latency within the local computation time, completely eliminating the GPU idle problem caused by communication fragmentation. This method not only increases the throughput of distributed inference by several times, but also enables efficient end-to-end smooth inference of 3D grids with a scale of tens of billions based on graph neural networks.
[0049] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Furthermore, in some possible implementations, each step in the above embodiments may be selectively executed according to actual circumstances; it may be partially or fully executed, without limitation here. All or part of any feature of any embodiment of this application can be freely and arbitrarily combined without contradiction. The combined technical solutions are also within the scope of this application.
[0050] Based on the methods in the above embodiments, this application also provides a distributed inference communication system for ultra-large-scale graph neural networks. Figure 4 A schematic diagram of the structure of a distributed inference communication system for a very large-scale graph neural network is shown. Figure 4 As shown, the system includes a subdomain partitioning module 401, an independent set construction module 402, an interleaved scheduling module 403, and an asynchronous inference module 404.
[0051] The subdomain partitioning module 401 is used to divide the global topology to be processed into multiple local subdomains that match the number of GPUs, and to determine the local master node allocated to each GPU and the boundary master node involved in GPU communication.
[0052] Independent set construction module 402 is used to calculate the coloring priority of each node and perform parallel graph coloring based on the coloring priority to construct a global independent set sequence, wherein the boundary master nodes are clustered in the earliest independent set;
[0053] The interleaved scheduling module 403 is used to map the global independent set sequence to the local subdomain of each GPU, generate a local task slice sequence, which includes boundary node slices and internal node slices, evaluate the communication load and computation load of the slices, use the internal node slices as computation fillers, and interleave the execution during the communication waiting period of the boundary node slices.
[0054] The asynchronous inference module 404 is used in the distributed inference phase, where each GPU executes inference tasks according to the local task slice sequence using an internally built multi-stream asynchronous pipeline, so that the cross-device synchronous communication of boundary node slices overlaps with the model computation depth of internal node slices.
[0055] It should be understood that the above system is used to execute the methods in the above embodiments. The corresponding program modules in the system are similar in implementation principle and technical effect to those described in the above methods. The working process of the system can be referred to the corresponding process in the above methods, and will not be repeated here.
[0056] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.
[0057] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.
[0058] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.
[0059] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.
[0060] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0061] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.
Claims
1. A distributed inference communication method for ultra-large-scale graph neural networks, characterized in that, The method includes: The global topology to be processed is divided into multiple local subdomains that match the number of GPUs, and the local master node allocated to each GPU and the boundary master node involved in GPU communication are determined. Calculate the coloring priority of each node, and perform parallel graph coloring based on the coloring priority to construct a global independent set sequence, wherein the boundary master nodes are clustered in the earliest independent set; The global independent set sequence is mapped to the local subdomain of each GPU to generate a local task slice sequence. The local task slice sequence contains boundary node slices and internal node slices. The communication load and computation load of the slices are evaluated. The internal node slices are used as computation fillers and are interspersed during the communication waiting period of the boundary node slices. During the distributed inference phase, each GPU executes inference tasks according to its local task slice sequence, utilizing an internally built multi-stream asynchronous pipeline, so that the cross-device synchronous communication of boundary node slices overlaps with the model computation depth of internal node slices.
2. The method according to claim 1, characterized in that, The calculation of the coloring priority of each node includes: Set different priority calculation strategies for boundary master nodes and internal master nodes; The coloring priority of boundary master nodes is higher than that of internal master nodes, and the coloring priority of nodes within the same category is positively correlated with the degree of the node.
3. The method according to claim 2, characterized in that, The formula for calculating shading priority is as follows: In the formula, The coloring priority of node v is represented. The communication priority weight coefficient is used to ensure that the priority of the boundary master node is strictly higher than that of the internal master nodes. Characterized by the cost of cross-GPU communication, i.e., the number of mirror nodes of node v on other GPUs. The degree of node v is represented by its characteristic. Represents the boundary master node, Characterizes the random perturbation values used to break symmetry. Represents the internal master node.
4. The method according to claim 1, characterized in that, The parallel graph coloring based on the coloring priority to construct a globally independent set sequence includes: A maximum degree-first heuristic strategy is adopted, which prioritizes processing nodes with higher degrees when processing nodes of the same type, in order to minimize the total number of independent sets generated.
5. The method according to claim 1, characterized in that, The evaluation slice's communication and computational load, using internal node slices as computational fillers, is interspersed and scheduled to execute during the communication waiting periods of boundary node slices, including: The communication latency of the boundary node slice is evaluated based on the number of mirror nodes required for the boundary node slice. The computational execution time of the internal node slice is evaluated based on the number of nodes contained in the internal node slice. After a boundary node slice, one or more internal node slices are executed consecutively, such that the total computation execution time of the interleaved internal node slices matches the communication latency time of the boundary node slice.
6. The method according to claim 1, characterized in that, The multi-stream asynchronous pipeline comprises three asynchronously executed streams maintained within each GPU: The first stream is used to process topology tasks on the CPU side, including master node acquisition, neighbor node retrieval, and subgraph construction. The second stream is used to process data transmission and synchronization tasks on the GPU side, including node feature acquisition and assembly, and asynchronous synchronization of boundary mirror nodes across devices; The third stream is used to process core model inference computation tasks on the GPU side; The execution of the third flow is decoupled from the first flow and the second flow.
7. The method according to claim 6, characterized in that, When the second stream acquires and assembles node features, it employs different data retrieval strategies based on the currently executed slice type: When performing boundary node slicing, neighbor node data is jointly assembled from GPU local memory and local cache, wherein the local cache is used to store mirror node data received asynchronously over the network; When performing internal node slicing, all neighbor node data is obtained directly from the GPU's local video memory.
8. The method according to claim 6, characterized in that, The multi-stream asynchronous pipeline is implemented using a producer-consumer model: The first and second streams act as producers, placing data-ready task slices into a queue; The third stream acts as a consumer, retrieving task slices from the queue and executing them; When the queue contains internal node slices, the third stream preferentially retrieves and executes the internal node slices to fill the idle time slots of the second stream waiting for cross-device communication.
9. A distributed inference communication system for a large-scale graph neural network, characterized in that, The system includes: The subdomain partitioning module is used to divide the global topology to be processed into multiple local subdomains that match the number of GPUs, and to determine the local master node allocated to each GPU and the boundary master node involved in GPU communication. An independent set construction module is used to calculate the coloring priority of each node and perform parallel graph coloring based on the coloring priority to construct a global independent set sequence, wherein the boundary master nodes are clustered in the earliest independent set; The interleaved scheduling module is used to map the global independent set sequence to the local subdomain of each GPU, generate a local task slice sequence, which contains boundary node slices and internal node slices, evaluate the communication load and computation load of the slices, use the internal node slices as computation fillers, and interleave them during the communication waiting period of the boundary node slices. The asynchronous inference module is used in the distributed inference phase, where each GPU executes inference tasks according to the local task slice sequence using an internally built multi-stream asynchronous pipeline, so that the cross-device synchronous communication of boundary node slices and the model computation depth of internal node slices overlap.
10. A computer-readable storage medium comprising computer-readable instructions that, when read and executed by a computer, cause the computer to perform the method as described in any one of claims 1-8.