Communication method, device and equipment of graphics processor cluster and storage medium

Abstract

This application provides a communication method, apparatus, computer device, and storage medium for a graphics processing unit (GPU) cluster, relating to the field of computer application technology. The method includes: for each first target GPU in a first tensor parallel group, pre-establishing a second communication link between the first target GPU and each second target GPU in a second tensor parallel group; wherein the first tensor parallel group is any one of a plurality of tensor parallel groups, and the second tensor parallel group is a tensor parallel group different from the first tensor parallel group; controlling a first sub-GPU in the first tensor parallel group to divide internally stored data to be reduced into multiple sub-data to be reduced, and transmitting the multiple sub-data to be reduced to the second sub-GPUs in the first tensor parallel group respectively via the first communication link and the second communication link. This method can improve bandwidth utilization and communication efficiency.

Description

Technical Field

[0001] This application relates to the field of computer application technology, and more specifically, to a communication method, apparatus, computer device, and storage medium for a graphics processor cluster. Background Technology

[0002] Against the backdrop of the rapid development of artificial intelligence technology, and with the increasing prevalence of AI training and inference tasks, high-performance computing clusters have become the core infrastructure supporting large-scale deep learning training. In order to improve the processing efficiency of AI tasks, the Tensor Parallelism (TP) strategy groups multiple graphics processing units (GPUs) and splits a single huge model layer across these GPU groups for parallel computation.

[0003] However, each GPU within a TP group only communicates with each other through communication links directly connected to GPUs within the same TP group. This results in some direct connection ports being idle, leading to insufficient bandwidth utilization and thus affecting communication efficiency. Summary of the Invention

[0004] This application provides a communication method, apparatus, computer device, and storage medium for a graphics processor cluster.

[0005] A first aspect of this application provides a communication method for a graphics processing unit (GPU) cluster. The GPU cluster is divided into multiple tensor parallel groups, each containing multiple GPUs. The multiple GPUs within each tensor parallel group communicate with each other via a first communication link. The method includes: for each first target GPU in the first tensor parallel group, pre-constructing a second communication link between the first target GPU and each second target GPU in a second tensor parallel group; wherein the first tensor parallel group is any one of the multiple tensor parallel groups, and the second tensor parallel group is a tensor parallel group different from the first tensor parallel group; controlling a first sub-GPU in the first tensor parallel group to divide internally stored data to be reduced into multiple sub-data to be reduced, and transmitting the multiple sub-data to be reduced to the second sub-GPU in the first tensor parallel group via the first communication link and the second communication link, respectively.

[0006] A second aspect of this application provides a communication device for a graphics processing unit (GPU) cluster. The GPU cluster is divided into multiple tensor parallel groups, and each tensor parallel group contains multiple GPUs. The multiple GPUs within each tensor parallel group communicate with each other via a first communication link. The device includes: a communication link construction module, configured to pre-construct a second communication link between each first target GPU in the first tensor parallel group and each second target GPU in a second tensor parallel group; wherein the first tensor parallel group is any one of the multiple tensor parallel groups, and the second tensor parallel group is a tensor parallel group different from the first tensor parallel group; and a data transmission module, configured to control a first sub-GPU in the first tensor parallel group to divide internally stored data to be reduced into multiple sub-data to be reduced, and to transmit the multiple sub-data to be reduced to the second sub-GPU in the first tensor parallel group via the first communication link and the second communication link, respectively.

[0007] A third aspect of this application provides a computer device, including: a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any of the above-mentioned graphics processor cluster communication methods.

[0008] A fourth aspect of the embodiments of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the communication method of the graphics processor cluster as described in any of the preceding claims.

[0009] A fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the communication method of a graphics processor cluster as described in any of the preceding claims.

[0010] The technical solution of this application has the following beneficial effects: In the communication method of the above-mentioned graphics processor cluster, a second communication link is pre-constructed between each first target graphics processor in the first tensor parallel group and each second target graphics processor in the second tensor parallel group for each first target graphics processor in the first tensor parallel group; wherein, the first tensor parallel group is any tensor parallel group among a plurality of tensor parallel groups, and the second tensor parallel group is a tensor parallel group among a plurality of tensor parallel groups that is different from the first tensor parallel group; the first sub-graphics processor in the first tensor parallel group is controlled to divide the internally stored data to be reduced into a plurality of data to be reduced, and the plurality of data to be reduced are transmitted to the second sub-graphics processor in the first tensor parallel group respectively based on the first communication link and the second communication link.

[0011] This method can pre-build cross-group communication links to expand the communication links of each GPU, thereby making full use of the bandwidth resources corresponding to each GPU port and improving resource utilization. Then, during data transmission, the data to be reduced stored inside the graphics processor is divided into multiple sub-data to be reduced, and synchronous data transmission is performed based on multiple communication paths composed of the constructed multiple communication links, improving data transmission / communication efficiency and thus ensuring high efficiency in data processing. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram illustrating an application scenario of a communication method for a graphics processor cluster in one of the prior art solutions provided in this application embodiment; Figure 2 A schematic diagram illustrating an application scenario of another communication method for a graphics processor cluster provided in an embodiment of this application; Figure 3 A flowchart illustrating a communication method for a graphics processor cluster provided in an embodiment of this application; Figure 4 A schematic diagram of an 8-card graphics processor cluster topology provided in an embodiment of this application; Figure 5 A schematic diagram of the communication device structure of one type of graphics processor cluster provided in an embodiment of this application; Figure 6 This is a schematic diagram of one type of computer device structure provided in an embodiment of this application. Detailed Implementation

[0013] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0014] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0015] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0016] To facilitate understanding, we will first briefly introduce some relevant technical terms involved in the embodiments of this application.

[0017] 1) Tensor Parallelism (TP): is a fine-grained model parallelism technique in the training / inference of large distributed artificial intelligence models. Its core is to split the tensors and weight matrices inside a single layer and compute them in parallel on multiple graphics processing units (GPUs), solving the problem that ultra-large models cannot be stored on a single GPU.

[0018] 2) Graphics Processing Unit (GPU): Also known as a display core or display chip, it is a microprocessor specifically designed to handle graphics and image processing.

[0019] In the process of developing this application, the inventors discovered that with the rapid development of artificial intelligence technology, the application of large models for tasks such as AI training and inference is becoming increasingly widespread, which inevitably places higher demands on high-performance computing clusters. For example, in some AI training or inference application scenarios, a single huge model layer (such as the Attention layer or MLP layer in Transformer) is usually split across multiple GPUs for joint computation, and multiple GPUs within a TP group typically share the computational task of the same model layer.

[0020] Within a single TP (Transmission Terminal) group, multiple GPUs communicate collectively via direct connections. This means that GPUs within each TP group can only communicate through links directly connected to GPUs within the same TP group. For example, in a 4-GPU fully interconnected topology, see [link to relevant documentation]. Figure 1As shown, the topology contains four GPU units: GPU0, GPU1, GPU2, and GPU3. These four GPUs are divided into two groups, forming two TP2 groups (TP2(A) and TP2(B)) to perform parallel computing for artificial intelligence training or inference tasks.

[0021] Continue to refer to Figure 1 As shown, TP2(A) and TP2(B) belong to completely independent and non-interfering NCCL communication domains. Their respective group communication tasks are independently scheduled by the communication library. That is, for each GPU in TP2(A) and TP2(B), it can only communicate through the link directly connected to the GPU in its own TP group. For example, for GPU0 in TP2(A), it can communicate with GPU1 in TP2(A) through the direct link between them. For GPU2 in TP2(B), it can communicate with GPU3 in TP2(B) through the direct link between them. This allows each TP group to exclusively occupy the NVLink link resources corresponding to 2 cards, without occupying bandwidth across groups. Both groups can use the NVLink bandwidth within their respective groups simultaneously.

[0022] However, the above Figure 1 The multi-GPU interconnected topology shown can improve the efficiency of AI training or inference tasks by using TP groups, allowing each TP group to operate in its own independent NCCL communication domain. However, it also inevitably results in each TP group often only using a portion of the GPU for computation. Even if the GPU only uses a small number of ports to send and receive data, it leads to insufficient bandwidth utilization and consequently, low GPU resource utilization.

[0023] For example, continue to combine Figure 1 As shown, a GPU typically contains multiple ports (e.g., 8), and each port's corresponding communication path consumes a portion of bandwidth resources. Taking GPU0 in TP2(A) as an example, GPU0 only uses one port to directly connect with one of the ports of GPU1. The other ports are idle, and when the bandwidth of the direct NVLink link between GPU0 and GPU1 is occupied, other tasks need to wait for bandwidth resources to become available before they can continue to be transmitted from GPU0 to GPU1. This process results in low utilization of GPU bandwidth resources and insufficient data transmission paths between GPUs, which increases communication time and consequently affects data processing efficiency.

[0024] To address the aforementioned technical issues, this application provides a communication method for a graphics processing unit cluster. This method can expand the communication paths between GPUs in a TP group, thereby improving the utilization of GPU bandwidth resources and thus improving data processing efficiency.

[0025] The following Figure 1 Based on the 4-card fully interconnected topology shown, please refer to... Figure 2 The diagram illustrates an application scenario of another communication method for a graphics processing unit (GPU) cluster provided by an embodiment of this application to address the aforementioned technical problems. Figure 2 As shown, taking GPU0 in TP2(A) group as an example, in implementing interactive communication between GPU0 and GPU1, in addition to the direct communication link between GPU0 and GPU1, a communication path can also be constructed between GPU0 and GPU2 and GPU3 in another TP group through jump points. This allows GPU0 to communicate with GPU1 via the forwarding function of GPU2, and also allows GPU0 to communicate with GPU1 via the forwarding function of GPU3. This method expands the communication paths of each GPU, thereby fully utilizing the bandwidth resources of each port corresponding to a single GPU, thus improving the bandwidth resource utilization of the GPU and ultimately improving data processing efficiency.

[0026] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0027] Next, we will combine the following Figure 3 One of the communication methods for a graphics processor cluster provided in this application is illustrated by way of example.

[0028] Figure 3 This is a flowchart illustrating a communication method for a graphics processor cluster according to an embodiment of this application. In this method, the graphics processor cluster is divided into multiple tensor parallel groups, and each tensor parallel group contains multiple graphics processors. The multiple graphics processors within each tensor parallel group communicate with each other via a first communication link. (See also...) Figure 3 The communication method for a graphics processor cluster provided in this application includes the following steps 301-302: Step 301: For each first target graphics processor in the first tensor parallel group, a second communication link is pre-constructed between the first target graphics processor and each second target graphics processor in the second tensor parallel group; wherein, the first tensor parallel group is any tensor parallel group among multiple tensor parallel groups, and the second tensor parallel group is a tensor parallel group that is different from the first tensor parallel group among multiple tensor parallel groups.

[0029] Step 302: Control the first sub-graphics processor in the first tensor parallel group to divide the internally stored data to be reduced into multiple sub-data to be reduced, and transmit the multiple sub-data to be reduced to the second sub-graphics processor in the first tensor parallel group respectively through the first communication link and the second communication link.

[0030] pass Figure 3 The illustrated embodiment demonstrates that this method can pre-build cross-group communication links to extend the communication links of each GPU, thereby fully utilizing the bandwidth resources corresponding to each GPU port and improving resource utilization. Then, during data transmission, the data to be reduced stored internally by the graphics processor is divided into multiple sub-data to be reduced, and synchronous data transmission is performed based on multiple communication paths composed of the constructed multiple communication links, improving data transmission / communication efficiency and ensuring high-efficiency data processing.

[0031] The following will describe in conjunction with specific embodiments Figure 3 The specific implementation methods of each step in the illustrated embodiment are described in detail below: In step 301, for each first target graphics processor in the first tensor parallel group, a second communication link is pre-constructed between the first target graphics processor and each second target graphics processor in the second tensor parallel group; wherein, the first tensor parallel group is any tensor parallel group among multiple tensor parallel groups, and the second tensor parallel group is a tensor parallel group among multiple tensor parallel groups that is different from the first tensor parallel group.

[0032] For example, for each first target graphics processor in the first tensor parallel group, a second communication link needs to be pre-built with each second target graphics processor in the second tensor parallel group so that multiple ports provided by the first target graphics processor can be fully utilized, thereby realizing the formation of multiple communication links.

[0033] For example, continue to combine Figure 2As shown, assuming the first tensor parallel group is TP2(A) and the second tensor parallel group is TP2(B), then for GPU0 and GPU1 in TP2(A), a second communication link can be constructed between GPU0 and GPU2 in TP2(B); a second communication link can be constructed between GPU0 and GPU3 in TP2(B); a second communication link can be constructed between GPU1 and GPU2 in TP2(B); and a second communication link can be constructed between GPU1 and GPU3 in TP2(B).

[0034] When performing step 301 above, which involves pre-constructing a second communication link for each second target graphics processor in the first target graphics processor and the second tensor parallel group, in an optional embodiment of this application, the idle physical ports of the first target graphics processor and the idle physical ports of the second target graphics processor are connected by hardware wiring to form the second communication link.

[0035] The second target graphics processor stores the data routing list.

[0036] For example, when the first target graphics processor in the first tensor parallel group and each second target graphics processor in the second tensor parallel group meet specific requirements (e.g., a fully interconnected topology on the same NVSwitch, low hardware deployment cost, etc.), the construction of the second communication link can be implemented in hardware.

[0037] For example, if we put Figure 2 GPU0 and GPU2 are deployed in the same fully interconnected NVSwitch topology. After the system is powered on, the system driver automatically enumerates all NVSwitch ports, confirms that the idle NVLink links of GPU0 and GPU2 are connected to the available switching ports of the same NVSwitch, and that the physical status of the links is normal. Then, the routing rules can be issued through the control plane of the NVSwitch to directly connect and bind the NVSwitch port corresponding to GPU0 and the NVSwitch port corresponding to GPU2, bypassing the topology isolation restrictions between the original TP2 packets, and directly establishing a full-line-speed NVLink data channel between the two ports through hardware.

[0038] In this method, with Figure 2 Taking GPU2 as an example, it stores the corresponding data routing list internally, so that when it receives the data to be reduced from GPU0, it can directly forward it to GPU1 according to the data routing list.

[0039] When the specific requirements of hardware deployment are not met, in another alternative embodiment of this application, the first target graphics processor and each second target graphics processor in the second tensor parallel group can be added to the same pre-created shared communication domain by a software program to build a second communication link in the shared communication domain.

[0040] For example, the method can construct a shared communication domain through a software program, in which a direct link can be established between the first target graphics processor and the second target graphics processor.

[0041] For example, when the graphics processing unit (GPU) performs internal data computation, it receives an external computing power request, creates a task object based on the request, adds the task object to the ready queue and registers its status as waiting, creates a computing power coroutine for the task object, and generates a corresponding computing power device affinity tag; based on the computing power coroutine, it initiates asynchronous result retrieval requests in parallel to the N remote nodes it depends on, and sets the unfinished dependency count to N; after all requests are initiated, if the count is not zero, the computing power coroutine enters a single-time suspension state; during suspension, it saves an intermediate merged state snapshot and reuses the device memory handle already mapped to the computing power device; whenever the computation result of any remote node is received, the unfinished... The dependency count is decremented, and the result is written to the intermediate merge state snapshot. When the incomplete dependency count decreases to zero, the computing power coroutine is awakened and enters the computing power stage based on the latest updated intermediate merge state. The coroutine is scheduled to the corresponding worker thread of the device according to the computing power device affinity tag. The device memory is accessed directly through the reused device memory handle. The computing power calculation is performed and the computing power result is written to the shared device buffer in the device memory. The computing power result is returned to the coroutine through the shared device buffer pointer and asynchronously returned directly through the asynchronous write-back interface, without copying in the host memory. The task status is set to completed. The coroutine stack context and device memory handle are reclaimed and added to the reuse pool. The coroutine scheduler applies a first scheduling quantum and a preemption threshold to computing power coroutines occupying computing power devices, and applies a second scheduling quantum to input / output coroutines that do not occupy computing power devices. The first scheduling quantum is less than the second scheduling quantum to avoid computing power coroutines monopolizing computing power devices for a long time.

[0042] The task object includes a task ID, task type, input data reference, required resources, a list of dependent remote nodes, timeout, and priority. The computing power device affinity tag is generated through the following steps: querying the computing power device capability table for compatible computing power device types based on the task type; determining the associated worker thread and NUMA node based on the device topology; and encoding the device type, associated worker thread, and NUMA node as a computing power device affinity tag.

[0043] In one optional embodiment of this application, the first communication link is a dedicated link, used to provide dedicated resources for communication between multiple graphics processors contained in the tensor parallel group; the second communication link is a shared link of the graphics processor cluster.

[0044] For example, the first communication link is a dedicated direct link; optionally, the direct link is at least one of an NVLink link and a PCIe link.

[0045] The first communication link provides dedicated link resources only for communication between multiple graphics processors in the tensor parallel group; while the second communication link is a shared link for the graphics processor cluster, meaning that GPUs in cross-group communication can all use this bandwidth resource.

[0046] In step 302, the first sub-graphics processor in the first tensor parallel group is controlled to divide the internally stored data to be reduced into multiple sub-data to be reduced, and the multiple sub-data to be reduced are transmitted to the second sub-graphics processor in the first tensor parallel group based on the first communication link and the second communication link, respectively.

[0047] For example, when multiple communication links exist, the data to be reduced can be divided into multiple sub-data to be reduced, so that it can be transmitted simultaneously based on multiple communication links, achieving the effect of parallel transmission to the second sub-graphics processor in the first tensor parallel group.

[0048] In an optional embodiment of this application, the step of transmitting multiple sub-data to be reduced to a second sub-graphics processor in a first tensor parallel packet via a first communication link and a second communication link respectively includes: controlling the first sub-graphics processor in the first tensor parallel packet to determine the resource status of the second communication link; when the resource status of the second communication link is idle, controlling the first sub-graphics processor to transmit the sub-data to be reduced to the second sub-graphics processor in the first tensor parallel packet via the second communication link; and updating the resource status of the second communication link to busy.

[0049] For example, before data transmission, it is necessary to determine whether the resource status of each communication link is idle. When the resource status of the communication link is idle, that is, when the bandwidth resources are not occupied, data transmission can be carried out based on the idle second communication link.

[0050] In an optional embodiment of this application, the method further includes: When the resource status of the second communication link is busy, the first sub-graphics processor is controlled to wait until the resource status of the second communication link is idle.

[0051] For example, when the resource status of the communication link is busy, that is, when the bandwidth resources are occupied, the first sub-graphics processor needs to be controlled to wait until the resource status of the second communication link is idle, and then data transmission is carried out based on the second communication link.

[0052] For example, with Figure 2 In the example shown, when GPU0 determines that the communication link between GPU0, GPU2, and GPU1 is busy, i.e., the bandwidth resources are occupied, it cannot use the communication link and needs to wait for the communication link to become idle before it can be used.

[0053] In another scenario, where the second tensor parallel group contains at least three sub-graphics processors, and the jump point GPU after cross-grouping is greater than 2, the aforementioned step of transmitting multiple sub-data to be reduced to the second sub-graphics processor in the first tensor parallel group via the first communication link and the second communication link respectively includes: Determine the shortest route path among the routing paths for the first sub-graphics processor; obtain the load status of each third sub-graphics processor in the second tensor parallel group on the shortest route path; if the load status is less than the load threshold, then transmit the sub-data to be reduced to the second sub-graphics processor in the first tensor parallel group based on the first target communication link in the shortest route path; wherein, the first target communication link belongs to the communication link in the second communication link.

[0054] For example, when the hop point GPU after crossing the group is greater than 2, then the path representing GPU0-GPU1 does not just use one path, but includes multiple optional routing paths. For example, with Figure 4 Taking the 8-card topology shown as an example, assuming the first tensor parallel group is TP2(C) and the second tensor parallel group is TP2(D), then for GPU0, GPU1, GPU2, and GPU3 in TP2(C), and GPU4, GPU5, GPU6, and GPU7 in TP2(D), a corresponding cross-group second communication link can be constructed for each GPU.

[0055] Taking GPU0 as an example, it has a first communication link that directly connects to GPUs 1, 2, and 3 within the same group, and also establishes a second communication link across groups. When GPU0 communicates with GPU 1, it can communicate via GPU0-GPU7-GPU1 (in which case, this routing path includes one hop GPU after crossing the group), or via GPU0-GPU7-GPU6-GPU1 (in which case, this routing path includes two hop GPUs after crossing the group), and so on. Of course, to ensure data transmission efficiency, the number of hop GPUs after the group should not be too large, and can be limited to a preset number (for example, the number of hop GPUs is less than 3). If the number of hop GPUs exceeds the preset number and there is still no idle communication link, the system determines that it will wait for a communication link to become idle before using it.

[0056] In an optional embodiment of this application, the method further includes: if the load status is not less than the load threshold, determining the load status of each fourth sub-graphics processor in the second tensor parallel group on the second shortest routing path, until the load status is less than the load threshold.

[0057] For example, data transmission paths can be selected by comparing routing paths and GPU load. That is, the shortest routing path is preferred, and then the load status of each third sub-graphics processor in the second tensor parallel group involved in the shortest routing path is taken into account to achieve optimal path selection. This process can continue to select other communication links, i.e., communication links on the second shortest routing path, by adding hop GPUs when the second communication link on the shortest routing path is occupied, and so on.

[0058] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0059] Please see Figure 5 One embodiment of this application provides a communication device 500 for a graphics processing unit cluster, which includes a communication link construction module 501 and a data transmission module 502.

[0060] The communication link construction module 501 is used to pre-construct a second communication link between each first target graphics processor in the first tensor parallel group and each second target graphics processor in the second tensor parallel group for each first target graphics processor in the first tensor parallel group; wherein, the first tensor parallel group is any tensor parallel group among multiple tensor parallel groups, and the second tensor parallel group is a tensor parallel group that is different from the first tensor parallel group among multiple tensor parallel groups; the data transmission module 502 is used to control the first sub-graphics processor in the first tensor parallel group to divide the internally stored data to be reduced into multiple sub-data to be reduced, and to transmit the multiple sub-data to be reduced to the second sub-graphics processor in the first tensor parallel group respectively based on the first communication link and the second communication link.

[0061] In one optional embodiment of this application, the communication link construction module 501 is specifically used to connect the idle physical ports of the first target graphics processor and the idle physical ports of the second target graphics processor by hardware wiring to form a second communication link, wherein the second target graphics processor stores a data routing list; or, by software program, the first target graphics processor and each second target graphics processor in the second tensor parallel group are added to the same pre-created shared communication domain to construct the second communication link in the shared communication domain.

[0062] In one optional embodiment of this application, the first communication link is a dedicated link, which provides dedicated resources for communication between multiple graphics processors contained in the tensor parallel group; the second communication link is a shared link of the graphics processor cluster.

[0063] In one optional embodiment of this application, the data transmission module 502 is configured to control the first sub-graphics processor in the first tensor parallel group to determine the resource status of the second communication link; when the resource status of the second communication link is idle, control the first sub-graphics processor to transmit the sub-data to be reduced to the second sub-graphics processor in the first tensor parallel group via the second communication link; and update the resource status of the second communication link to busy.

[0064] In an optional embodiment of this application, the data transmission module 502 is further configured to control the first sub-graphics processor to wait until the resource status of the second communication link is idle when the resource status of the second communication link is busy.

[0065] In one optional embodiment of this application, the second tensor parallel group includes at least three sub-graphics processors, and the data transmission module 502 is further configured to determine the shortest routing path among the routing paths for the first sub-graphics processor. Obtain the load status of each third sub-graphics processor in the second tensor parallel group on the shortest routing path; If the load is less than the load threshold, the sub-data to be reduced is transmitted to the second sub-graphics processor in the first tensor parallel group based on the first target communication link in the shortest routing path; wherein, the first target communication link is a communication link in the second communication link.

[0066] In an optional embodiment of this application, the data transmission module 502 is further configured to determine the load status of each fourth sub-graphics processor in the second tensor parallel group on the second shortest routing path if the load status is not less than the load threshold, until the load status is less than the load threshold.

[0067] Specific limitations regarding the communication device of the aforementioned graphics processing unit cluster can be found in the limitations on the communication method of the graphics processing unit cluster described above, and will not be repeated here. Each module in the aforementioned device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can invoke and execute the operations corresponding to each module.

[0068] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 6 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a communication method for a graphics processing unit (GPU) cluster as described above. It includes: memory and a processor; the memory stores a computer program; and the processor executes the computer program to implement any step of the communication method for the GPU cluster as described above.

[0069] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can perform any step of the communication method of the graphics processor cluster described above.

[0070] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0075] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A communication method for a graphics processing unit cluster, characterized in that, The graphics processing unit cluster is divided into multiple tensor parallel groups, each tensor parallel group containing multiple graphics processors, and the multiple graphics processors in each tensor parallel group communicate with each other through a first communication link. The method includes: For each first target graphics processor in the first tensor parallel group, a second communication link is pre-constructed between the first target graphics processor and each second target graphics processor in the second tensor parallel group; wherein, the first tensor parallel group is any tensor parallel group among the plurality of tensor parallel groups, and the second tensor parallel group is a tensor parallel group among the plurality of tensor parallel groups that is different from the first tensor parallel group. The first sub-graphics processor in the first tensor parallel group is controlled to divide the internally stored data to be reduced into multiple sub-data to be reduced, and the multiple sub-data to be reduced are transmitted to the second sub-graphics processor in the first tensor parallel group via the first communication link and the second communication link, respectively.

2. The method according to claim 1, characterized in that, The step of pre-constructing a second communication link for each second target graphics processor in the first target graphics processor and the second tensor parallel group includes: The idle physical ports of the first target graphics processor and the second target graphics processor are connected by wiring in hardware to form the second communication link. The second target graphics processor stores a data routing list. Alternatively, the first target graphics processor and each of the second target graphics processors in the second tensor parallel group can be added to the same pre-created shared communication domain by a software program to build the second communication link in the shared communication domain.

3. The method according to claim 1 or 2, characterized in that, The first communication link is a dedicated link, which provides dedicated resources for communication between multiple graphics processors contained in the tensor parallel group; the second communication link is a shared link of the graphics processor cluster.

4. The method according to claim 1, characterized in that, The step of transmitting multiple sub-data to be reduced to the second sub-graphics processor in the first tensor parallel packet via the first communication link and the second communication link respectively includes: The first sub-graphics processor in the first tensor parallel group is controlled to determine the resource status of the second communication link; When the resource status of the second communication link is idle, the first sub-graphics processor is controlled to transmit the sub-data to be reduced to the second sub-graphics processor in the first tensor parallel group via the second communication link. In addition, the resource status of the second communication link is updated to busy.

5. The method according to claim 4, characterized in that, The method further includes: When the resource status of the second communication link is busy, the first sub-graphics processor is controlled to wait until the resource status of the second communication link is idle.

6. The method according to claim 1, characterized in that, The second tensor parallel packet contains at least three sub-graphics processors. The step of transmitting multiple sub-data to be reduced to the second sub-graphics processor in the first tensor parallel packet via the first communication link and the second communication link respectively includes: Determine the shortest route path among the routing paths for the first sub-graphics processor; Obtain the load status of each third sub-graphics processor in the second tensor parallel group on the shortest routing path; If the load status is less than the load threshold, the sub-data to be reduced is transmitted to the second sub-graphics processor in the first tensor parallel group based on the first target communication link in the shortest routing path; wherein, the first target communication link belongs to the communication link in the second communication link.

7. The method according to claim 6, characterized in that, The method further includes: If the load status is not less than the load threshold, then determine the load status of each fourth sub-graphics processor in the second tensor parallel group on the second shortest routing path until the load status is less than the load threshold.

8. A communication device for a graphics processing unit cluster, characterized in that, The graphics processing unit cluster is divided into multiple tensor parallel groups, each tensor parallel group containing multiple graphics processors, and the multiple graphics processors in each tensor parallel group communicate with each other through a first communication link. The device includes: A communication link construction module is used to pre-construct a second communication link between each first target graphics processor in a first tensor parallel group and each second target graphics processor in a second tensor parallel group for each first target graphics processor in a first tensor parallel group; wherein, the first tensor parallel group is any tensor parallel group among the plurality of tensor parallel groups, and the second tensor parallel group is a tensor parallel group among the plurality of tensor parallel groups that is different from the first tensor parallel group. The data transmission module is used to control the first sub-graphics processor in the first tensor parallel group to divide the internally stored data to be reduced into multiple sub-data to be reduced, and to transmit the multiple sub-data to be reduced to the second sub-graphics processor in the first tensor parallel group respectively through the first communication link and the second communication link.

9. A computer device, comprising: A memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images