Hybrid GPU cluster, communication method, chip, and device

Abstract

Embodiments of the present application relate to the technical field of computers. Disclosed are a hybrid GPU cluster, a communication method, a chip, and a device. The hybrid GPU cluster comprises a plurality of computing nodes, each computing node comprising at least one GPU; and the plurality of computing nodes comprise at least two types of computing nodes. At least one communication group is created in the hybrid GPU cluster. A first computing node is configured to: acquire global hardware information, and create communication group information of the first computing node, wherein the communication group information is used for indicating a hybrid communication group in the hybrid GPU cluster, and the hybrid communication group comprises a homogeneous sub-communication group and a general sub-communication group; and on the basis of the communication group information, communicate with the homogeneous sub-communication group by means of a homogeneous communication library, and communicate with the general sub-communication group by means of a general communication library. The embodiments of the present application break communication barriers between different types of GPUs, and enable effective utilization of different types of GPUs in the hybrid GPU cluster for task processing, thereby improving processing efficiency.

Description

Hybrid GPU clusters, communication methods, chips and devices

[0001] This application claims priority to Chinese Patent Application No. 202411576983.3, filed on November 6, 2024, entitled "Hybrid GPU Cluster, Communication Method, Chip and Device", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to a hybrid GPU cluster, communication method, chip, and device. Background Technology

[0003] A hybrid GPU (Graphics Processing Unit) cluster is a computer cluster composed of different types of GPUs. Hybrid GPU clusters typically include GPUs of different models, architectures, and even from different manufacturers. Different types of GPUs have performance advantages in different aspects; by using different types of GPUs in combination, their performance advantages can be complemented, improving the overall performance of the entire cluster. Summary of the Invention

[0004] This application provides a hybrid GPU cluster, communication method, chip, and device that breaks down communication barriers between different types of GPUs, effectively utilizing different types of GPUs in the hybrid GPU cluster to process tasks and improving processing efficiency. The technical solution is as follows.

[0005] On one hand, a hybrid GPU cluster is provided, comprising multiple compute nodes, each compute node including at least one graphics processing unit (GPU); wherein the multiple compute nodes include at least two types of compute nodes, and the types of GPUs in the different types of compute nodes are different; the hybrid GPU cluster creates at least one communication group, the communication group being a communication set composed of multiple GPUs; a first compute node is used to acquire global hardware information, the global hardware information including the hardware information of each compute node in the hybrid GPU cluster, the hardware information including the type of GPU in the compute node; based on the global hardware information, the first compute node is created. The communication group information is used to indicate the hybrid communication group in the hybrid GPU cluster. The hybrid communication group includes a similar sub-communication group and a general sub-communication group. The similar sub-communication group includes GPUs that conform to the first compute node type, and the general sub-communication group includes GPUs that do not conform to the first compute node type. The first compute node is also used to communicate with the similar sub-communication group through a similar communication library and with the general sub-communication group through a general communication library based on the communication group information. The general communication library refers to a communication library that can communicate between different types of GPUs. The first compute node is any compute node in the hybrid GPU cluster.

[0006] On the other hand, a communication method is provided, wherein a hybrid GPU cluster includes multiple computing nodes, each computing node including at least one graphics processing unit (GPU); wherein the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in the different types of computing nodes are different; the hybrid GPU cluster creates at least one communication group, the communication group being a communication set composed of multiple GPUs; the method includes: a first computing node acquiring global hardware information, the global hardware information including hardware information of each computing node in the hybrid GPU cluster, the hardware information including the type of GPU in the computing node; based on the global hardware information, creating communication group information for the first computing node, the communication group information being used to indicate a hybrid communication group in the hybrid GPU cluster, the hybrid communication group including a similar sub-communication group and a general sub-communication group, the similar sub-communication group including GPUs conforming to the type of the first computing node, and the general sub-communication group including GPUs not conforming to the type of the first computing node; the first computing node, based on the communication group information, communicating with the similar sub-communication group through a similar communication library, and communicating with the general sub-communication group through a general communication library, the general communication library being a communication library capable of communication between different types of GPUs; wherein the first computing node is any computing node in the hybrid GPU cluster.

[0007] On the other hand, a chip is provided configured in a hybrid GPU cluster, the hybrid GPU cluster comprising multiple chips, each chip comprising at least one graphics processing unit (GPU); wherein the multiple chips comprise at least two types of chips, and the types of GPUs in the different types of chips are different; the hybrid GPU cluster creates at least one communication group, the communication group being a communication set composed of multiple GPUs; the chip is used to acquire global hardware information, the global hardware information including hardware information of each chip in the hybrid GPU cluster, the hardware information including the type of GPU in the chip; based on the global hardware information, the chip creates communication group information, the communication group information being used to indicate hybrid communication groups in the hybrid GPU cluster, the hybrid communication groups including similar sub-communication groups and general sub-communication groups, the similar sub-communication groups including GPUs conforming to the chip type, and the general sub-communication groups including GPUs not conforming to the chip type; the chip is also used to communicate with the similar sub-communication groups through a similar communication library, and with the general sub-communication groups through a general communication library, the general communication library being a communication library capable of communication between different types of GPUs, based on the communication group information.

[0008] On the other hand, a communication device is provided, the device being configured on a first computing node in a hybrid GPU cluster, the hybrid GPU cluster including multiple computing nodes, each computing node including at least one graphics processing unit (GPU); wherein, the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in the different types of computing nodes are different; the hybrid GPU cluster creates at least one communication group, the communication group being a communication set composed of multiple GPUs;

[0009] The apparatus includes: an information processing module for acquiring global hardware information, the global hardware information including hardware information of each computing node in the hybrid GPU cluster, the hardware information including the type of GPU in the computing node; creating communication group information for the first computing node based on the global hardware information, the communication group information indicating a hybrid communication group in the hybrid GPU cluster, the hybrid communication group including a similar sub-communication group and a general sub-communication group, the similar sub-communication group including GPUs conforming to the type of the first computing node, and the general sub-communication group including GPUs not conforming to the type of the first computing node; and a communication module for communicating with the similar sub-communication group through a similar communication library and with the general sub-communication group through a general communication library, the general communication library being a communication library capable of communication between different types of GPUs; wherein, the first computing node is any computing node in the hybrid GPU cluster.

[0010] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to perform the operations performed by the communication method as described above.

[0011] On the other hand, a computer-readable storage medium is provided that stores at least one computer program, which is loaded and executed by a processor to perform the operations performed by the communication method described above.

[0012] On the other hand, a computer program product is provided, including a computer program loaded and executed by a processor to perform the operations performed by the communication method described above. Attached Figure Description

[0013] Figure 1 is a schematic diagram of a GPU cluster provided by related technologies.

[0014] Figure 2 is a schematic diagram of another GPU cluster provided by related technologies.

[0015] Figure 3 is a schematic diagram of a hybrid GPU cluster provided in an embodiment of this application.

[0016] Figure 4 is a schematic diagram of a communication group and communication library of a first type of computing node provided in an embodiment of this application.

[0017] Figure 5 is a schematic diagram of a communication group and communication library of a second type of computing node provided in an embodiment of this application.

[0018] Figure 6 is a schematic diagram of a communication group in a hybrid GPU cluster provided in an embodiment of this application.

[0019] Figure 7 is a schematic diagram of a communication process using a general communication library provided in an embodiment of this application.

[0020] Figure 8 is a schematic diagram of communication between GPUs of the same type provided in an embodiment of this application.

[0021] Figure 9 is a schematic diagram of a process for collective communication between node 1 and node 2 provided in an embodiment of this application.

[0022] Figure 10 is a schematic diagram of a process for collective communication between node 0 and node 1 provided in an embodiment of this application.

[0023] Figure 11 is a schematic diagram of a process for collective communication between node 2 and node 3 provided in an embodiment of this application.

[0024] Figure 12 is a schematic diagram of the model layer allocated to each computing node in a hybrid GPU cluster provided in an embodiment of this application.

[0025] Figure 13 is a schematic diagram of the model layer allocated to each computing node in another hybrid GPU cluster provided in an embodiment of this application.

[0026] Figure 14 is a flowchart of a communication method provided in an embodiment of this application.

[0027] Figure 15 is a schematic diagram of the structure of a communication device provided in an embodiment of this application.

[0028] Figure 16 is a schematic diagram of the structure of a terminal provided in an embodiment of this application.

[0029] Figure 17 is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0030] It is understood that the terms "first," "second," etc., used in this application may be used to describe various concepts herein, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of this application, a first computing node may be referred to as a second computing node, and similarly, a second computing node may be referred to as a first computing node.

[0031] Here, "at least two" refers to two or more computing nodes. For example, at least two computing nodes can be any integer number of computing nodes greater than or equal to two, such as two computing nodes, three computing nodes, etc. "Each" refers to each of the at least two computing nodes. For example, "each computing node" refers to each of the at least two computing nodes. If the at least two computing nodes are three computing nodes, then "each computing node" refers to each of the three computing nodes.

[0032] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices) involved in this application have all been fully authorized by the user or relevant parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0033] First, the concepts involved in the embodiments of this application will be explained as follows.

[0034] 1. RDMA (Remote Direct Memory Access): This technology was developed to solve the latency of server-side data processing in network transmission. RDMA directly transfers data to the computer's memory area over the network, quickly moving data from one system to a remote system's memory without affecting the operating system.

[0035] RoCE V2 (RDMA over Converged Ethernet V2) is an RDMA protocol designed to achieve low-latency, high-throughput data transmission in Ethernet environments. It implements a direct memory access mechanism between systems, minimizing CPU involvement and reducing communication latency.

[0036] 2. GDR (GPU Direct RDMA): A technology for direct data access and communication between the GPU and RDMA network devices, which bypasses the host memory and CPU to directly transmit data between the GPU and RDMA network devices.

[0037] 3. H2D (Host to Device): Data is moved from CPU memory to GPU high-speed bandwidth memory.

[0038] 4. D2H (Device to Host): Data is moved from the GPU's high-speed bandwidth memory to the CPU's memory.

[0039] 5. Communication group: A communication set consisting of multiple GPUs.

[0040] Figure 1 is a schematic diagram of a GPU cluster provided by related technologies. Referring to Figure 1, the GPU cluster includes multiple computing nodes (Figure 1 uses 4 computing nodes as an example), and each computing node includes at least one GPU (Figure 1 uses 8 GPUs as an example). All GPUs in this GPU cluster are of the same type, therefore, all computing nodes in this GPU cluster are of the same type. In this embodiment, for ease of distinction, the GPUs in the GPU cluster shown in Figure 1 are referred to as first-type GPUs, and the communication library used by the first-type GPUs is referred to as the first communication library.

[0041] Figure 2 is a schematic diagram of another GPU cluster provided by related technologies. Referring to Figure 2, the GPU cluster includes multiple computing nodes (Figure 2 uses 4 computing nodes as an example), and each computing node includes at least one GPU (Figure 2 uses 8 GPUs as an example). All GPUs in this GPU cluster are of the same type, therefore, all computing nodes in this GPU cluster are of the same type. Furthermore, the type of GPU in Figure 2 is different from the type of GPU in Figure 1, therefore, the type of computing nodes in Figure 2 is also different from the type of computing nodes in Figure 1. In this embodiment, for ease of distinction, the GPUs in the GPU cluster shown in Figure 2 are referred to as second-type GPUs, and the communication library used by the second-type GPUs is referred to as the second communication library.

[0042] Figure 3 is a schematic diagram of a hybrid GPU cluster provided in an embodiment of this application. Referring to Figure 3, the hybrid GPU cluster includes multiple computing nodes (Figure 3 uses 4 computing nodes as an example), and each computing node includes at least one GPU (Figure 3 uses 8 GPUs as an example). The same computing node includes GPUs of the same type, and multiple computing nodes include at least two types of computing nodes. The types of GPUs in different types of computing nodes are different. Each GPU in the hybrid GPU cluster is an independent unit that can provide data processing functions.

[0043] Each computing node in a hybrid GPU cluster can be a terminal, a server, or a chip, etc., and this application embodiment does not limit this. Optionally, the terminal can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. Optionally, the server can be an independent physical server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, and big data and artificial intelligence platforms. A hybrid GPU cluster can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers.

[0044] Referring to Figure 3, nodes 0 and 1 are first-type computing nodes, including first-type GPUs, and the communication library used by nodes 0 and 1 is the first communication library. Nodes 2 and 3 are second-type computing nodes, including second-type GPUs, and the communication library used by nodes 2 and 3 is the second communication library. Furthermore, the communication library used when communicating between different types of computing nodes is a general-purpose communication library. This general-purpose communication library refers to a communication library capable of communication between different types of GPUs, and can be the GLOO communication library or other communication libraries, etc., which is not limited in this embodiment.

[0045] It should be noted that, for ease of illustration, Figure 1 only shows four compute nodes in the hybrid GPU cluster and eight GPUs in each compute node. In reality, the hybrid GPU cluster can include more compute nodes, and each compute node can include more GPUs; this embodiment does not limit this. Furthermore, the GPUs involved in this embodiment can be GPGPUs (General-Purpose computing on Graphics Processing Units) or other types of GPUs; this embodiment does not limit this. GPGPUs utilize graphics processing units that handle graphics tasks to compute general-purpose computing tasks originally processed by the central processing unit. These general-purpose computing tasks may be unrelated to graphics processing.

[0046] This application embodiment takes a first computing node as an example, where the first computing node is any computing node in a hybrid GPU cluster. The first computing node is used to obtain global hardware information, including the hardware information of each computing node in the hybrid GPU cluster, and the type of GPU in the computing node. Based on the global hardware information, it creates communication group information for the first computing node. This communication group information indicates hybrid communication groups in the hybrid GPU cluster. Hybrid communication groups include similar sub-communication groups and general sub-communication groups. Similar sub-communication groups include GPUs that match the type of the first computing node, and general sub-communication groups include GPUs that do not match the type of the first computing node. Furthermore, the first computing node is also used to communicate with similar sub-communication groups through similar communication libraries and with general sub-communication groups through general communication libraries, based on the communication group information. The functions of other computing nodes in the hybrid GPU cluster are similar to those of the first computing node, and will not be described again in this application embodiment.

[0047] In addition to the GPU, the first computing node may also include a CPU, RoCE V2 network interface card (NIC), switch, or other units; this application embodiment does not limit this. Besides including the type of GPU, the hardware information of each computing node may also include the number of RoCE V2 NICs in the computing node, the manufacturer, version number, and production date of the computing node; this application embodiment does not limit this.

[0048] In one possible implementation, considering that the hybrid GPU cluster includes different types of GPUs, each GPU in the hybrid GPU cluster can communicate with other GPUs through a common communication library to obtain the hardware information of other GPUs, thereby obtaining global hardware information. This avoids the problem of different types of GPUs being unable to communicate and ensures that global hardware information is obtained smoothly and accurately.

[0049] In this embodiment, at least one communication group, such as one or more, can be created in a hybrid GPU cluster. These communication groups can be PP (Pipeline Parallelism), TP (Tensor Parallelism), or DP (Data Parallelism), etc. Communication groups can be created by default by the hybrid GPU cluster, or as instructed by a technician, or within the framework of the neural network model currently being processed by the hybrid GPU cluster, or according to the type of task to be processed. This embodiment does not limit the scope of these creations. A single communication group can include GPUs located on the same computing node or GPUs located on different computing nodes. Furthermore, a single communication group can include only GPUs of the same type or different types of GPUs. Communication occurs between different GPUs within each communication group, thus completing the communication tasks of the hybrid GPU cluster. If a communication group includes different types of GPUs, communication between these different types of GPUs is required, necessitating the breaking down of communication barriers between them.

[0050] This application provides an adaptive method for creating communication groups. The first computing node can determine the type of GPU in each computing node in the hybrid GPU cluster based on global hardware information, thereby determining the type of GPU in each communication group in the hybrid GPU cluster, thus determining the global topology of the hybrid GPU cluster. Then, the first computing node can adaptively create the communication group information of the first computing node.

[0051] If the GPUs in any communication group are of the same type, such as GPUs with the same architecture or GPUs from the same manufacturer, then the communication group is considered a single communication group. In this case, if the GPUs in the communication group match the type of the first compute node, then the communication group is considered a homogeneous communication group. If none of the GPUs in the communication group match the type of the first compute node, then the communication group is considered a heterogeneous communication group, or the communication group is ignored.

[0052] If any communication group includes GPUs of different types, such as GPUs with different architectures or GPUs from different manufacturers, it is considered a hybrid communication group. In this case, the GPUs in the hybrid communication group are divided into sub-communication groups of the same type and general type. Sub-communication groups of the same type include GPUs that conform to the type of the first computing node, and general type sub-communication groups include GPUs that do not conform to the type of the first computing node. This allows the first computing node to communicate with the sub-communication groups of the same type and general type through different communication libraries.

[0053] Among them, a GPU that conforms to the type of the first computing node refers to a GPU of the same type as the GPU in the first computing node, and a GPU that does not conform to the type of the first computing node refers to a GPU of a different type than the GPU in the first computing node.

[0054] After classifying the communication groups in the hybrid GPU cluster, the first compute node can determine that it can communicate with the same type of communication group and the same type of sub-communication group through the same type of communication library, while it needs to communicate with the general sub-communication group through the general communication library. This is equivalent to setting up two different communication libraries for the hybrid communication group, ensuring that the first compute node can communicate with the same type of GPU and different types of GPUs in the hybrid communication group through different communication libraries respectively.

[0055] For example, a hybrid GPU cluster includes first-type compute nodes and second-type compute nodes. First-type compute nodes include first-type GPUs, and second-type compute nodes include second-type GPUs. Referring to Figure 4, if the first compute node is a first-type compute node and determines that the communication group only includes first-type GPUs and not second-type GPUs, then this communication group is designated as the first communication group (i.e., a homogeneous communication group), and the first communication group uses the first communication library. However, if the communication group includes both first-type and second-type GPUs, then this communication group is designated as a hybrid communication group, and this communication group is further divided into a first sub-communication group (i.e., a homogeneous sub-communication group) and a general sub-communication group according to the GPU type. The first sub-communication group uses the first communication library, while the general sub-communication group uses the general communication library.

[0056] Referring to Figure 5, when the first computing node is a type II computing node, if the first computing node determines that the communication group only includes type II GPUs and not type I GPUs, then this communication group is designated as a second communication group (i.e., a homogeneous communication group), and the second communication group uses the second communication library. However, if the communication group includes both type I and type II GPUs, then this communication group is designated as a hybrid communication group, and this communication group is further divided into second sub-communication groups (i.e., homogeneous sub-communication groups) and general sub-communication groups according to the GPU type. The second sub-communication groups use the second communication library, while the general sub-communication groups use the general communication library.

[0057] Referring to Figure 6, the hybrid GPU cluster includes four compute nodes: node 0, node 1, node 2, and node 3. Each compute node includes eight GPUs, and three communication groups are created in the hybrid GPU cluster: communication group 601, communication group 602, and communication group 603. Communication group 601 includes first-type GPUs, so communication group 601 is the first communication group. Communication group 602 includes second-type GPUs, so communication group 602 is the second communication group. Communication group 603 includes both first-type and second-type GPUs, so communication group 603 is a hybrid communication group. In this hybrid communication group, the first-type GPUs located on node 0 and node 1 constitute the first sub-communication group, and the second-type GPUs located on node 2 and node 3 constitute the second sub-communication group.

[0058] For nodes 0 and 1, communication group 601 is a sub-communication group of the same type. Furthermore, in communication group 603, the first type of GPUs located on nodes 0 and 1 constitute a sub-communication group of the same type, while the second type of GPUs located on nodes 2 and 3 constitute a general sub-communication group. For nodes 2 and 3, communication group 602 is a sub-communication group of the same type. Furthermore, in communication group 603, the first type of GPUs located on nodes 0 and 1 constitute a general sub-communication group, while the second type of GPUs located on nodes 2 and 3 constitute a sub-communication group of the same type.

[0059] It should be noted that the process of determining the communication library corresponding to each communication group can be implemented during the initialization of the hybrid GPU cluster. In related technologies, when a hybrid GPU cluster is initialized, multiple communication groups are determined, and corresponding communication libraries are also determined for communication groups of the same type as the current compute node. However, for communication groups of different types from the current compute node, due to the communication barrier between the current compute node and this communication group, it is impossible to determine which communication library to use, thus leading to initialization failure. In the embodiments of this application, the communication barrier is broken, and even for communication groups of different types from the current compute node, the appropriate communication library is adaptively determined based on the type of GPU in the communication group.

[0060] This application provides a scheme for adaptive communication in a hybrid GPU cluster. Each compute node in the hybrid GPU cluster creates communication group information based on the hardware information of each GPU, thereby dividing the communication groups in the hybrid GPU cluster according to GPU type. This allows for the determination of the hybrid communication groups for compute nodes, further dividing these groups into similar sub-communication groups and general sub-communication groups. Appropriate communication libraries are adaptively selected based on the type of the sub-communication group. Communication with similar sub-communication groups is achieved using the similar communication library, while communication with general sub-communication groups is achieved using the general communication library. This enables communication between GPUs of the same type and between different types of GPUs, breaking down communication barriers between different types of GPUs. This effectively utilizes the different types of GPUs in the hybrid GPU cluster to process tasks, improving processing efficiency. Furthermore, this application can be applied to any type of GPU and can solve the problem of communication barriers between GPUs from different manufacturers.

[0061] Based on the above embodiments, the process of the first computing node creating communication group information will be described below.

[0062] The first compute node creates communication group information for each communication group in the hybrid GPU cluster, based on the type of GPU in that communication group and the type of GPU in the first compute node, thereby obtaining the communication group information for each communication group in the hybrid GPU cluster.

[0063] In one possible implementation, the first compute node is used to create first communication group information corresponding to any communication group in the hybrid GPU cluster that, based on global hardware information, includes GPUs that conform to the first compute node type and also includes GPUs that do not conform to the first compute node type. The first communication group information is used to indicate that the communication group is a hybrid communication group, in which GPUs that conform to the first compute node type constitute similar sub-communication groups, and in which GPUs that do not conform to the first compute node type constitute general sub-communication groups.

[0064] In another possible implementation, the first compute node is further configured to create second communication group information corresponding to any communication group in the hybrid GPU cluster that, based on global hardware information, includes GPUs that conform to the type of the first compute node and does not include GPUs that do not conform to the type of the first compute node; wherein the second communication group information is used to indicate that the communication group is a communication group of the same type.

[0065] In the hybrid GPU cluster provided in this application embodiment, the first computing node can determine whether the type of GPU in each communication group in the hybrid GPU cluster matches the type of the first computing node based on global hardware information, thereby adaptively creating communication group information corresponding to each communication group. Different types of communication groups are distinguished by different communication group information, which realizes the accurate division of communication groups and lays the foundation for adaptively selecting the communication library corresponding to each communication group.

[0066] In addition, the first compute node is also used to create third communication group information corresponding to any communication group in the hybrid GPU cluster that, based on global hardware information, includes GPUs that do not conform to the type of the first compute node and does not include GPUs that conform to the type of the first compute node. The third communication group information is used to indicate that the communication group is a heterogeneous communication group.

[0067] In addition, considering that the heterogeneous communication group will not include the GPU in the first compute node, the first compute node will not communicate with the heterogeneous communication group. Therefore, the process of creating the third communication group information mentioned above can also be omitted. That is, the first compute node is also used to ignore the communication group if it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes a GPU that does not conform to the type of the first compute node and does not include a GPU that conforms to the type of the first compute node.

[0068] In one possible implementation, the hybrid GPU cluster provided in this application embodiment is used in model training scenarios to process various tasks during the training of neural network models. This allows for the combined use of different types of GPUs in model training, effectively reducing GPU idle time, expanding the scale of model training, and improving training efficiency. Therefore, training efficiency can reach over 90% or even higher. Furthermore, the hybrid GPU cluster does not require modification of the training framework code during communication; thus, this communication method can be used independently, achieving decoupling between communication and computation.

[0069] In another possible implementation, the hybrid GPU cluster provided in this application embodiment can also be used in model inference scenarios, enabling the execution of corresponding inference tasks through the trained neural network model, thereby improving inference efficiency.

[0070] The hybrid GPU clusters provided in related technologies have high coupling with the training framework, with both communication and computation tasks coupled within the neural network model training framework. This makes it impossible to optimize communication or computation issues independently, and they are primarily applied to neural network model training scenarios, not to scenarios involving inference based on neural network models. In contrast, the hybrid GPU cluster provided in this application fully decouples communication and computation tasks, resulting in low coupling and the ability to optimize either communication or computation independently. It also offers strong scalability. Furthermore, it can be applied to both neural network model training and inference scenarios, making its application range very wide.

[0071] Based on the above embodiments, in point-to-point communication tasks, if two GPUs need to communicate, the communication library to be used needs to be adaptively determined based on whether the two GPUs are of the same type.

[0072] In one possible implementation, a first computing node is used to send communication data of the first GPU to a first CPU in the first computing node when the communication target of the first GPU is determined to be the second GPU in the general sub-communication group based on the communication group information. The first CPU then sends communication data to the second CPU in the second computing node through the general communication library. Here, the first GPU is any GPU in the first computing node, the second computing node is the computing node where the second GPU is located, and the second CPU is used to receive communication data and send communication data to the second GPU through the general communication library.

[0073] This embodiment of the application takes the example of a first GPU needing to send communication data to a second GPU. The first computing node receives a point-to-point communication task and has already created communication group information. Based on the point-to-point communication task and the communication group information, the first and second GPUs that need to communicate, as well as the communication groups they belong to, can be determined. If the first GPU is located on the first computing node, and the first and second GPUs are in the same hybrid communication group, and the second GPU is located in the general sub-communication group of the first computing node, this indicates that the first and second GPUs are different types of GPUs. Therefore, when the first GPU and the second GPU communicate, they need to communicate through a general communication library. Referring to Figure 6, taking node 1 as the first computing node, if communication group 603 needs to execute a communication task, node 1 determines that GPU 4 in node 1 needs to send communication data to GPU 0 in node 2. Since GPU 0 in node 2 is located in the general sub-communication group of node 1, communication needs to be performed through a general communication library.

[0074] The first computing node includes a first GPU and a first CPU, and the second computing node includes a second GPU and a second CPU. In the communication process between the first GPU and the second GPU, the first GPU first performs a D2H operation to move the communication data from the first GPU to the first CPU. Then, the first CPU sends the communication data to the second CPU through a general database. Finally, the second CPU performs an H2D operation to move the communication data from the second CPU to the second GPU.

[0075] Referring to Figure 7, the first GPU is GPU0 in Figure 7, and the second GPU is GPU1 in Figure 7; they are GPUs of different types. During point-to-point communication, the first GPU performs a D2H operation to transfer the device data (i.e., the communication data in the first GPU) to the first CPU, thus obtaining the host data. Then, through the GLOO communication library, the host data is transferred to the RoCE V2 network card of the first compute node. Using the RoCE V2 protocol, the host data is sent to the RoCE V2 network card of the second compute node. The RoCE V2 network card of the second compute node then transfers the host data to the second CPU through the GLOO communication library. The second CPU executes an H2D operation to transfer the host data to the second GPU, thus obtaining the device data sent by the first GPU.

[0076] In another possible implementation, the first computing node is used to send communication data of the third GPU to the fourth GPU through a similar communication library when the communication target of the third GPU is determined to be the fourth GPU in the same sub-communication group based on the communication group information; wherein the third GPU is any GPU in the first computing node.

[0077] This application embodiment takes the example of a third GPU needing to send communication data to a fourth GPU. The third GPU is located in the first computing node, while the fourth GPU is located in the same type of sub-communication group of the first computing node. This indicates that the third GPU and the fourth GPU are GPUs of the same type. Therefore, when the third GPU and the fourth GPU communicate with each other, they need to communicate through the same type of communication library.

[0078] For example, referring to Figure 8, if the third GPU and the fourth GPU are of the first type of GPU, then the communication library of the same type is the first communication library; or, if the third GPU and the fourth GPU are of the second type of GPU, then the communication library of the same type is the second communication library.

[0079] In this embodiment, communication group information is created based on global hardware information, thereby dividing the hybrid communication group into similar sub-communication groups and general sub-communication groups. This allows communication between the first and second GPUs of different types within the hybrid communication group to be achieved through a general communication library, with the assistance of the first and second CPUs forwarding communication data. This solves the communication barrier problem between different types of GPUs. Furthermore, for the third and fourth GPUs of the same type within the hybrid communication group, they can communicate directly through a similar communication library. Therefore, this embodiment can adaptively select different communication libraries within the same communication group, rather than being limited to using the same communication library within the same group, thus improving flexibility.

[0080] The above embodiments use a hybrid communication group, including a first compute node, in a hybrid GPU cluster as an example to illustrate the process of determining a hybrid communication group. In another possible implementation, the communication group information created by the first compute node is also used to indicate similar communication groups within the hybrid GPU cluster. These similar communication groups include GPUs of the same type as the first compute node; that is, the GPUs in these similar communication groups are all of the same type as the GPUs in the first compute node. In this case, the first compute node is also used to communicate with these similar communication groups through a similar communication library based on the communication group information.

[0081] For example, taking the case where a fifth GPU needs to send communication data to a sixth GPU, the first computing node, upon determining based on communication group information that the communication target of the fifth GPU is the sixth GPU in the same communication group, sends the communication data of the fifth GPU to the sixth GPU through a similar communication library. Here, the fifth GPU can be any GPU in the first computing node. Since the sixth GPU is located in the same communication group of the first computing node, it indicates that the fifth GPU and the sixth GPU are of the same type, and they can communicate through the same communication library. The sixth GPU can be located in the first computing node, or in another computing node of the same type as the first computing node; this embodiment does not limit this.

[0082] The above communication process is the same as the process in the above embodiment where the first computing node communicates with the same type of sub-communication group through the same type of communication library, and will not be repeated here.

[0083] In this embodiment, communication group information is created based on global hardware information, thereby accurately distinguishing between similar communication libraries and hybrid communication libraries. For similar communication libraries, communication can be directly performed through the similar communication library. For hybrid communication libraries, either similar communication libraries or general communication libraries can be accurately selected. Therefore, this embodiment can adaptively select communication libraries, improving flexibility.

[0084] Based on the above embodiments, in a collective communication task, at least two GPUs need to perform collective communication operations. The required collective communication operations vary depending on the type of collective communication task. For example, collective communication operations may include: Reduce operation, AllReduce operation, Broadcast operation, Scatter operation, Gather operation, AllGather operation, etc. This application embodiment does not limit the specific operations in this regard.

[0085] The Reduce operation reduces data from multiple GPUs to a single GPU. This reduction operation can be a summation operation, a product operation, a maximum value operation, a minimum value operation, etc. The AllReduce operation performs the same reduction operation on all GPUs in the communication group and then sends the reduced data to all GPUs. The Broadcast operation broadcasts data from one GPU to other GPUs. The Scatter operation slices data from one GPU and sends the slices to other GPUs. The Gather operation collects data from multiple GPUs to a single GPU. The AllGather operation collects data from multiple GPUs to a single GPU and then distributes the collected data back to each GPU.

[0086] In one possible implementation, the third compute node is of a different type than the first compute node. The GPUs in the third compute node and the GPUs in the first compute node form a hybrid communication group. The GPUs belonging to the first compute node form a sub-communication group of the same type within the first compute node, and the GPUs belonging to the third compute node form a general sub-communication group within the first compute node. The first compute node, in response to a collective communication task for the hybrid communication group, uses a similar communication library to perform collective communication operations matching the collective communication task between GPUs belonging to the same sub-communication group within the first compute node, obtaining first collective communication data. The first compute node is also used to send the first collective communication data to the third compute node via a general communication library. The GPUs in the third compute node belonging to the general sub-communication group of the first compute node then perform collective communication operations matching the collective communication task based on the first collective communication data, obtaining second collective communication data.

[0087] The processing of aggregate communication tasks in a hybrid communication group requires the participation of at least two types of GPUs in the hybrid communication group. Since the hybrid communication group includes homogeneous sub-communication groups and general sub-communication groups, the first computing node needs to adaptively determine the communication library to be used based on the type of GPU that is communicating.

[0088] For example, after receiving the first set of communication data, one GPU in the first computing node communicates point-to-point with one GPU in the third computing node through a general communication library, sending the first set of communication data to that GPU in the third computing node. The GPU in the third computing node that receives the first set of communication data can then broadcast it to the other GPUs in the third computing node. Afterwards, each GPU that received the first set of communication data can perform set communication operations to obtain the second set of communication data.

[0089] For example, after the third computing node obtains the second set of communication data, it can also send the second set of communication data to the first computing node through a general communication library. The second set of communication data is the final processing result of the set communication task.

[0090] Referring to Figure 9, taking Node 1 as the first computing node and Node 2 as the third computing node as an example, Node 1 includes a first type of GPU and uses a first communication library, while Node 2 includes a second type of GPU and uses a second communication library. The aggregate communication task is an AllReduce task, which instructs the summation operation of the data of each GPU in the hybrid communication group. From the perspective of Node 1, in response to the AllReduce task for the hybrid communication group, Node 1 determines that the hybrid communication group includes a similar sub-communication group of Node 1 and a general sub-communication group of Node 2. The similar sub-communication group includes 8 GPUs on Node 1, while the general sub-communication group includes 8 GPUs on Node 2. Therefore, the AllReduce operation is first performed among the 8 GPUs on Node 1 through the first communication library to sum the data of the 8 GPUs. The resulting first aggregate communication data is the sum of the data of the 8 GPUs. Afterwards, Node 1 sends the first set of communication data to Node 2 through the general communication library. Based on the first set of communication data, the eight GPUs in Node 2 can perform AllReduce operations among the eight GPUs through the second communication library, thereby continuing to perform summation operations based on the first set of communication data. The resulting second set of communication data is the sum of the data of the 16 GPUs in the hybrid communication group.

[0091] Additionally, node 2 can send second set communication data to node 1 via a general communication library. The GPU on node 1 that receives the second set communication data then broadcasts it to other GPUs in node 1, ensuring that each GPU receives the second set communication data. The communication process via the general communication library is similar to the process shown in Figure 7 above and will not be repeated here.

[0092] It should be noted that the above embodiments involve communication between GPUs on the same computing node before communication with GPUs on another computing node. In another embodiment, if the number of RoCE V2 network cards on the two computing nodes involved is equal, the GPUs in the hybrid communication group can communicate point-to-point independently. This allows each GPU to obtain data from each of the other GPUs in the hybrid communication group after communicating with them, enabling the summation of the data from all GPUs in the hybrid communication group. However, if the number of RoCE V2 network cards on the two computing nodes involved is unequal, communication between GPUs on the first computing node and GPUs on the third computing node can result in different GPUs sharing the same RoCE V2 network card, leading to communication lag and affecting efficiency. Therefore, in this case, communication between GPUs on the same computing node can be completed first, and then the resulting aggregated communication data can be used to communicate with another computing node.

[0093] This application provides a scheme for processing aggregated communication tasks. After communication is completed between GPUs on the same computing node, the obtained aggregated communication data is then used to communicate with another computing node through a general communication library, avoiding communication lag and improving communication efficiency.

[0094] In another possible implementation, the fourth compute node is of the same type as the first compute node, and the GPUs in the fourth compute node and the GPUs in the first compute node constitute a similar communication group for the first compute node. The first compute node, in response to a collective communication task for the similar communication group, uses a similar communication library to perform collective communication operations between GPUs belonging to that similar communication group within the first compute node, matching the collective communication task, to obtain third collective communication data. The first compute node is also used to send the third collective communication data to the fourth compute node through the similar communication library, and the GPUs belonging to that similar communication group in the fourth compute node perform collective communication operations based on the third collective communication data, matching the collective communication task, to obtain fourth collective communication data.

[0095] For example, after receiving the third set communication data, one GPU in the first computing node communicates point-to-point with one GPU in the fourth computing node through a similar communication library, sending the third set communication data to that GPU in the fourth computing node. The GPU in the fourth computing node that receives the third set communication data can then broadcast it to the other GPUs in the fourth computing node. Afterwards, each GPU that received the third set communication data can perform set communication operations to obtain the fourth set communication data.

[0096] For example, after the fourth computing node obtains the fourth set of communication data, it can also send the fourth set of communication data to the first computing node through a similar communication library. This fourth set of communication data is the final processing result of the set communication task.

[0097] Referring to Figure 10, taking Node 0 as the first computing node and Node 1 as the fourth computing node as an example, both Node 0 and Node 1 are first-type chips using the first communication library. The set communication task is an AllReduce task, which instructs the summation operation of the data of each GPU in the communication group. From the perspective of Node 0, in response to the AllReduce task for the same type of communication group, Node 0 first performs an AllReduce operation among the 8 GPUs on Node 0 through the first communication library, thereby summing the data of the 8 GPUs. The resulting third set communication data is the sum of the data of the 8 GPUs. Afterwards, Node 0 sends the third set communication data to Node 1 through the first communication library. The 8 GPUs in Node 1 can then perform an AllReduce operation among themselves based on this third set communication data, thereby continuing the summation operation based on the third set communication data. The resulting fourth set communication data is the sum of the data of the 16 GPUs in this same type of communication group.

[0098] In addition, node 1 can also send fourth set communication data to node 0 through the first communication library. The GPU on node 0 that receives the fourth set communication data will broadcast the fourth set communication data to other GPUs in node 0, so that each GPU can obtain the fourth set communication data.

[0099] Referring to Figure 11, taking Node 2 as the first computing node and Node 3 as the fourth computing node as an example, both Node 2 and Node 3 are second-type chips. The aggregate communication task is an AllReduce task, which instructs the summation operation of the data of each GPU in the communication group. From the perspective of Node 2, in response to the AllReduce task for the same type of communication group, Node 2 first performs an AllReduce operation among the 8 GPUs on Node 2 through the second communication library, thereby summing the data of the 8 GPUs. The resulting third aggregate communication data is the sum of the data of the 8 GPUs. Then, Node 2 sends the third aggregate communication data to Node 3 through the second communication library. The 8 GPUs in Node 3 can then perform an AllReduce operation among themselves based on this third aggregate communication data, thereby continuing the summation operation based on the third aggregate communication data. The resulting fourth aggregate communication data is the sum of the data of the 16 GPUs in this same type of communication group.

[0100] In addition, node 3 can also send fourth set communication data to node 2 through the second communication library. The GPU on node 2 that receives the fourth set communication data will broadcast the fourth set communication data to other GPUs in node 2, so that each GPU can obtain the fourth set communication data.

[0101] Building upon the above embodiments, this application also provides a load balancing scheme. This is because different types of computing nodes have varying hardware configurations and processing efficiencies, resulting in different computing powers and processing times even with the same load configuration. If the loads of different types of computing nodes in a hybrid GPU cluster are unbalanced, the cluster will be hampered by a bottleneck effect, significantly reducing its processing efficiency. Therefore, to improve processing efficiency, in the hybrid GPU cluster provided by this application, the ratio of the computing power of any two different types of computing nodes is equal to the ratio of their loads. This ensures load balancing across all computing nodes in the hybrid GPU cluster, avoiding bottleneck effects and maximizing the overall processing efficiency of the hybrid GPU cluster.

[0102] The computing power ratio of any two different types of computing nodes can be the reciprocal of the ratio of the time spent by these two types of computing nodes in processing the same amount of tasks. Alternatively, if the two types of computing nodes contain the same number of GPUs, the computing power ratio of these two types of computing nodes can be the ratio of the computing power of a single GPU on these two types of computing nodes. For example, if the theoretical computing power ratio of the first type of GPU to the second type of GPU is 1:1, then the computing power ratio of the first type of computing node to the second type of computing node is 1:1.

[0103] For example, considering that the theoretical computing power of GPUs may have errors, an offline approach can also be adopted to run the same amount of tasks on the first type of GPU and the second type of GPU respectively, and to count the running time consumed by the first type of GPU and the second type of GPU. The reciprocal of the ratio of the running time of the first type of GPU and the second type of GPU is determined as the computing power ratio of the first type of GPU and the second type of GPU.

[0104] The load of a compute node represents the amount of work that the compute node needs to complete, and can be allocated to compute nodes by a hybrid GPU cluster. For example, when performing a neural network model training task through a hybrid GPU cluster, the neural network model includes multiple model layers. During the training task, the model parameters in each model layer need to be trained. Therefore, the load of each compute node can be represented by the number of model layers allocated to each compute node.

[0105] In a hybrid GPU cluster, the load needs to be allocated to each computing node according to the computing power ratio of different types of computing nodes and the total workload, so as to ensure that the computing power ratio and load ratio of different computing nodes are equal.

[0106] For example, the computing power of the first type of GPU is cc. n The computing power of the second type of GPU is cc. a The total number of layers in a neural network model is Layer. total Then the following formula is satisfied:

[0107] Among them, L n The number of model layers allocated to the first type of computing nodes, L a The number of model layers allocated to the second type of computing nodes.

[0108] Referring to Figure 12, the neural network model comprises model layers 0-15, totaling 16. With a computing power ratio of 1:1 between the first and second types of GPUs, each node (nodes 0, 1, 2, and 3) is allocated 4 model layers. Therefore, node 0 is assigned to model layers 0-3, node 1 to model layers 4-7, node 2 to model layers 8-11, and node 3 to model layers 12-15. Referring to Figure 13, the neural network model comprises model layers 0-17, totaling 18. With a computing power ratio of 2:1 between the first and second types of GPUs, nodes 0 and 1 are allocated 6 model layers each, and nodes 2 and 3 are allocated 3 model layers each. Therefore, node 0 is assigned to model layers 0-5, node 1 to model layers 6-11, node 2 to model layers 12-14, and node 3 to model layers 15-17.

[0109] Based on the above embodiments, Figure 14 is a flowchart of a communication method provided in this application embodiment, which is executed by a first computing node. The hybrid GPU cluster includes multiple computing nodes, each including at least one graphics processing unit (GPU); wherein the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in the different types of computing nodes are different. The hybrid GPU cluster creates at least one communication group, which refers to a communication set composed of multiple GPUs. The first computing node is any computing node in the hybrid GPU cluster. Referring to Figure 14, the method includes the following steps.

[0110] 1401. The first compute node obtains global hardware information, which includes the hardware information of each compute node in the hybrid GPU cluster, including the type of GPU in the compute node; based on the global hardware information, it creates communication group information for the first compute node, which is used to indicate the hybrid communication group in the hybrid GPU cluster. The hybrid communication group includes similar sub-communication groups and general sub-communication groups. Similar sub-communication groups include GPUs that conform to the type of the first compute node, and general sub-communication groups include GPUs that do not conform to the type of the first compute node.

[0111] 1402. The first computing node communicates with similar sub-communication groups through similar communication libraries based on communication group information, and communicates with general sub-communication groups through a general communication library. The general communication library refers to a communication library that can enable communication between different types of GPUs.

[0112] In one possible implementation, the first computing node communicates with similar sub-communication groups through similar communication libraries and with general sub-communication groups through a general communication library, based on communication group information. This includes: when the first computing node determines, based on the communication group information, that the communication target of the first GPU is the second GPU in the general sub-communication group, it sends communication data of the first GPU to the first CPU in the first computing node; the first CPU then sends communication data to the second CPU in the second computing node through the general communication library. Here, the first GPU is any GPU in the first computing node, the second computing node is the computing node where the second GPU is located, and the second CPU is used to receive communication data and send communication data to the second GPU through the general communication library.

[0113] In one possible implementation, the first computing node communicates with a sub-communication group of the same type through a similar communication library, and with a general sub-communication group through a general communication library, based on communication group information. This includes: when the first computing node determines that the communication target of the third GPU is the fourth GPU in the sub-communication group of the same type based on the communication group information, it sends the communication data of the third GPU to the fourth GPU through the similar communication library; wherein the third GPU is any GPU in the first computing node.

[0114] In one possible implementation, communication group information is created based on global hardware information, including: when it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes GPUs conforming to the first compute node type and also includes GPUs that do not conform to the first compute node type, first communication group information corresponding to the communication group is created; wherein, the first communication group information is used to indicate that the communication group is a hybrid communication group, the GPUs conforming to the first compute node type in the communication group constitute similar sub-communication groups, and the GPUs that do not conform to the first compute node type in the communication group constitute general sub-communication groups.

[0115] In one possible implementation, the communication group information is also used to indicate similar communication groups in a hybrid GPU cluster, the similar communication groups including GPUs that conform to the first compute node type; the method further includes: communicating with similar communication groups through similar communication libraries based on the communication group information.

[0116] In one possible implementation, communication with a similar communication group via a similar communication library is performed based on communication group information, including: if the communication target of the fifth GPU is determined to be the sixth GPU in the same communication group based on the communication group information, the communication data of the fifth GPU is sent to the sixth GPU via the similar communication library; wherein the fifth GPU is any GPU in the first computing node.

[0117] In one possible implementation, communication group information is created based on global hardware information, including: if it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes GPUs that conform to the first compute node type, but does not include GPUs that do not conform to the first compute node type, then a second communication group information corresponding to the communication group is created; wherein the second communication group information is used to indicate that the communication group is a communication group of the same type.

[0118] In one possible implementation, the first computing node communicates with similar sub-communication groups through a similar communication library and with general sub-communication groups through a general communication library based on communication group information. This includes: in response to a collective communication task for a hybrid communication group, performing a collective communication operation matching the collective communication task between GPUs belonging to the same sub-communication group in the first computing node through the similar communication library to obtain first collective communication data; and sending the first collective communication data to a third computing node through the general communication library, whereby GPUs belonging to the general sub-communication group in the third computing node perform a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

[0119] In one possible implementation, in a hybrid GPU cluster, the ratio of computing power between any two different types of computing nodes is equal to the ratio of load between any two different types of computing nodes.

[0120] This application provides a scheme for adaptive communication in a hybrid GPU cluster. Each computing node in the hybrid GPU cluster creates communication group information based on the hardware information of each GPU, thereby dividing the communication groups in the hybrid GPU cluster according to GPU type. Thus, the hybrid communication groups of computing nodes can be determined, and the hybrid communication groups are divided into similar sub-communication groups and general sub-communication groups. The appropriate communication library is selected adaptively according to the type of the sub-communication group. Communication with similar sub-communication groups is carried out through similar communication libraries, and communication with general sub-communication groups is carried out through general communication libraries. This enables communication between GPUs of the same type as well as between GPUs of different types, thereby breaking down the communication barriers between different types of GPUs. It can effectively utilize different types of GPUs in the hybrid GPU cluster to process tasks and improve processing efficiency.

[0121] Based on the above embodiments, this application also provides a chip configured in a hybrid GPU cluster. The hybrid GPU cluster includes multiple chips, each chip including at least one graphics processing unit (GPU). The multiple chips include at least two types of chips, and the types of GPUs in the different types of chips are different. The hybrid GPU cluster creates at least one communication group, which refers to a communication set composed of multiple GPUs.

[0122] The chip is used to acquire global hardware information, which includes the hardware information of each chip in the hybrid GPU cluster, including the type of GPU in the chip. Based on the global hardware information, the chip creates communication group information, which indicates the hybrid communication group in the hybrid GPU cluster. The hybrid communication group includes similar sub-communication groups and general sub-communication groups. Similar sub-communication groups include GPUs that match the chip type, and general sub-communication groups include GPUs that do not match the chip type. The chip is also used to communicate with similar sub-communication groups through similar communication libraries and with general sub-communication groups through general communication libraries, which are communication libraries that enable communication between different types of GPUs.

[0123] In one possible implementation, the chip is used to send communication data of the first GPU to the first CPU in the chip when the communication target of the first GPU is determined to be the second GPU in the general sub-communication group based on the communication group information. The first CPU then sends communication data to the second CPU in another chip through the general communication library. Here, the first GPU is any GPU in the chip, the other chip is the chip where the second GPU is located, and the second CPU is used to receive communication data through the general communication library and send communication data to the second GPU.

[0124] In one possible implementation, the chip is used to send communication data of the third GPU to the fourth GPU through a similar communication library when the communication target of the third GPU is determined to be the fourth GPU in the same sub-communication group based on communication group information; wherein the third GPU is any GPU in the chip.

[0125] In one possible implementation, the chip is used to create first communication group information corresponding to a communication group when it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes GPUs that conform to the chip type and also includes GPUs that do not conform to the chip type; wherein the first communication group information is used to indicate that the communication group is a hybrid communication group, in which GPUs that conform to the chip type constitute a sub-communication group of the same type, and in which GPUs that do not conform to the chip type constitute a general sub-communication group.

[0126] In one possible implementation, the communication group information is also used to indicate similar communication groups in a hybrid GPU cluster, which include GPUs that conform to the chip type; the chip is also used to communicate with similar communication groups through similar communication libraries based on the communication group information.

[0127] In one possible implementation, the chip is used to send communication data of the fifth GPU to the sixth GPU through a similar communication library when the communication target of the fifth GPU is determined to be the sixth GPU in the same communication group based on communication group information; wherein the fifth GPU is any GPU in the chip.

[0128] In one possible implementation, the chip is further configured to create second communication group information corresponding to any communication group in a hybrid GPU cluster, provided that, based on global hardware information, any communication group includes GPUs that conform to the chip type but does not include GPUs that do not conform to the chip type; wherein the second communication group information is used to indicate that the communication group is a communication group of the same type.

[0129] In one possible implementation, the chip is used to respond to a collective communication task for a hybrid communication group, and through a similar communication library, to perform a collective communication operation matching the collective communication task between GPUs belonging to the same sub-communication group in the chip to obtain first collective communication data; the chip is also used to send the first collective communication data to another chip through a general communication library, and the GPUs belonging to the general sub-communication group in the other chip perform a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

[0130] In one possible implementation, in a hybrid GPU cluster, the computing power ratio of any two different types of chips is equal to the load ratio of any two different types of chips.

[0131] Figure 15 is a schematic diagram of a communication device provided in an embodiment of this application. The device is configured on a first computing node in a hybrid GPU cluster. The hybrid GPU cluster includes multiple computing nodes, each computing node including at least one graphics processing unit (GPU). The multiple computing nodes include at least two types of computing nodes, with different types of GPUs in the different types of computing nodes. The hybrid GPU cluster creates at least one communication group, which refers to a communication set composed of multiple GPUs.

[0132] Referring to Figure 15, the device includes:

[0133] Information processing module 1501 is used to acquire global hardware information, which includes hardware information of each computing node in the hybrid GPU cluster, including the type of GPU in the computing node; based on the global hardware information, it creates communication group information for the first computing node, which is used to indicate the hybrid communication group in the hybrid GPU cluster. The hybrid communication group includes similar sub-communication groups and general sub-communication groups. Similar sub-communication groups include GPUs that conform to the type of the first computing node, and general sub-communication groups include GPUs that do not conform to the type of the first computing node.

[0134] The communication module 1502 is used to communicate with similar sub-communication groups through similar communication libraries based on communication group information, and to communicate with general sub-communication groups through a general communication library. The general communication library refers to a communication library that can communicate between different types of GPUs; wherein, the first computing node is any computing node in the hybrid GPU cluster.

[0135] In one possible implementation, the communication module 1502 is used to send communication data of the first GPU to the first CPU in the first computing node when the communication target of the first GPU is determined to be the second GPU in the general sub-communication group based on the communication group information. The first CPU then sends communication data to the second CPU in the second computing node through the general communication library. Here, the first GPU is any GPU in the first computing node, the second computing node is the computing node where the second GPU is located, and the second CPU is used to receive communication data and send communication data to the second GPU through the general communication library.

[0136] In one possible implementation, the communication module 1502 is used to send communication data of the third GPU to the fourth GPU through a similar communication library when the communication target of the third GPU is determined to be the fourth GPU in the same sub-communication group based on the communication group information; wherein the third GPU is any GPU in the first computing node.

[0137] In one possible implementation, the information processing module 1501 is used to create first communication group information corresponding to the communication group when it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes GPUs that conform to the first computing node type and also includes GPUs that do not conform to the first computing node type; wherein, the first communication group information is used to indicate that the communication group is a hybrid communication group, the GPUs that conform to the first computing node type in the communication group constitute the same type of sub-communication group, and the GPUs that do not conform to the first computing node type in the communication group constitute the general sub-communication group.

[0138] In one possible implementation, the communication group information is also used to indicate similar communication groups in a hybrid GPU cluster, the similar communication groups including GPUs that conform to the first compute node type; the communication module 1502 is also used to communicate with similar communication groups through similar communication libraries based on the communication group information.

[0139] In one possible implementation, the communication module 1502 is used to send communication data of the fifth GPU to the sixth GPU through a similar communication library when the communication target of the fifth GPU is determined to be the sixth GPU in the same communication group based on the communication group information; wherein the fifth GPU is any GPU in the first computing node.

[0140] In one possible implementation, the information processing module 1501 is further configured to create second communication group information corresponding to the communication group when it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes GPUs that conform to the first computing node type and does not include GPUs that do not conform to the first computing node type; wherein the second communication group information is used to indicate that the communication group is a communication group of the same type.

[0141] In one possible implementation, the communication module 1502 is configured to, in response to a collective communication task for a hybrid communication group, perform a collective communication operation matching the collective communication task between GPUs belonging to the same sub-communication group in the first computing node using a similar communication library, to obtain first collective communication data; the communication module 1502 is further configured to send the first collective communication data to a third computing node using a general communication library, and the GPUs belonging to the general sub-communication group in the third computing node perform a collective communication operation matching the collective communication task based on the first collective communication data, to obtain second collective communication data.

[0142] In one possible implementation, in a hybrid GPU cluster, the ratio of computing power between any two different types of computing nodes is equal to the ratio of load between any two different types of computing nodes.

[0143] It should be noted that the communication device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the communication device and communication method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0144] This application also provides a computer device including a processor and a memory. The memory stores at least one computer program, which is loaded and executed by the processor to perform the operations described in the communication method of the above embodiments. The communication method provided in this application is executed by the computer device.

[0145] Optionally, the computer device is provided as a terminal. Figure 16 shows a schematic diagram of the structure of a terminal 1600 provided in an exemplary embodiment of this application. The terminal 1600 includes a processor 1601 and a memory 1602. The processor 1601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. In some embodiments, the processor 1601 may also include an AI (Artificial Intelligence) processor for processing computational operations related to machine learning. The memory 1602 may include one or more computer-readable storage media, which may be non-transitory. In some embodiments, the non-transitory computer-readable storage media in the memory 1602 is used to store at least one computer program, which is used by the processor 1601 to implement the communication method provided in the method embodiments of this application.

[0146] In some embodiments, the terminal 1600 may also optionally include: a peripheral device interface 1603 and at least one peripheral device. The processor 1601, memory 1602, and peripheral device interface 1603 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1603 via a bus, signal line, or circuit board. Optionally, the peripheral device includes at least one of: a radio frequency circuit 1604, a display screen 1605, a camera assembly 1606, and a power supply 1607.

[0147] Those skilled in the art will understand that the structure shown in FIG16 does not constitute a limitation on the terminal 1600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0148] Optionally, the computer device is provided as a server. Figure 17 is a schematic diagram of the structure of a server provided in an embodiment of this application. The server 1700 may vary considerably due to different configurations or accuracy requirements, and may include one or more Central Processing Units (CPUs) 1701 and one or more memories 1702. The memories 1702 store at least one computer program, which is loaded and executed by the processor 1701 to implement the methods provided in the above-described method embodiments. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated here.

[0149] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the operations performed by the communication method of the above embodiments.

[0150] This application also provides a computer program product, including a computer program loaded and executed by a processor to perform the operations performed by the communication method of the above embodiments.

[0151] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0152] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application should be included within the protection scope of the present application.

Claims

1. A hybrid GPU cluster, the hybrid GPU cluster comprising multiple compute nodes, each compute node comprising at least one graphics processing unit (GPU); wherein, The plurality of computing nodes include at least two types of computing nodes, and the types of GPUs in the different types of computing nodes are different; the hybrid GPU cluster creates at least one communication group, and the communication group refers to a communication set composed of multiple GPUs; A first computing node is configured to acquire global hardware information, which includes hardware information for each computing node in the hybrid GPU cluster, including the type of GPU in the computing node. Based on the global hardware information, communication group information for the first computing node is created. The communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster. The hybrid communication group includes a similar sub-communication group and a general sub-communication group. The similar sub-communication group includes GPUs that conform to the type of the first computing node, and the general sub-communication group includes GPUs that do not conform to the type of the first computing node. The first computing node is also configured to communicate with the same type of sub-communication group through the same type of communication library based on the communication group information, and to communicate with the general sub-communication group through a general communication library, wherein the general communication library refers to a communication library that can communicate between different types of GPUs; The first computing node is any computing node in the hybrid GPU cluster.

2. The hybrid GPU cluster according to claim 1, wherein, The first computing node is configured to send communication data of the first GPU to the first CPU in the first computing node when the communication target of the first GPU is determined to be the second GPU in the general sub-communication group based on the communication group information, and the first CPU sends the communication data to the second CPU in the second computing node through the general communication library. Wherein, the first GPU is any GPU in the first computing node, the second computing node is the computing node where the second GPU is located, and the second CPU is used to receive the communication data through the general communication library and send the communication data to the second GPU.

3. The hybrid GPU cluster according to claim 1 or 2, wherein, The first computing node is configured to send the communication data of the third GPU to the fourth GPU through the same type of communication library when the communication target of the third GPU is determined to be the fourth GPU in the same type of sub-communication group based on the communication group information. The third GPU can be any GPU in the first computing node.

4. The hybrid GPU cluster according to any one of claims 1-3, wherein, The first computing node is configured to create first communication group information corresponding to the communication group when, based on the global hardware information, it is determined that any communication group in the hybrid GPU cluster includes GPUs that conform to the type of the first computing node and also includes GPUs that do not conform to the type of the first computing node. The first communication group information is used to indicate that the communication group is a hybrid communication group, in which GPUs that conform to the first computing node type constitute a sub-communication group of the same type, and in which GPUs that do not conform to the first computing node type constitute a general sub-communication group.

5. The hybrid GPU cluster according to any one of claims 1-4, wherein, The communication group information is also used to indicate similar communication groups in the hybrid GPU cluster, the similar communication groups including GPUs that conform to the first compute node type; The first computing node is also configured to communicate with the same type of communication group through the same type of communication library based on the communication group information.

6. The hybrid GPU cluster according to claim 5, wherein, The first computing node is configured to send communication data of the fifth GPU to the sixth GPU through the same type of communication library when the communication target of the fifth GPU is determined to be the sixth GPU in the same type of communication group based on the communication group information. The fifth GPU can be any GPU in the first computing node.

7. The hybrid GPU cluster according to any one of claims 1-6, wherein, The first computing node is further configured to create second communication group information corresponding to the communication group when it is determined, based on the global hardware information, that any communication group in the hybrid GPU cluster includes GPUs that conform to the type of the first computing node and does not include GPUs that do not conform to the type of the first computing node; The second communication group information is used to indicate that the communication group is the same type of communication group.

8. The hybrid GPU cluster according to any one of claims 1-7, wherein, The first computing node is configured to respond to a collective communication task for the hybrid communication group by performing a collective communication operation matching the collective communication task among GPUs belonging to the same sub-communication group within the first computing node through the same type of communication library, thereby obtaining first collective communication data. The first computing node is further configured to send the first set communication data to the third computing node through the general communication library, and the GPU in the third computing node belonging to the general sub-communication group performs a set communication operation matching the set communication task based on the first set communication data to obtain the second set communication data.

9. The hybrid GPU cluster according to any one of claims 1 to 8, wherein, In the hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

10. A communication method, wherein a hybrid GPU cluster comprises multiple compute nodes, each compute node comprising at least one graphics processing unit (GPU); wherein, The plurality of computing nodes includes at least two types of computing nodes, with different types of GPUs in the different types of computing nodes; the hybrid GPU cluster creates at least one communication group, which refers to a communication set composed of multiple GPUs; the method includes: The first compute node acquires global hardware information, which includes hardware information for each compute node in the hybrid GPU cluster, including the type of GPU in the compute node. Based on the global hardware information, it creates communication group information for the first compute node. The communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster. The hybrid communication group includes a similar sub-communication group and a general sub-communication group. The similar sub-communication group includes GPUs that conform to the type of the first compute node, and the general sub-communication group includes GPUs that do not conform to the type of the first compute node. Based on the communication group information, the first computing node communicates with the same type of sub-communication group through the same type of communication library, and communicates with the general sub-communication group through the general communication library. The general communication library refers to a communication library that can communicate between different types of GPUs. The first computing node is any computing node in the hybrid GPU cluster.

11. A chip configured in a hybrid GPU cluster, the hybrid GPU cluster comprising a plurality of chips, each chip comprising at least one graphics processing unit (GPU); wherein, The multiple chips include at least two types of chips, and the types of GPUs in the different types of chips are different; the hybrid GPU cluster creates at least one communication group, which refers to a communication set composed of multiple GPUs; The chip is used to acquire global hardware information, which includes hardware information of each chip in the hybrid GPU cluster, including the type of GPU in the chip; based on the global hardware information, communication group information of the chip is created, which indicates a hybrid communication group in the hybrid GPU cluster, the hybrid communication group including a similar sub-communication group and a general sub-communication group, the similar sub-communication group including GPUs that conform to the chip type, and the general sub-communication group including GPUs that do not conform to the chip type; The chip is also used to communicate with the same type of sub-communication group through the same type of communication library based on the communication group information, and to communicate with the general sub-communication group through a general communication library. The general communication library refers to a communication library that can communicate between different types of GPUs.

12. A communication device configured on a first computing node in a hybrid GPU cluster, the hybrid GPU cluster comprising multiple computing nodes, each computing node comprising at least one graphics processing unit (GPU); wherein, The plurality of computing nodes include at least two types of computing nodes, and the types of GPUs in the different types of computing nodes are different; the hybrid GPU cluster creates at least one communication group, and the communication group refers to a communication set composed of multiple GPUs; The device includes: An information processing module is used to acquire global hardware information, which includes hardware information of each computing node in the hybrid GPU cluster, including the type of GPU in the computing node; based on the global hardware information, to create communication group information for the first computing node, which indicates a hybrid communication group in the hybrid GPU cluster, the hybrid communication group including a similar sub-communication group and a general sub-communication group, the similar sub-communication group including GPUs that conform to the type of the first computing node, and the general sub-communication group including GPUs that do not conform to the type of the first computing node; The communication module is also used to communicate with the same type of sub-communication group through the same type of communication library based on the communication group information, and to communicate with the general sub-communication group through a general communication library. The general communication library refers to a communication library that can communicate between different types of GPUs. The first computing node is any computing node in the hybrid GPU cluster.

13. A computer device comprising a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to perform the operations performed by the communication method of claim 10.

14. A computer-readable storage medium storing at least one computer program, the at least one computer program being loaded and executed by a processor to perform the operations performed by the communication method of claim 10.

15. A computer program product comprising a computer program loaded and executed by a processor to perform the operations performed by the communication method of claim 10.

Images