Hybrid GPU cluster, communication method, chip and device

Abstract

The embodiments of the present application disclose a hybrid GPU cluster, a communication method, a chip and a device, and belong to the field of computer technology. The hybrid GPU cluster includes multiple computing nodes, each computing node includes at least one GPU; the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different. The first computing node is used to obtain global hardware information and create communication group information. The communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster. The hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group; based on the communication group information, the communication is performed with the homogeneous sub-communication group through the homogeneous communication library, and the communication is performed with the general sub-communication group through the general communication library. The embodiments of the present application break the communication barriers between different types of GPUs, and can effectively utilize different types of GPUs in the hybrid GPU cluster to process tasks, thereby improving processing efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a hybrid GPU cluster, a communication method, a chip, and a device. Background Art

[0002] A hybrid GPU (Graphics Processing Unit) cluster is a computer cluster composed of different types of GPUs. A hybrid GPU cluster usually contains GPUs of different models, different architectures, and even from different manufacturers. Different types of GPUs have performance advantages in different aspects. By mixing different types of GPUs, the performance advantages can be complemented to improve the overall performance of the entire cluster.

[0003] However, there are communication barriers between different types of GPUs. How to solve the communication barriers between different types of GPUs has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a hybrid GPU cluster, communication method, chip and device, which break the communication barriers between different types of GPUs, can effectively use different types of GPUs in the hybrid GPU cluster to process tasks, and improve processing efficiency. The technical solution is as follows.

[0005] In one aspect, a hybrid GPU cluster is provided, the hybrid GPU cluster comprising a plurality of computing nodes, each computing node comprising at least one graphics processing unit (GPU); wherein the plurality of computing nodes comprise at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different;

[0006] A first computing node is configured to obtain global hardware information, wherein the global hardware information includes hardware information of each computing node in the hybrid GPU cluster, and the hardware information includes the type of GPU in the computing node; based on the global hardware information, communication group information is created, wherein the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, wherein the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, wherein the homogeneous sub-communication group includes GPUs that conform to the first computing node type, and the general sub-communication group includes GPUs that do not conform to the first computing node type;

[0007] The first computing node is further configured to communicate with the homogeneous sub-communication group through a homogeneous communication library and communicate with the universal sub-communication group through a universal communication library based on the communication group information;

[0008] The first computing node is any computing node in the hybrid GPU cluster.

[0009] On the other hand, a communication method is provided, wherein a hybrid GPU cluster includes a plurality of computing nodes, each computing node includes at least one graphics processing unit (GPU); wherein the plurality of computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different; the method includes:

[0010] The first computing node obtains global hardware information, wherein the global hardware information includes hardware information of each computing node in the hybrid GPU cluster, and the hardware information includes the type of GPU in the computing node; based on the global hardware information, creates communication group information, wherein the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, wherein the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, wherein the homogeneous sub-communication group includes a GPU that conforms to the first computing node type, and the general sub-communication group includes a GPU that does not conform to the first computing node type;

[0011] The first computing node communicates with the same sub-communication group through a same-type communication library and communicates with the general sub-communication group through a general communication library based on the communication group information;

[0012] The first computing node is any computing node in the hybrid GPU cluster.

[0013] In a possible implementation, the first computing node communicates with the homogeneous sub-communication group through a homogeneous communication library and communicates with the universal sub-communication group through a universal communication library based on the communication group information, including:

[0014] When the first computing node determines, based on the communication group information, that the communication target of the first GPU is the second GPU in the general sub-communication group, the first computing node sends the communication data of the first GPU to the first CPU in the first computing node, and the first CPU sends the communication data to the second CPU in the second computing node through the general communication library;

[0015] The first GPU is any GPU in the first computing node, the second computing node is a computing node where the second GPU is located, and the second CPU is used to receive the communication data through the universal communication library and send the communication data to the second GPU.

[0016] In a possible implementation, the first computing node communicates with the homogeneous sub-communication group through a homogeneous communication library and communicates with the universal sub-communication group through a universal communication library based on the communication group information, including:

[0017] When the first computing node determines, based on the communication group information, that the communication target of the third GPU is a fourth GPU in the homogeneous sub-communication group, sending the communication data of the third GPU to the fourth GPU through the homogeneous communication library;

[0018] The third GPU is any GPU in the first computing node.

[0019] In a possible implementation, the creating the communication group information based on the global hardware information includes:

[0020] In a case where it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and includes a GPU that does not conform to the first computing node type, creating first communication group information corresponding to the communication group;

[0021] The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the first computing node type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the first computing node type constitute a general sub-communication group.

[0022] In a possible implementation, the communication group information is further used to indicate a homogeneous communication group in the hybrid GPU cluster, where the homogeneous communication group includes GPUs that conform to the first computing node type; and the method further includes:

[0023] Based on the communication group information, communication is performed with the same communication group through the same communication library.

[0024] In a possible implementation, the communicating with the same communication group through the same communication library based on the communication group information includes:

[0025] When it is determined based on the communication group information that the communication target of the fifth GPU is a sixth GPU in the homogeneous communication group, sending the communication data of the fifth GPU to the sixth GPU through the homogeneous communication library;

[0026] The fifth GPU is any GPU in the first computing node.

[0027] In a possible implementation, the creating the communication group information based on the global hardware information includes:

[0028] When it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type, creating second communication group information corresponding to the communication group;

[0029] The second communication group information is used to indicate that the communication group is the same type of communication group.

[0030] In a possible implementation, the first computing node communicates with the homogeneous sub-communication group through a homogeneous communication library and communicates with the universal sub-communication group through a universal communication library based on the communication group information, including:

[0031] In response to the collective communication task for the hybrid communication group, performing a collective communication operation matching the collective communication task between the GPUs belonging to the homogeneous sub-communication group in the first computing node through the homogeneous communication library to obtain first collective communication data;

[0032] The first collective communication data is sent to a third computing node through the general communication library, and a GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

[0033] In a possible implementation, in the hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

[0034] On the other hand, a chip is provided, the chip being configured in a hybrid GPU cluster, the hybrid GPU cluster comprising a plurality of chips, each chip comprising at least one graphics processing unit (GPU); wherein the plurality of chips comprises at least two types of chips, and the types of GPUs in the different types of chips are different;

[0035] The chip is used to obtain global hardware information, the global hardware information includes hardware information of each chip in the hybrid GPU cluster, and the hardware information includes the type of GPU in the chip; based on the global hardware information, communication group information is created, the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, the homogeneous sub-communication group includes GPUs that conform to the chip type, and the general sub-communication group includes GPUs that do not conform to the chip type;

[0036] The chip is also used to communicate with the same sub-communication group through a same-type communication library and communicate with the general sub-communication group through a general communication library based on the communication group information.

[0037] In a possible implementation, the chip is used to send communication data of the first GPU to a first CPU in the chip when it is determined based on the communication group information that the communication target of the first GPU is the second GPU in the general sub-communication group, and the first CPU sends the communication data to a second CPU in another chip through the general communication library;

[0038] The first GPU is any GPU in the chips, the other chip is the chip where the second GPU is located, and the second CPU is used to receive the communication data through the universal communication library and send the communication data to the second GPU.

[0039] In a possible implementation, the chip is used to send the communication data of the third GPU to the fourth GPU through the homogeneous communication library when it is determined based on the communication group information that the communication target of the third GPU is the fourth GPU in the homogeneous sub-communication group;

[0040] The third GPU is any GPU in the chip.

[0041] In a possible implementation, the chip is used to create first communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the chip type and includes a GPU that does not conform to the chip type;

[0042] The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the chip type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the chip type constitute a general sub-communication group.

[0043] In a possible implementation, the communication group information is further used to indicate a homogeneous communication group in the hybrid GPU cluster, where the homogeneous communication group includes GPUs that conform to the chip type;

[0044] The chip is also used to communicate with the same communication group through the same communication library based on the communication group information.

[0045] In a possible implementation, the chip is used to send the communication data of the fifth GPU to the sixth GPU through the homogeneous communication library when it is determined based on the communication group information that the communication target of the fifth GPU is the sixth GPU in the homogeneous communication group;

[0046] The fifth GPU is any GPU in the chip.

[0047] In a possible implementation, the chip is further used to create second communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the chip type and does not include a GPU that does not conform to the chip type;

[0048] The second communication group information is used to indicate that the communication group is the same type of communication group.

[0049] In a possible implementation, the chip is used to respond to the collective communication task for the hybrid communication group, and to perform, through the homogeneous communication library, a collective communication operation matching the collective communication task between GPUs belonging to the homogeneous sub-communication group in the chip to obtain first collective communication data;

[0050] The chip is also used to send the first collective communication data to another chip through the general communication library, and the GPU belonging to the general sub-communication group in the other chip performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

[0051] In a possible implementation, in the hybrid GPU cluster, the computing power ratio of any two different types of chips is equal to the load ratio of any two different types of chips.

[0052] On the other hand, a communication device is provided, the device being configured at a first computing node in a hybrid GPU cluster, the hybrid GPU cluster comprising a plurality of computing nodes, each computing node comprising at least one graphics processing unit (GPU); wherein the plurality of computing nodes comprises at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different;

[0053] The device comprises:

[0054] An information processing module, configured to obtain global hardware information, wherein the global hardware information includes hardware information of each computing node in the hybrid GPU cluster, wherein the hardware information includes the type of GPU in the computing node; based on the global hardware information, create communication group information, wherein the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, wherein the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, wherein the homogeneous sub-communication group includes a GPU that conforms to the first computing node type, and the general sub-communication group includes a GPU that does not conform to the first computing node type;

[0055] The communication module is further configured to communicate with the same type of sub-communication group through a same type of communication library and communicate with the general sub-communication group through a general communication library based on the communication group information;

[0056] The first computing node is any computing node in the hybrid GPU cluster.

[0057] In a possible implementation, the communication module is configured to, when it is determined based on the communication group information that the communication target of the first GPU is the second GPU in the general sub-communication group, send the communication data of the first GPU to the first CPU in the first computing node, and the first CPU sends the communication data to the second CPU in the second computing node through the general communication library;

[0058] The first GPU is any GPU in the first computing node, the second computing node is a computing node where the second GPU is located, and the second CPU is used to receive the communication data through the universal communication library and send the communication data to the second GPU.

[0059] In a possible implementation, the communication module is configured to send the communication data of the third GPU to the fourth GPU through the homogeneous communication library when it is determined based on the communication group information that the communication target of the third GPU is a fourth GPU in the homogeneous sub-communication group;

[0060] The third GPU is any GPU in the first computing node.

[0061] In a possible implementation, the information processing module is used to create first communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and includes a GPU that does not conform to the first computing node type;

[0062] The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the first computing node type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the first computing node type constitute a general sub-communication group.

[0063] In a possible implementation, the communication group information is further used to indicate a homogeneous communication group in the hybrid GPU cluster, where the homogeneous communication group includes GPUs that conform to the first computing node type;

[0064] The communication module is further used to communicate with the same communication group through the same communication library based on the communication group information.

[0065] In a possible implementation, the communication module is configured to send the communication data of the fifth GPU to the sixth GPU through the homogeneous communication library when it is determined based on the communication group information that the communication target of the fifth GPU is the sixth GPU in the homogeneous communication group;

[0066] The fifth GPU is any GPU in the first computing node.

[0067] In a possible implementation, the information processing module is further configured to, when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type, create second communication group information corresponding to the communication group;

[0068] The second communication group information is used to indicate that the communication group is the same type of communication group.

[0069] In a possible implementation, the communication module is configured to, in response to a collective communication task for the hybrid communication group, perform a collective communication operation matching the collective communication task between GPUs belonging to the homogeneous sub-communication group in the first computing node through the homogeneous communication library to obtain first collective communication data;

[0070] The communication module is also used to send the first collective communication data to the third computing node through the general communication library, and the GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

[0071] In a possible implementation, in the hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

[0072] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the communication method described in the above aspects.

[0073] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the communication method described in the above aspects.

[0074] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is loaded and executed by a processor to implement the operations performed by the communication method as described in the above aspects.

[0075] An embodiment of the present application provides a solution for adaptive communication in a hybrid GPU cluster. Any computing node in the hybrid GPU cluster creates communication group information based on the hardware information of each GPU, thereby dividing the communication groups in the hybrid GPU cluster according to the GPU type. Therefore, the hybrid communication group of the computing node can be determined, and the hybrid communication group is divided into the same sub-communication group and the general sub-communication group, so as to adaptively select the appropriate communication library according to the type of the sub-communication group, communicate with the same sub-communication group through the same communication library, and communicate with the general sub-communication group through the general communication library, which can realize communication between GPUs of the same type and communication between GPUs of different types, thereby breaking the communication barriers between different types of GPUs, and being able to effectively utilize different types of GPUs in the hybrid GPU cluster to process tasks, thereby improving processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0077] Figure 1 It is a schematic diagram of a GPU cluster provided by the related technology.

[0078] Figure 2 It is a schematic diagram of another GPU cluster provided by the related technology.

[0079] Figure 3 is a schematic diagram of a hybrid GPU cluster provided in an embodiment of the present application.

[0080] Figure 4 It is a schematic diagram of a communication group and a communication library of a first type of computing node provided in an embodiment of the present application.

[0081] Figure 5 It is a schematic diagram of a communication group and a communication library of a second type of computing node provided in an embodiment of the present application.

[0082] Figure 6 This is a schematic diagram of a communication group in a hybrid GPU cluster provided in an embodiment of the present application.

[0083] Figure 7It is a flow chart of communicating through a universal communication library provided in an embodiment of the present application.

[0084] Figure 8 This is a schematic diagram of communication between GPUs of the same type provided in an embodiment of the present application.

[0085] Fig. 9 It is a flow chart of collective communication between node 1 and node 2 provided in an embodiment of the present application.

[0086] Fig.10 It is a flow chart of collective communication between node 0 and node 1 provided in an embodiment of the present application.

[0087] Fig.11 It is a flow chart of collective communication between node 2 and node 3 provided in an embodiment of the present application.

[0088] Fig.12 This is a schematic diagram of a model layer allocated to each computing node in a hybrid GPU cluster provided in an embodiment of the present application.

[0089] Fig.13 This is a schematic diagram of a model layer allocated to each computing node in another hybrid GPU cluster provided in an embodiment of the present application.

[0090] Fig.14 It is a flow chart of a communication method provided in an embodiment of the present application.

[0091] Fig.15 It is a structural diagram of a communication device provided in an embodiment of the present application.

[0092] Fig.16 It is a structural diagram of a terminal provided in an embodiment of the present application.

[0093] Fig.17 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0095] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of this application, a first computing node may be referred to as a second computing node, and similarly, a second computing node may be referred to as a first computing node.

[0096] Among them, at least two refers to two or more than two, for example, the at least two computing nodes can be two computing nodes, three computing nodes, or any integer number of computing nodes greater than or equal to two. Each refers to each of the at least two, for example, each computing node refers to each computing node of the at least two computing nodes, and if the at least two computing nodes are three computing nodes, each computing node refers to each computing node of the three computing nodes.

[0097] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all fully authorized by users or relevant parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0098] First, the concepts involved in the embodiments of the present application are explained as follows.

[0099] 1. NCCL (NVIDIA Collective Communication Library): is a communication library for communication between GPUs that can work between GPUs produced by NVIDIA.

[0100] 2. HCCL (Huawei Collective Communication Library): is another communication library for communication between GPUs, which can work between GPUs produced by Ascend.

[0101] 3. GLOO: It is another communication library for communication between GPUs. It is universal and can work between GPUs produced by different manufacturers.

[0102] 4. RDMA (Remote Direct Memory Access): is a technology developed to solve the delay of server-side data processing in network transmission. RDMA transfers data directly to the computer's storage area through the network, quickly moving data from one system to the remote system memory without causing any impact on the operating system.

[0103] RoCE V2 (RDMA over Converged Ethernet V2, Ethernet converged RDMA technology version 2): is an RDMA protocol designed for low-latency, high-throughput data transmission in Ethernet environments. It implements a direct memory access mechanism between systems, minimizes CPU involvement and reduces communication latency.

[0104] 5. GDR (GPU Direct RDMA, graphics processing unit communicates directly with RDMA): A technology for direct data access and communication between GPU and RDMA network devices. It bypasses the host memory and CPU and directly transfers data between GPU and RDMA network devices.

[0105] 6. H2D (Host to Device): Data is moved from CPU memory to GPU high-bandwidth memory.

[0106] 7. D2H (Device to Host): Data is moved from GPU high-bandwidth memory to CPU memory.

[0107] 8. Communication group: a communication set consisting of multiple GPUs.

[0108] Figure 1 This is a schematic diagram of a GPU cluster provided by related technology, see Figure 1 , the GPU cluster consists of multiple computing nodes ( Figure 1 Taking 4 computing nodes as an example), each computing node includes at least one GPU ( Figure 1 Taking 8 GPUs as an example), and the GPUs in the GPU cluster are all of the same type, so the computing nodes in the GPU cluster are all of the same type.

[0109] In order to facilitate distinction in the embodiments of the present application, Figure 1 The GPUs in the GPU cluster shown are called first-type GPUs, and the communication library used by the first-type GPUs is called a first communication library.

[0110] Figure 2 This is a schematic diagram of a GPU cluster provided by related technology, see Figure 2 , the GPU cluster consists of multiple computing nodes ( Figure 2 Taking 4 computing nodes as an example), each computing node includes at least one GPU ( Figure 2 Take 8 GPUs as an example), and the GPUs in the GPU cluster are all of the same type, so the computing nodes in the GPU cluster are all of the same type. And, Figure 2 The type of GPU in Figure 1 The types of GPUs are different, so Figure 2 The type of computing nodes in Figure 1 The types of computing nodes in are also different.

[0111] In order to facilitate distinction in the embodiments of the present application, Figure 2The GPUs in the GPU cluster shown are called second-type GPUs, and the communication library used by the second-type GPUs is called a second communication library.

[0112] Figure 3 is a schematic diagram of a hybrid GPU cluster provided in an embodiment of the present application, see Figure 3 , a hybrid GPU cluster consists of multiple computing nodes ( Figure 3 Taking 4 computing nodes as an example), each computing node includes at least one GPU ( Figure 3 Taking 8 GPUs as an example), the same computing node includes the same type of GPU, and the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different. Among them, each GPU in the hybrid GPU cluster is an independent unit and can provide image data processing functions.

[0113] Each computing node in the hybrid GPU cluster can be a terminal, a server, or a chip, etc., which is not limited in the embodiments of the present application. Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. Optionally, the server is an independent physical server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, and big data and artificial intelligence platforms. The hybrid GPU cluster can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers.

[0114] See also Figure 3 , node 0 and node 1 are first-class computing nodes, including first-class GPUs, and the communication library used by node 0 and node 1 is the first communication library. Node 2 and node 3 are second-class computing nodes, including second-class GPUs, and the communication library used by node 2 and node 3 is the second communication library. In addition, the communication library used when communicating between different types of computing nodes is a universal communication library. The universal communication library refers to a communication library that can communicate between different types of GPUs, which can be a GLOO communication library or other communication libraries, etc., which is not limited in the embodiments of the present application.

[0115] For example, nodes 0 and 1 are NVIDIA chips, the first type of GPU is NVIDIA GPU, and the first communication library is NVIDIA collective communication library. Nodes 2 and 3 are Ascend chips, the second type of GPU is Ascend GPU, the second communication library is Huawei collective communication library, and the general communication library is GLOO communication library.

[0116] It should be noted that for the sake of convenience, Figure 1Only 4 computing nodes in the hybrid GPU cluster and 8 GPUs in each computing node are shown. In fact, the hybrid GPU cluster can also include more computing nodes, and each computing node can also include more GPUs, which is not limited in the embodiments of the present application. In addition, the GPU involved in the embodiments of the present application can be a GPGPU (General-Purpose computing on Graphics Processing Units) or other types of GPUs, which is not limited in the embodiments of the present application. Among them, GPGPU is a graphics processing unit that uses graphics processing units to process graphics tasks to calculate general computing tasks originally processed by the central processing unit. These general computing tasks may be unrelated to graphics processing.

[0117] The embodiment of the present application takes the first computing node as an example, and the first computing node is any computing node in the hybrid GPU cluster. The first computing node is used to obtain global hardware information, and the global hardware information includes the hardware information of each computing node in the hybrid GPU cluster, and the hardware information includes the type of GPU in the computing node; based on the global hardware information, communication group information is created, and the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, and the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group. The homogeneous sub-communication group includes a GPU that meets the type of the first computing node, and the general sub-communication group includes a GPU that does not meet the type of the first computing node. In addition, the first computing node is also used to communicate with the homogeneous sub-communication group through the homogeneous communication library based on the communication group information, and communicate with the general sub-communication group through the general communication library. The functions of other computing nodes in the hybrid GPU cluster are the same as those of the first computing node, and the embodiments of the present application will not be repeated.

[0118] In addition, in addition to the GPU, the first computing node may also include a CPU, a RoCE V2 network card, a switch or other units, which is not limited in the embodiment of the present application. In addition to the type of GPU, the hardware information of each computing node may also include the number of RoCE V2 network cards in the computing node, the manufacturer of the computing node, the version number, the production date and other information, which is not limited in the embodiment of the present application.

[0119] In one possible implementation, considering that the hybrid GPU cluster includes different types of GPUs, each GPU in the hybrid GPU cluster can communicate with other GPUs through a universal communication library to obtain hardware information of other GPUs, thereby obtaining global hardware information, avoiding the problem of different types of GPUs being unable to communicate with each other, and ensuring that global hardware information is obtained smoothly and accurately.

[0120] In an embodiment of the present application, multiple communication groups can be created in a hybrid GPU cluster, such as a PP (Pipeline Parallelism) communication group, a TP (Tensor Parallelism) communication group, or a DP (Data Parallelism) communication group. The multiple communication groups can be created by default by the hybrid GPU cluster, or created by a technician, or created in the framework of a neural network model that currently needs to be processed by the hybrid GPU cluster, or created according to the type of task currently required to be processed, and the embodiment of the present application does not limit this. The same communication group can include GPUs located in the same computing node or GPUs located in different computing nodes. In addition, the same communication group can include only GPUs of the same type, or can also include GPUs of different types. Communication will be performed between different GPUs in each communication group, thereby completing the communication task of the hybrid GPU cluster. Among them, if the communication group includes different types of GPUs, it is necessary to communicate between these different types of GPUs, which requires breaking the communication barriers between different types of GPUs.

[0121] An embodiment of the present application provides a method for adaptively creating a communication group. The first computing node can determine the type of GPU in each computing node in the hybrid GPU cluster based on global hardware information, thereby determining the type of GPU in each communication group in the hybrid GPU cluster, thereby determining the global topology of the hybrid GPU cluster. The first computing node can adaptively create communication group information of the first computing node.

[0122] If the GPUs in any communication group are of the same type, such as GPUs of the same architecture, or GPUs produced by the same manufacturer, etc., it means that the communication group is a single communication group. At this time, if the GPUs in the communication group meet the type of the first computing node, the communication group is regarded as a homogeneous communication group. If none of the GPUs in the communication group meet the type of the first computing node, the communication group is regarded as a heterogeneous communication group, or the communication group is ignored.

[0123] If any communication group includes GPUs of different types, such as GPUs of different architectures, or GPUs produced by different manufacturers, it means that the communication group is a mixed communication group. At this time, the GPUs in the mixed communication group are divided according to their types to obtain similar sub-communication groups and general sub-communication groups. The similar sub-communication groups include GPUs that meet the first computing node type, and the general sub-communication groups include GPUs that do not meet the first computing node type, so that subsequent communication can be performed with the similar sub-communication groups and the general sub-communication groups through different communication libraries.

[0124] The GPU that meets the first computing node type refers to a GPU of the same type as the GPU in the first computing node, and the GPU that does not meet the first computing node type refers to a GPU of a different type than the GPU in the first computing node.

[0125] After the first computing node classifies the communication groups in the hybrid GPU cluster, it can determine that communication with the same type of communication groups and the same type of sub-communication groups is carried out through the same type of communication library, while communication with the general sub-communication group needs to be carried out through the general communication library. This is equivalent to setting up two different communication libraries for the hybrid communication group, ensuring that the first computing node can communicate with the same type of GPU and different types of GPUs in the hybrid communication group through different communication libraries respectively.

[0126] For example, a hybrid GPU cluster includes a first type of computing node and a second type of computing node, where the first type of computing node includes a first type of GPU, and the second type of computing node includes a second type of GPU. Figure 4 In the case where the first computing node is a first-type computing node, if the first computing node determines that the communication group includes only the first-type GPUs but not the second-type GPUs, the communication group is determined to be the first communication group (i.e., the same-type communication group), and the first communication group is determined to use the first communication library. If it is determined that the communication group includes the first-type GPUs and the second-type GPUs, the communication group is determined to be a mixed communication group, and the communication group is divided into a first sub-communication group (i.e., the same-type sub-communication group) and a general sub-communication group according to different GPU types, and the first sub-communication group is determined to use the first communication library, while the general sub-communication group uses the general communication library.

[0127] See also Figure 5 , when the first computing node is a computing node of the second type, if the first computing node determines that the communication group includes only the second type of GPUs but not the first type of GPUs, the communication group is determined to be a second communication group (i.e., a communication group of the same type), and the second communication group is determined to use the second communication library. If it is determined that the communication group includes the first type of GPUs and the second type of GPUs, the communication group is determined to be a mixed communication group, and the communication group is divided into a second sub-communication group (i.e., a sub-communication group of the same type) and a general sub-communication group according to different GPU types, and the second sub-communication group is determined to use the second communication library, while the general sub-communication group uses the general communication library.

[0128] For example, for NVIDIA chips, when all members of the communication group are NVIDIA GPUs, the communication group is determined to be a homogeneous communication group, and the communication library used should be the NCCL communication library. When the communication group members include NVIDIA GPUs and Ascend GPUs, the communication group is determined to be a mixed communication group, which includes a homogeneous sub-communication group and a universal sub-communication group, wherein the communication library used by the homogeneous sub-communication group should be the NCCL communication library, and the communication library used by the universal sub-communication group should be the GLOO communication library. For Ascend chips, when all members of the communication group are Ascend GPUs, the communication group is determined to be a homogeneous communication group, and the communication library used should be the HCCL communication library. When the communication group members include NVIDIA GPUs and Ascend GPUs, the communication group is determined to be a mixed communication group, which includes a homogeneous sub-communication group and a universal sub-communication group, wherein the communication library used by the homogeneous sub-communication group should be the HCCL communication group, and the communication library used by the universal sub-communication group should be the GLOO communication library.

[0129] See also Figure 6 The hybrid GPU cluster includes four computing nodes: node 0, node 1, node 2 and node 3, each computing node includes eight GPUs, and three communication groups are created in the hybrid GPU cluster: communication group 601, communication group 602 and communication group 603, wherein communication group 601 includes the first type of GPU, so communication group 601 is the first communication group, communication group 602 includes the second type of GPU, so communication group 602 is the second communication group, and communication group 603 includes both the first type of GPU and the second type of GPU, so communication group 603 is a hybrid communication group, and in this hybrid communication group, the first type of GPU located on node 0 and node 1 constitutes a first sub-communication group, and the second type of GPU located on node 2 and node 3 constitutes a second sub-communication group.

[0130] For nodes 0 and 1, communication group 601 is a homogeneous sub-communication group, and in communication group 603, the first-class GPUs located on nodes 0 and 1 constitute a homogeneous sub-communication group, and the second-class GPUs located on nodes 2 and 3 constitute a general sub-communication group. For nodes 2 and 3, communication group 602 is a homogeneous sub-communication group, and in communication group 603, the first-class GPUs located on nodes 0 and 1 constitute a general sub-communication group, and the second-class GPUs located on nodes 2 and 3 constitute a homogeneous sub-communication group.

[0131] It should be noted that the above process of determining the communication library corresponding to each communication group can be implemented during the initialization process of the hybrid GPU cluster. In the related art, when the hybrid GPU cluster is initialized, multiple communication groups will be determined, and the corresponding communication library will be determined for the communication group of the same type as the current computing node. However, for a communication group of a different type from the current computing node, since there is a communication barrier between the current computing node and this communication group, it is impossible to determine which communication library this communication group uses, which leads to initialization failure. In the embodiment of the present application, the communication barrier is broken, and even if it is a communication group of a different type from the current computing node, the appropriate communication library will be adaptively determined based on the type of the GPU of this communication group.

[0132] An embodiment of the present application provides a solution for adaptive communication in a hybrid GPU cluster. Any computing node in the hybrid GPU cluster creates communication group information based on the hardware information of each GPU, thereby dividing the communication groups in the hybrid GPU cluster according to the GPU type. Therefore, the hybrid communication group of the computing node can be determined, and the hybrid communication group is divided into the same sub-communication group and the general sub-communication group, so as to adaptively select the appropriate communication library according to the type of the sub-communication group, communicate with the same sub-communication group through the same communication library, and communicate with the general sub-communication group through the general communication library, which can realize communication between GPUs of the same type and communication between GPUs of different types, thereby breaking the communication barriers between different types of GPUs, and being able to effectively utilize different types of GPUs in the hybrid GPU cluster to process tasks, thereby improving processing efficiency.

[0133] Furthermore, the embodiments of the present application can be applied to any type of current GPU, and can solve the problem that GPUs produced by different manufacturers cannot communicate with each other.

[0134] Based on the above embodiment, the process of the first computing node creating communication group information is described as follows.

[0135] The first computing node creates communication group information corresponding to each communication group in the hybrid GPU cluster based on the type of GPU in the communication group and the type of GPU in the first computing node, thereby obtaining communication group information corresponding to each communication group in the hybrid GPU cluster.

[0136] In one possible implementation, the first computing node is used to create first communication group information corresponding to any communication group in the hybrid GPU cluster when it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes GPUs that conform to the first computing node type and includes GPUs that do not conform to the first computing node type; wherein the first communication group information is used to indicate that the communication group is a hybrid communication group, the GPUs that conform to the first computing node type in the communication group constitute a similar sub-communication group, and the GPUs that do not conform to the first computing node type in the communication group constitute a general sub-communication group.

[0137] In another possible implementation, the first computing node is also used to create second communication group information corresponding to the communication group when it is determined based on global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type; wherein the second communication group information is used to indicate that the communication group is a similar communication group.

[0138] In the hybrid GPU cluster provided in the embodiment of the present application, the first computing node can determine whether the type of GPU in each communication group in the hybrid GPU cluster conforms to the type of the first computing node based on global hardware information, thereby adaptively creating communication group information corresponding to each communication group, and using different communication group information to distinguish different types of communication groups, thereby achieving accurate division of communication groups and laying a good foundation for adaptively selecting the communication library corresponding to each communication group.

[0139] In addition, the first computing node is also used to create third communication group information corresponding to any communication group in the hybrid GPU cluster when it is determined based on the global hardware information that the communication group includes a GPU that does not conform to the first computing node type and does not include a GPU that conforms to the first computing node type, wherein the third communication group information is used to indicate that the communication group is a heterogeneous communication group.

[0140] In addition, considering that the heterogeneous communication group will not include the GPU in the first computing node, the first computing node will not communicate with the heterogeneous communication group. Therefore, the above-mentioned process of creating the third communication group information may not be executed, that is, the first computing node is also used to ignore the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that does not conform to the first computing node type, and does not include a GPU that conforms to the first computing node type.

[0141] In one possible implementation, the hybrid GPU cluster provided by the embodiment of the present application is used in a model training scenario, and can process various tasks in the process of training a neural network model, so that different types of GPUs can be used together in model training, effectively reducing the idle rate of GPUs, expanding the scale of model training, and improving training efficiency, so that the training efficiency can reach more than 90% or even higher. In addition, the code of the training framework does not need to be changed during the communication of the hybrid GPU cluster, so this communication method can be used independently, realizing the decoupling of communication and computing.

[0142] In another possible implementation, the hybrid GPU cluster provided in the embodiment of the present application can also be used in a model reasoning scenario, and can perform corresponding reasoning tasks through a trained neural network model, thereby improving reasoning efficiency.

[0143] The hybrid GPU cluster provided in the related art has a high coupling to the training framework, and the communication tasks and computing tasks are coupled in the training framework of the neural network model. It is impossible to optimize the communication problem or the computing problem separately, and it is mainly used in the scenario of training the neural network model, and cannot be expanded to the scenario of reasoning based on the neural network model. The hybrid GPU cluster provided in the embodiment of the present application fully decouples the communication task from the computing task, has a low coupling of the solution, has the ability to optimize the communication or computing of the hybrid GPU cluster separately, and has strong scalability. Moreover, it can be used in the training scenario of the neural network model, and can also be used in the scenario of reasoning based on the neural network model, and has a very wide range of applications.

[0144] Based on the above embodiment, in a point-to-point communication task, two GPUs need to communicate with each other, and the communication library to be used needs to be adaptively determined based on whether the two GPUs are of the same type.

[0145] In one possible implementation, the first computing node is used to send communication data of the first GPU to the first CPU in the first computing node when it is determined that the communication target of the first GPU is the second GPU in the general sub-communication group based on the communication group information, and the first CPU sends the communication data to the second CPU in the second computing node through the general communication library; wherein the first GPU is any GPU in the first computing node, the second computing node is the computing node where the second GPU is located, and the second CPU is used to receive communication data through the general communication library and send communication data to the second GPU.

[0146] In the embodiment of the present application, the first GPU needs to send communication data to the second GPU as an example. The first computing node will receive a point-to-point communication task, and the first computing node has created communication group information. Based on the point-to-point communication task and the communication group information, the first GPU and the second GPU that need to communicate, as well as the communication groups in which the first GPU and the second GPU are located, can be determined. If the first GPU is located at the first computing node, the first GPU and the second GPU are located in the same hybrid communication group, and the second GPU is located in the general sub-communication group of the first computing node, this means that the first GPU and the second GPU are different types of GPUs. Therefore, when the first GPU and the second GPU communicate with each other, they need to communicate through the general communication library. See Figure 6 , taking the first computing node as node 1 as an example, if communication group 603 is currently needed to perform a communication task, node 1 determines that GPU4 in node 1 needs to send communication data to GPU0 in node 2, and GPU0 in node 2 is located in the general sub-communication group of node 1, so communication is required through the general communication library.

[0147] The first computing node includes a first GPU and a first CPU, and the second computing node includes a second GPU and a second CPU. During the communication process between the first GPU and the second GPU, the first GPU first performs a D2H operation to transfer communication data from the first GPU to the first CPU, and then the first CPU sends the communication data to the second CPU through a general database, and then the second CPU performs an H2D operation to transfer the communication data from the second CPU to the second GPU.

[0148] See also Figure 7 , the first GPU is Figure 7 GPU0 in the second GPU is Figure 7 GPU1 in the figure, both are different types of GPUs. When performing point-to-point communication, the first GPU performs a D2H operation to move the device data (i.e., Device Data, the communication data in the first GPU) to the first CPU, that is, the host data (Host Data) is obtained, and then the host data (Host Data) is moved to the RoCE V2 network card of the first computing node through the GLOO communication library, and the host data (Host Data) is sent to the RoCE V2 network card of the second computing node through the RoCE V2 protocol. The RoCE V2 network card of the second computing node then moves the host data (Host Data) to the second CPU through the GLOO communication library, and the second CPU performs an H2D operation to move the host data (Host Data) to the second GPU, that is, the device data (Device Data) sent by the first GPU is obtained.

[0149] In another possible implementation, the first computing node is used to send communication data of the third GPU to the fourth GPU through a similar communication library when it is determined based on the communication group information that the communication target of the third GPU is a fourth GPU in a similar sub-communication group; wherein the third GPU is any GPU in the first computing node.

[0150] In the embodiment of the present application, the third GPU needs to send communication data to the fourth GPU as an example. The third GPU is located in the first computing node, and the fourth GPU is located in the same type of sub-communication group of the first computing node. This means that the third GPU and the fourth GPU are GPUs of the same type. Therefore, when the third GPU and the fourth GPU communicate with each other, they need to communicate through the same type of communication library.

[0151] For example, see Figure 8 If the third GPU and the fourth GPU are first-type GPUs, then the same-type communication library is the first communication library; or, if the third GPU and the fourth GPU are second-type GPUs, then the same-type communication library is the second communication library.

[0152] Different types of the first computing node use different similar communication libraries. For example, if the first computing node is an NVIDIA chip, the similar communication library is the NCCL communication library. Alternatively, if the first computing node is an Ascend chip, the similar communication library is the HCCL communication library.

[0153] In the embodiment of the present application, communication group information is created based on global hardware information, thereby dividing the hybrid communication group into similar sub-communication groups and general sub-communication groups, so that for the first GPU and the second GPU of different types in the hybrid communication group, the communication data is forwarded through the general communication library with the help of the first CPU and the second CPU, so that the communication between the first GPU and the second GPU of different types is realized, and the communication barrier problem between different types of GPUs is solved. For the third GPU and the fourth GPU of the same type in the hybrid communication group, they can communicate directly through the similar communication library, so the embodiment of the present application can adaptively select different communication libraries in the same communication group, and will not be limited to the same communication group must use the same communication library, which improves flexibility.

[0154] In the above embodiment, a hybrid communication group including a first computing node in a hybrid GPU cluster is used as an example to illustrate the process of determining a hybrid communication group. In another possible implementation, the communication group information created by the first computing node is also used to indicate a similar communication group in the hybrid GPU cluster, and the similar communication group includes a GPU that meets the type of the first computing node, that is, the type of the GPU in the similar communication group is the same as the type of the GPU in the first computing node. In this case, the first computing node is also used to communicate with the similar communication group through a similar communication library based on the communication group information.

[0155] Exemplarily, in the embodiment of the present application, the fifth GPU needs to send communication data to the sixth GPU as an example. The first computing node is used to send the communication data of the fifth GPU to the sixth GPU through the same communication library when the communication target of the fifth GPU is determined to be the sixth GPU in the same communication group based on the communication group information. The fifth GPU is any GPU in the first computing node. Since the sixth GPU is located in the same communication group of the first computing node, it means that the fifth GPU and the sixth GPU are GPUs of the same type, and the two can communicate through the same communication library. The sixth GPU can be located in the first computing node, or in other computing nodes of the same type as the first computing node, and the embodiment of the present application does not limit this.

[0156] The above communication process is the same as the process in which the first computing node communicates with the same type of sub-communication group through the same type of communication library in the above embodiment, and will not be repeated here.

[0157] In an embodiment of the present application, communication group information is created based on global hardware information, so as to accurately distinguish between similar communication libraries and hybrid communication libraries. For similar communication libraries, communication can be directly carried out through similar communication libraries, and for hybrid communication libraries, similar communication libraries or general communication libraries can also be accurately selected. Therefore, the embodiment of the present application can adaptively select communication libraries, thereby improving flexibility.

[0158] On the basis of the above embodiments, in a collective communication task, a collective communication operation needs to be performed between at least two GPUs. Different types of collective communication tasks require different collective communication operations. For example, collective communication operations may include: Reduce operation, AllReduce operation, Broadcast operation, Scatter operation, Gather operation, AllGather operation, etc., which are not limited in the embodiments of the present application.

[0159] The Reduce operation is used to reduce the data on multiple GPUs to one GPU, and the reduction operation can be a cumulative sum operation, a cumulative product operation, a maximum value operation, a minimum value operation, etc. The AllReduce operation is used to perform the same reduction operation on all GPUs in the communication group, and then send the data obtained after the reduction operation to all GPUs. The Broadcast operation is used to broadcast data on one GPU to other GPUs, the Scatter operation is used to slice the data on one GPU and then send it to other GPUs separately, the Gather operation is used to collect data on multiple GPUs to one GPU, and the AllGather operation is used to collect data on multiple GPUs to one GPU and then distribute the collected data to each GPU.

[0160] In one possible implementation, the first computing node is used to respond to the collective communication task for the hybrid communication group, and to perform a collective communication operation matching the collective communication task between GPUs belonging to the same sub-communication group in the first computing node through the same communication library to obtain first collective communication data. The first computing node is also used to send the first collective communication data to the third computing node through the general communication library, and the GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

[0161] During the processing of the collective communication task of the hybrid communication group, the participation of at least two types of GPUs in the hybrid communication group is required. Since the hybrid communication group includes the same type of sub-communication groups and general sub-communication groups, for the first computing node, it is necessary to adaptively determine the communication library to be used based on the type of GPU performing the communication.

[0162] Exemplarily, after obtaining the first collective communication data, one of the GPUs in the first computing node performs point-to-point communication with one of the GPUs in the third computing node through the general communication library, and sends the first collective communication data to the GPU in the third computing node. The GPU in the third computing node that receives the first collective communication data can perform a broadcast operation to broadcast the first collective communication data to other GPUs in the third computing node. Afterwards, each GPU that receives the first collective communication data can perform a collective communication operation to obtain the second collective communication data.

[0163] Exemplarily, after obtaining the second collective communication data, the third computing node may also send the second collective communication data to the first computing node through the general communication library, and the second collective communication data is the final processing result of the collective communication task.

[0164] See also Fig. 9, taking the first computing node as node 1 and the third computing node as node 2 as an example, node 1 is an NVIDIA chip and node 2 is an Ascend chip. The collective communication task is an AllReduce task, and the AllReduce task indicates that the data of each GPU in the hybrid communication group is summed. From the perspective of node 1, node 1 responds to the AllReduce task for the hybrid communication group and determines that the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group. The homogeneous sub-communication group includes 8 GPUs on node 1, and the general sub-communication group includes 8 GPUs on node 2. First, the AllReduce operation is performed between the 8 GPUs on node 1 through the NCCL communication library, so as to sum the data of the 8 GPUs, and the first collective communication data obtained is the sum of the data of the 8 GPUs. Afterwards, node 1 sends the first collective communication data to node 2 through the GLOO communication library. Based on the first collective communication data, the 8 GPUs in node 2 can perform an AllReduce operation among the 8 GPUs through the HCCL communication library, thereby continuing to perform a summation operation based on the first collective communication data. The obtained second collective communication data is the sum of the data of the 16 GPUs in the hybrid communication group.

[0165] In addition, node 2 can also send the second collective communication data to node 1 through the GLOO communication library, and the GPU on node 1 that receives the second collective communication data broadcasts the second collective communication data to other GPUs in node 1, so that each GPU can obtain the second collective communication data.

[0166] The process of communicating through the GLOO communication library is similar to the above Figure 7 The process of communicating through the general communication library is similar, so I will not go into details here.

[0167] It should be noted that the above embodiment communicates with the GPUs on the same computing node before communicating with the GPUs on another computing node. In another embodiment, if the number of RoCE V2 network cards on the two computing nodes involved is equal, the GPUs in the hybrid communication group can communicate point-to-point respectively, so that each GPU can obtain the data of each other GPU after completing the communication with each other GPU in the hybrid communication group, so that the data of each GPU in the hybrid communication group can be summed to obtain the sum result. If the number of RoCE V2 network cards on the two computing nodes involved is not equal, when the GPU located on the first computing node and the GPU located on the third computing node communicate, different GPUs may share the same RoCE V2 network card, resulting in communication jams and affecting communication efficiency. Therefore, in this case, after the communication between the GPUs on the same computing node is completed, the obtained collective communication data can be communicated with another computing node.

[0168] An embodiment of the present application provides a solution for processing collective communication tasks, which can first complete the communication between GPUs on the same computing node, and then communicate the obtained collective communication data with another computing node through a general communication library, thereby avoiding communication jams and improving communication efficiency.

[0169] In another possible implementation, the first computing node is used to respond to the collective communication task for the homogeneous communication group, and to perform a collective communication operation matching the collective communication task between GPUs belonging to the homogeneous communication group in the first computing node through a homogeneous communication library to obtain third collective communication data. The first computing node is also used to send the third collective communication data to the fourth computing node through the homogeneous communication library, and the GPU belonging to the homogeneous communication group in the fourth computing node performs a collective communication operation matching the collective communication task based on the third collective communication data to obtain fourth collective communication data.

[0170] Exemplarily, after obtaining the third collective communication data, one of the GPUs in the first computing node performs point-to-point communication with one of the GPUs in the fourth computing node through a similar communication library, and sends the third collective communication data to the GPU in the fourth computing node. The GPU in the fourth computing node that receives the third collective communication data can perform a broadcast operation to broadcast the third collective communication data to other GPUs in the fourth computing node. Afterwards, each GPU that receives the third collective communication data can perform a collective communication operation between each other, thereby obtaining the fourth collective communication data.

[0171] Exemplarily, after obtaining the fourth collective communication data, the fourth computing node may also send the fourth collective communication data to the first computing node through a similar communication library, and the fourth collective communication data is the final processing result of the collective communication task.

[0172] See also Fig.10 , taking the first computing node as node 0 and the fourth computing node as node 1 as an example, both node 0 and node 1 are NVIDIA chips. The collective communication task is an AllReduce task, and the AllReduce task indicates that the data of each GPU in the communication group is summed. From the perspective of node 0, node 0 responds to the AllReduce task for the same communication group, and first performs an AllReduce operation between the 8 GPUs on node 0 through the NCCL communication library, thereby summing the data of the 8 GPUs, and the third collective communication data obtained is the sum of the data of the 8 GPUs. Afterwards, node 0 sends the third collective communication data to node 1 through the NCCL communication library, and the 8 GPUs in node 1 can perform an AllReduce operation between these 8 GPUs through the NCCL communication library on the basis of the third collective communication data, thereby continuing to perform a summation operation on the basis of the third collective communication data, and the fourth collective communication data obtained is the sum of the data of the 16 GPUs in this same communication group.

[0173] In addition, node 1 can also send the fourth collective communication data to node 0 through the NCCL communication library, and the GPU on node 0 that receives the fourth collective communication data broadcasts the fourth collective communication data to other GPUs in node 0, so that each GPU can obtain the fourth collective communication data.

[0174] See also Fig.11 , taking the first computing node as node 2 and the fourth computing node as node 3 as an example, both node 2 and node 3 are Ascend chips. The collective communication task is an AllReduce task, and the AllReduce task indicates that the data of each GPU in the communication group is summed. From the perspective of node 2, node 2 responds to the AllReduce task for the same communication group, and first performs an AllReduce operation between the 8 GPUs on node 2 through the HCCL communication library, thereby summing the data of the 8 GPUs, and the third collective communication data obtained is the sum of the data of the 8 GPUs. Afterwards, node 2 sends the third collective communication data to node 3 through the HCCL communication library, and the 8 GPUs in node 3 can perform an AllReduce operation between these 8 GPUs based on the third collective communication data through the HCCL communication library, thereby continuing to perform a summation operation based on the third collective communication data, and the fourth collective communication data obtained is the sum of the data of the 16 GPUs in this same communication group.

[0175] In addition, node 3 can also send the fourth collective communication data to node 2 through the HCCL communication library, and the GPU on node 2 that receives the fourth collective communication data broadcasts the fourth collective communication data to other GPUs in node 2, so that each GPU can obtain the fourth collective communication data.

[0176] On the basis of the above-mentioned embodiments, the embodiments of the present application also provide a load balancing solution. This is because different types of computing nodes have different hardware configurations and processing efficiencies, and different computing powers. Even if the same load is configured, the processing time spent is different. If the loads of different types of computing nodes in the hybrid GPU cluster are unbalanced, the hybrid GPU cluster will be subject to the short board effect, and the efficiency of processing tasks will be greatly reduced. Therefore, in order to improve the efficiency of processing tasks, in the hybrid GPU cluster provided by the embodiments of the present application, the computing power ratio of any two different types of computing nodes is equal to the load ratio of the two different types of computing nodes. In this way, the load of each computing node in the hybrid GPU cluster can be guaranteed to be balanced, avoiding the limitation of the short board effect, and the overall processing efficiency of the hybrid GPU cluster can be maximized.

[0177] The computing power ratio of any two different types of computing nodes can be the reciprocal of the ratio of the time spent by the two computing nodes in processing the same amount of tasks, or, when the two computing nodes contain the same number of GPUs, the computing power ratio of the two computing nodes can be the ratio of the computing power of a single GPU on the two computing nodes. For example, if the theoretical computing power ratio of the first type of GPU to the second type of GPU is 1:1, then the computing power ratio of the first type of computing node to the second type of computing node is 1:1.

[0178] For example, considering that there may be errors in the theoretical computing power of the GPU, an offline method can also be used to run the same amount of tasks on the first type of GPU and the second type of GPU, respectively, and count the running time consumed by the first type of GPU and the second type of GPU, and determine the inverse of the ratio of the running time of the first type of GPU and the second type of GPU as the computing power ratio of the first type of GPU and the second type of GPU.

[0179] The load of a computing node represents the amount of tasks that the computing node needs to complete, which can be allocated to the computing node by the hybrid GPU cluster. For example, when executing a training task of a neural network model through a hybrid GPU cluster, the neural network model includes multiple model layers, and the model parameters in each model layer need to be trained during the execution of the training task. Then the load of each computing node can be represented by the number of model layers allocated to each computing node.

[0180] In a hybrid GPU cluster, it is necessary to allocate load to each computing node according to the computing power ratio of different types of computing nodes and the total task volume to ensure that the computing power ratio and load ratio of different computing nodes are equal.

[0181] For example, the computing power of the first type of GPU is , the computing power of the second type of GPU is , the total number of model layers of the neural network model is , then the following formula is satisfied:

[0182] ;

[0183] ;

[0184] in, The number of model layers assigned to the first type of compute nodes, The number of model layers to allocate to the second type of compute nodes.

[0185] See also Fig.12 , the total number of model layers of the neural network model is 16. When the computing power ratio of the first type of GPU to the second type of GPU is 1:1, the number of model layers allocated to each of Node 0, Node 1, Node 2, and Node 3 is 4. Fig.13 , the total number of model layers of the neural network model is 18. When the computing power ratio of the first type of GPU to the second type of GPU is 2:1, the number of model layers allocated to nodes 0 and 1 is 6 respectively, and the number of model layers allocated to nodes 2 and 3 is 3 respectively.

[0186] Based on the above embodiments, Fig.14 is a flow chart of a communication method provided by an embodiment of the present application, and the embodiment of the present application is executed by a first computing node. The hybrid GPU cluster includes multiple computing nodes, each computing node includes at least one graphics processing unit GPU; wherein the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different. The first computing node is any computing node in the hybrid GPU cluster. See Fig.14 , the method comprises the following steps.

[0187] 1401. A first computing node obtains global hardware information, where the global hardware information includes hardware information of each computing node in a hybrid GPU cluster, where the hardware information includes the type of GPU in the computing node; based on the global hardware information, communication group information is created, where the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, where the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, where the homogeneous sub-communication group includes a GPU that meets the type of the first computing node, and where the general sub-communication group includes a GPU that does not meet the type of the first computing node.

[0188] 1402. Based on the communication group information, the first computing node communicates with the same sub-communication group through the same communication library, and communicates with the general sub-communication group through the general communication library.

[0189] In a possible implementation, the first computing node communicates with the same type of sub-communication group through the same type of communication library and communicates with the general sub-communication group through the general communication library based on the communication group information, including:

[0190] When the first computing node determines that the communication target of the first GPU is the second GPU in the general sub-communication group based on the communication group information, the first computing node sends the communication data of the first GPU to the first CPU in the first computing node, and the first CPU sends the communication data to the second CPU in the second computing node through the general communication library;

[0191] The first GPU is any GPU in the first computing node, the second computing node is a computing node where the second GPU is located, and the second CPU is used to receive communication data through a general communication library and send communication data to the second GPU.

[0192] In a possible implementation, the first computing node communicates with the same type of sub-communication group through the same type of communication library and communicates with the general sub-communication group through the general communication library based on the communication group information, including:

[0193] When the first computing node determines, based on the communication group information, that the communication target of the third GPU is the fourth GPU in the same type of sub-communication group, the first computing node sends the communication data of the third GPU to the fourth GPU through the same type of communication library;

[0194] The third GPU is any GPU in the first computing node.

[0195] In a possible implementation, creating communication group information based on global hardware information includes:

[0196] When it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and includes a GPU that does not conform to the first computing node type, creating first communication group information corresponding to the communication group;

[0197] The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the first computing node type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the first computing node type constitute a general sub-communication group.

[0198] In a possible implementation, the communication group information is further used to indicate a homogeneous communication group in the hybrid GPU cluster, where the homogeneous communication group includes GPUs that conform to the first computing node type; and the method further includes:

[0199] Based on the communication group information, communicate with similar communication groups through the same communication library.

[0200] In a possible implementation, based on the communication group information, communicating with a similar communication group through a similar communication library includes:

[0201] When it is determined based on the communication group information that the communication target of the fifth GPU is a sixth GPU in the same communication group, sending the communication data of the fifth GPU to the sixth GPU through the same communication library;

[0202] The fifth GPU is any GPU in the first computing node.

[0203] In a possible implementation, creating communication group information based on global hardware information includes:

[0204] When it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type, create second communication group information corresponding to the communication group;

[0205] The second communication group information is used to indicate that the communication group is a communication group of the same type.

[0206] In a possible implementation, the first computing node communicates with the same type of sub-communication group through the same type of communication library and communicates with the general sub-communication group through the general communication library based on the communication group information, including:

[0207] In response to a collective communication task for the hybrid communication group, a collective communication operation matching the collective communication task is performed between GPUs belonging to a similar sub-communication group in the first computing node through a similar communication library to obtain first collective communication data;

[0208] The first collective communication data is sent to the third computing node through the general communication library, and the GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain the second collective communication data.

[0209] In one possible implementation, in a hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

[0210] An embodiment of the present application provides a solution for adaptive communication in a hybrid GPU cluster. Any computing node in the hybrid GPU cluster creates communication group information based on the hardware information of each GPU, thereby dividing the communication groups in the hybrid GPU cluster according to the GPU type. Therefore, the hybrid communication group of the computing node can be determined, and the hybrid communication group is divided into the same sub-communication group and the general sub-communication group, so as to adaptively select the appropriate communication library according to the type of the sub-communication group, communicate with the same sub-communication group through the same communication library, and communicate with the general sub-communication group through the general communication library, which can realize communication between GPUs of the same type and communication between GPUs of different types, thereby breaking the communication barriers between different types of GPUs, and being able to effectively utilize different types of GPUs in the hybrid GPU cluster to process tasks, thereby improving processing efficiency.

[0211] On the basis of the above embodiments, an embodiment of the present application further provides a chip, the chip is configured in a hybrid GPU cluster, the hybrid GPU cluster includes multiple chips, each chip includes at least one graphics processing unit GPU; wherein the multiple chips include at least two types of chips, and the types of GPUs in different types of chips are different;

[0212] The chip is used to obtain global hardware information, the global hardware information includes the hardware information of each chip in the hybrid GPU cluster, and the hardware information includes the type of GPU in the chip; based on the global hardware information, communication group information is created, and the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, the homogeneous sub-communication group includes a GPU that conforms to the chip type, and the general sub-communication group includes a GPU that does not conform to the chip type;

[0213] The chip is also used to communicate with the same sub-communication group through the same communication library based on the communication group information, and to communicate with the general sub-communication group through the general communication library.

[0214] In one possible implementation, the chip is used to send communication data of the first GPU to the first CPU in the chip when it is determined based on the communication group information that the communication target of the first GPU is the second GPU in the general sub-communication group, and the first CPU sends the communication data to the second CPU in another chip through the general communication library;

[0215] The first GPU is any GPU in the chip, the other chip is the chip where the second GPU is located, and the second CPU is used to receive communication data through the general communication library and send communication data to the second GPU.

[0216] In one possible implementation, the chip is used to send communication data of the third GPU to the fourth GPU through a homogeneous communication library when it is determined based on the communication group information that the communication target of the third GPU is a fourth GPU in a homogeneous sub-communication group;

[0217] The third GPU is any GPU in the chip.

[0218] In one possible implementation, the chip is used to create first communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the chip type and includes a GPU that does not conform to the chip type;

[0219] The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs that meet the chip type in the communication group constitute a similar sub-communication group, and the GPUs that do not meet the chip type in the communication group constitute a general sub-communication group.

[0220] In a possible implementation, the communication group information is further used to indicate a homogeneous communication group in a hybrid GPU cluster, where the homogeneous communication group includes GPUs that conform to the chip type;

[0221] The chip is also used to communicate with a similar communication group through a similar communication library based on the communication group information.

[0222] In one possible implementation, the chip is used to send communication data of the fifth GPU to the sixth GPU through a homogeneous communication library when it is determined based on the communication group information that the communication target of the fifth GPU is a sixth GPU in the homogeneous communication group;

[0223] The fifth GPU is any GPU in the chip.

[0224] In one possible implementation, the chip is further used to create second communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the chip type and does not include a GPU that does not conform to the chip type;

[0225] The second communication group information is used to indicate that the communication group is a communication group of the same type.

[0226] In one possible implementation, the chip is used to respond to the collective communication task for the mixed communication group, and to perform a collective communication operation matching the collective communication task between GPUs belonging to the same sub-communication group in the chip through a same communication library to obtain first collective communication data;

[0227] The chip is also used to send first collective communication data to another chip through a general communication library, and a GPU belonging to a general sub-communication group in the other chip performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

[0228] In one possible implementation, in a hybrid GPU cluster, the computing power ratio of any two different types of chips is equal to the load ratio of any two different types of chips.

[0229] Fig.15 This is a schematic diagram of the structure of a communication device provided in an embodiment of the present application. The device is configured in a first computing node in a hybrid GPU cluster, the hybrid GPU cluster includes multiple computing nodes, each computing node includes at least one graphics processing unit (GPU); wherein the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different.

[0230] See also Fig.15 , the device comprises:

[0231] The information processing module 1501 is used to obtain global hardware information, the global hardware information includes the hardware information of each computing node in the hybrid GPU cluster, and the hardware information includes the type of GPU in the computing node; based on the global hardware information, create communication group information, the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, the homogeneous sub-communication group includes a GPU that meets the first computing node type, and the general sub-communication group includes a GPU that does not meet the first computing node type;

[0232] The communication module 1502 is further used to communicate with the same type of sub-communication group through the same type of communication library and communicate with the general sub-communication group through the general communication library based on the communication group information;

[0233] The first computing node is any computing node in the hybrid GPU cluster.

[0234] In one possible implementation, the communication module 1502 is used to send the communication data of the first GPU to the first CPU in the first computing node when it is determined based on the communication group information that the communication target of the first GPU is the second GPU in the general sub-communication group, and the first CPU sends the communication data to the second CPU in the second computing node through the general communication library;

[0235] The first GPU is any GPU in the first computing node, the second computing node is a computing node where the second GPU is located, and the second CPU is used to receive communication data through a general communication library and send communication data to the second GPU.

[0236] In one possible implementation, the communication module 1502 is configured to send communication data of the third GPU to the fourth GPU through a homogeneous communication library when it is determined based on the communication group information that the communication target of the third GPU is a fourth GPU in a homogeneous sub-communication group;

[0237] The third GPU is any GPU in the first computing node.

[0238] In one possible implementation, the information processing module 1501 is used to create first communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and includes a GPU that does not conform to the first computing node type;

[0239] The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the first computing node type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the first computing node type constitute a general sub-communication group.

[0240] In a possible implementation, the communication group information is further used to indicate a homogeneous communication group in the hybrid GPU cluster, where the homogeneous communication group includes GPUs that conform to the first computing node type;

[0241] The communication module 1502 is further used to communicate with a similar communication group through a similar communication library based on the communication group information.

[0242] In one possible implementation, the communication module 1502 is configured to send communication data of the fifth GPU to the sixth GPU through a homogeneous communication library when it is determined based on the communication group information that the communication target of the fifth GPU is a sixth GPU in the homogeneous communication group;

[0243] The fifth GPU is any GPU in the first computing node.

[0244] In one possible implementation, the information processing module 1501 is further configured to create second communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type;

[0245] The second communication group information is used to indicate that the communication group is a communication group of the same type.

[0246] In one possible implementation, the communication module 1502 is configured to, in response to a collective communication task for the hybrid communication group, perform a collective communication operation matching the collective communication task between GPUs belonging to a similar sub-communication group in the first computing node through a similar communication library to obtain first collective communication data;

[0247] The communication module 1502 is also used to send the first collective communication data to the third computing node through the general communication library, and the GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain the second collective communication data.

[0248] In one possible implementation, in a hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

[0249] It should be noted that: the communication device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the communication device provided in the above embodiment and the communication method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0250] An embodiment of the present application further provides a computer device, which includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the communication method of the above embodiment.

[0251] The communication method provided in the embodiment of the present application is executed by a computer device.

[0252] Optionally, the computer device is provided as a terminal. Fig.16 A schematic diagram of the structure of a terminal 1600 provided by an exemplary embodiment of the present application is shown.

[0253] The terminal 1600 includes a processor 1601 and a memory 1602 .

[0254] The processor 1601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0255] The memory 1602 may include one or more computer-readable storage media, which may be non-transitory. The memory 1602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1602 is used to store at least one computer program, which is used to be possessed by the processor 1601 to implement the communication method provided in the method embodiment of the present application.

[0256] In some embodiments, the terminal 1600 may further optionally include: a peripheral device interface 1603 and at least one peripheral device. The processor 1601, the memory 1602 and the peripheral device interface 1603 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1603 via a bus, a signal line or a circuit board. Optionally, the peripheral device includes: at least one of a radio frequency circuit 1604, a display screen 1605, a camera assembly 1606 and a power supply 1607.

[0257] The peripheral device interface 1603 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1601 and the memory 1602. In some embodiments, the processor 1601, the memory 1602, and the peripheral device interface 1603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1601, the memory 1602, and the peripheral device interface 1603 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0258] The radio frequency circuit 1604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1604 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1604 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1604 can communicate with other devices through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1604 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0259] The display screen 1605 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1605 is a touch display screen, the display screen 1605 also has the ability to collect touch signals on the surface or above the surface of the display screen 1605. The touch signal can be input as a control signal to the processor 1601 for processing. At this time, the display screen 1605 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1605 can be one, which is set on the front panel of the terminal 1600; in other embodiments, the display screen 1605 can be at least two, which are respectively set on different surfaces of the terminal 1600 or are folded; in other embodiments, the display screen 1605 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 1600. Even, the display screen 1605 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1605 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0260] The camera component 1606 is used to capture images or videos. Optionally, the camera component 1606 includes a front camera and a rear camera. The front camera is set on the front panel of the terminal 1600, and the rear camera is set on the back of the terminal 1600. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera component 1606 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0261] The power supply 1607 is used to power various components in the terminal 1600. The power supply 1607 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1607 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0262] Those skilled in the art will understand that Fig.16The structure shown in the figure does not constitute a limitation on the terminal 1600, and the terminal 1600 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0263] Optionally, the computer device is provided as a server. Fig.17 It is a structural diagram of a server provided in an embodiment of the present application. The server 1700 may have relatively large differences due to different configurations or accuracy, and may include one or more processors (Central Processing Units, CPU) 1701 and one or more memories 1702, wherein the memory 1702 stores at least one computer program, and the at least one computer program is loaded and executed by the processor 1701 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be described in detail here.

[0264] An embodiment of the present application further provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the communication method of the above embodiment.

[0265] An embodiment of the present application further provides a computer program product, including a computer program, which is loaded and executed by a processor to implement the operations performed by the communication method of the above embodiment.

[0266] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0267] The above description is only an optional embodiment of the embodiments of the present application and is not intended to limit the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A hybrid GPU cluster, characterized in that: The hybrid GPU cluster includes a plurality of computing nodes, each computing node includes at least one graphics processing unit (GPU); wherein the plurality of computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different; the hybrid GPU cluster is created with a plurality of communication groups, wherein a communication group refers to a communication set composed of a plurality of GPUs, and the same communication group includes at least one of a GPU in the same computing node and a GPU in a different computing node, and the same communication group includes at least one of a GPU of the same type and a GPU of a different type; A first computing node, configured to obtain global hardware information, wherein the global hardware information includes hardware information of each computing node in the hybrid GPU cluster, and the hardware information includes the type of GPU in the computing node; based on the global hardware information, the plurality of communication groups are classified to create communication group information of the first computing node, wherein the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, wherein the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, wherein the homogeneous sub-communication group includes a GPU that conforms to the type of the first computing node, and the general sub-communication group includes a GPU that does not conform to the type of the first computing node; The first computing node is further configured to communicate with the same sub-communication group through a same-type communication library based on the communication group information, and communicate with the general sub-communication group through a general communication library, wherein the general communication library refers to a communication library capable of communicating between different types of GPUs; The first computing node is any computing node in the hybrid GPU cluster.

2. The hybrid GPU cluster according to claim 1, characterized in that: The first computing node is configured to send communication data of the first GPU to a first CPU in the first computing node when it is determined based on the communication group information that the communication target of the first GPU is the second GPU in the general sub-communication group, and the first CPU sends the communication data to a second CPU in a second computing node through the general communication library; The first GPU is any GPU in the first computing node, the second computing node is a computing node where the second GPU is located, and the second CPU is used to receive the communication data through the universal communication library and send the communication data to the second GPU.

3. The hybrid GPU cluster according to claim 1, characterized in that: The first computing node is configured to send the communication data of the third GPU to the fourth GPU through the homogeneous communication library when determining, based on the communication group information, that the communication target of the third GPU is a fourth GPU in the homogeneous sub-communication group; The third GPU is any GPU in the first computing node.

4. The hybrid GPU cluster according to claim 1, characterized in that: The first computing node is used to create first communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and includes a GPU that does not conform to the first computing node type; The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the first computing node type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the first computing node type constitute a general sub-communication group.

5. The hybrid GPU cluster according to claim 1, characterized in that: The communication group information is also used to indicate a homogeneous communication group in the hybrid GPU cluster, wherein the homogeneous communication group includes a GPU that conforms to the first computing node type; The first computing node is further configured to communicate with the same communication group through the same communication library based on the communication group information.

6. The hybrid GPU cluster according to claim 5, characterized in that: the first computing node is configured to send the communication data of the fifth GPU to the sixth GPU through the homogeneous communication library when determining, based on the communication group information, that the communication target of the fifth GPU is a sixth GPU in the homogeneous communication group; The fifth GPU is any GPU in the first computing node.

7. The hybrid GPU cluster according to claim 5, characterized in that: The first computing node is further configured to create second communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type; The second communication group information is used to indicate that the communication group is the same type of communication group.

8. The hybrid GPU cluster according to claim 1, characterized in that: The first computing node is configured to, in response to a collective communication task for the hybrid communication group, perform a collective communication operation matching the collective communication task between GPUs belonging to the homogeneous sub-communication group in the first computing node through the homogeneous communication library to obtain first collective communication data; The first computing node is also used to send the first collective communication data to the third computing node through the general communication library, and the GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

9. The hybrid GPU cluster according to any one of claims 1 to 8, characterized in that: In the hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

10. A communication method, characterized in that: A hybrid GPU cluster includes multiple computing nodes, each computing node includes at least one graphics processing unit (GPU); wherein the multiple computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different; the hybrid GPU cluster creates multiple communication groups, and a communication group refers to a communication set composed of multiple GPUs, and the same communication group includes at least one of a GPU in the same computing node and a GPU in a different computing node, and the same communication group includes at least one of a GPU of the same type and a GPU of a different type; the method includes: The first computing node obtains global hardware information, wherein the global hardware information includes hardware information of each computing node in the hybrid GPU cluster, and the hardware information includes the type of GPU in the computing node; based on the global hardware information, the multiple communication groups are classified to create communication group information of the first computing node, wherein the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, wherein the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, wherein the homogeneous sub-communication group includes a GPU that conforms to the type of the first computing node, and the general sub-communication group includes a GPU that does not conform to the type of the first computing node; The first computing node communicates with the same sub-communication group through a same-type communication library based on the communication group information, and communicates with the general sub-communication group through a general communication library, where the general communication library refers to a communication library capable of communicating between different types of GPUs; The first computing node is any computing node in the hybrid GPU cluster.

11. The communication method according to claim 10, characterized in that: The first computing node communicates with the same type of sub-communication group through a same type of communication library and communicates with the general sub-communication group through a general communication library based on the communication group information, including: When the first computing node determines, based on the communication group information, that the communication target of the first GPU is the second GPU in the general sub-communication group, the first computing node sends the communication data of the first GPU to the first CPU in the first computing node, and the first CPU sends the communication data to the second CPU in the second computing node through the general communication library; The first GPU is any GPU in the first computing node, the second computing node is a computing node where the second GPU is located, and the second CPU is used to receive the communication data through the universal communication library and send the communication data to the second GPU.

12. The communication method according to claim 10, characterized in that: The first computing node communicates with the same type of sub-communication group through a same type of communication library and communicates with the general sub-communication group through a general communication library based on the communication group information, including: When the first computing node determines, based on the communication group information, that the communication target of the third GPU is a fourth GPU in the homogeneous sub-communication group, sending the communication data of the third GPU to the fourth GPU through the homogeneous communication library; The third GPU is any GPU in the first computing node.

13. The communication method according to claim 10, characterized in that: The classifying the plurality of communication groups based on the global hardware information and creating the communication group information of the first computing node includes: In a case where it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and includes a GPU that does not conform to the first computing node type, creating first communication group information corresponding to the communication group; The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the first computing node type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the first computing node type constitute a general sub-communication group.

14. The communication method according to claim 10, characterized in that: The communication group information is further used to indicate a homogeneous communication group in the hybrid GPU cluster, wherein the homogeneous communication group includes a GPU that conforms to the first computing node type; the method further includes: Based on the communication group information, communication is performed with the same type communication group through the same type communication library.

15. The communication method according to claim 14, characterized in that: The communicating with the same communication group through the same communication library based on the communication group information includes: When it is determined based on the communication group information that the communication target of the fifth GPU is a sixth GPU in the homogeneous communication group, sending the communication data of the fifth GPU to the sixth GPU through the homogeneous communication library; The fifth GPU is any GPU in the first computing node.

16. The communication method according to claim 14, characterized in that: The classifying the plurality of communication groups based on the global hardware information and creating the communication group information of the first computing node includes: When it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type, creating second communication group information corresponding to the communication group; The second communication group information is used to indicate that the communication group is the same type of communication group.

17. The communication method according to claim 10, characterized in that: The first computing node communicates with the same type of sub-communication group through a same type of communication library and communicates with the general sub-communication group through a general communication library based on the communication group information, including: In response to the collective communication task for the hybrid communication group, performing a collective communication operation matching the collective communication task between the GPUs belonging to the homogeneous sub-communication group in the first computing node through the homogeneous communication library to obtain first collective communication data; The first collective communication data is sent to a third computing node through the general communication library, and a GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

18. The communication method according to any one of claims 10 to 17, characterized in that: In the hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

19. A chip, characterized in that: The chip is configured in a hybrid GPU cluster, the hybrid GPU cluster includes multiple chips, each chip includes at least one graphics processing unit GPU; wherein the multiple chips include at least two types of chips, and the types of GPUs in different types of chips are different; the hybrid GPU cluster creates multiple communication groups, the communication group refers to a communication set composed of multiple GPUs, the same communication group includes at least one of a GPU in the same chip and a GPU in a different chip, and the same communication group includes at least one of a GPU of the same type and a GPU of a different type; The chip is used to obtain global hardware information, the global hardware information includes hardware information of each chip in the hybrid GPU cluster, and the hardware information includes the type of GPU in the chip; based on the global hardware information, the multiple communication groups are classified to create communication group information of the chip, the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, the homogeneous sub-communication group includes a GPU that conforms to the chip type, and the general sub-communication group includes a GPU that does not conform to the chip type; The chip is also used to communicate with the same sub-communication group through a same-type communication library based on the communication group information, and communicate with the general sub-communication group through a general communication library, wherein the general communication library refers to a communication library capable of communicating between different types of GPUs.

20. The chip according to claim 19, characterized in that: The chip is used to send communication data of the first GPU to the first CPU in the chip when it is determined based on the communication group information that the communication target of the first GPU is the second GPU in the general sub-communication group, and the first CPU sends the communication data to the second CPU in another chip through the general communication library; The first GPU is any GPU in the chips, the other chip is the chip where the second GPU is located, and the second CPU is used to receive the communication data through the universal communication library and send the communication data to the second GPU.

21. The chip according to claim 19, characterized in that: The chip is used for sending the communication data of the third GPU to the fourth GPU through the homogeneous communication library when it is determined based on the communication group information that the communication target of the third GPU is the fourth GPU in the homogeneous sub-communication group; The third GPU is any GPU in the chip.

22. The chip according to claim 19, characterized in that: The chip is used to create first communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the chip type and includes a GPU that does not conform to the chip type; The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the chip type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the chip type constitute a general sub-communication group.

23. The chip according to claim 19, characterized in that: The communication group information is also used to indicate a similar communication group in the hybrid GPU cluster, wherein the similar communication group includes a GPU that conforms to the chip type; The chip is also used to communicate with the same communication group through the same communication library based on the communication group information.

24. The chip according to claim 23, characterized in that: The chip is used for sending the communication data of the fifth GPU to the sixth GPU through the homogeneous communication library when it is determined based on the communication group information that the communication target of the fifth GPU is the sixth GPU in the homogeneous communication group; The fifth GPU is any GPU in the chip.

25. The chip according to claim 23, characterized in that The chip is further used to create second communication group information corresponding to the communication group when it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the chip type and does not include a GPU that does not conform to the chip type; The second communication group information is used to indicate that the communication group is the same type of communication group.

26. The chip according to claim 19, characterized in that The chip is used to respond to the collective communication task for the hybrid communication group, and to perform a collective communication operation matching the collective communication task between GPUs belonging to the homogeneous sub-communication group in the chip through the homogeneous communication library to obtain first collective communication data; The chip is also used to send the first collective communication data to another chip through the general communication library, and the GPU belonging to the general sub-communication group in the other chip performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

27. The chip according to any one of claims 19 to 26, characterized in that: In the hybrid GPU cluster, the computing power ratio of any two different types of chips is equal to the load ratio of any two different types of chips.

28. A communication device, characterized in that: The device is configured at a first computing node in a hybrid GPU cluster, wherein the hybrid GPU cluster includes a plurality of computing nodes, each computing node includes at least one graphics processing unit (GPU); wherein the plurality of computing nodes include at least two types of computing nodes, and the types of GPUs in different types of computing nodes are different; the hybrid GPU cluster is created with a plurality of communication groups, wherein a communication group refers to a communication set consisting of a plurality of GPUs, wherein the same communication group includes at least one of a GPU in the same computing node and a GPU in a different computing node, and the same communication group includes at least one of a GPU of the same type and a GPU of a different type; The device comprises: An information processing module, configured to obtain global hardware information, wherein the global hardware information includes hardware information of each computing node in the hybrid GPU cluster, and the hardware information includes the type of GPU in the computing node; based on the global hardware information, the multiple communication groups are classified to create communication group information of the first computing node, wherein the communication group information is used to indicate a hybrid communication group in the hybrid GPU cluster, wherein the hybrid communication group includes a homogeneous sub-communication group and a general sub-communication group, wherein the homogeneous sub-communication group includes a GPU that conforms to the first computing node type, and the general sub-communication group includes a GPU that does not conform to the first computing node type; The communication module is further used to communicate with the same sub-communication group through a same-type communication library based on the communication group information, and communicate with the general sub-communication group through a general communication library, wherein the general communication library refers to a communication library capable of communicating between different types of GPUs; The first computing node is any computing node in the hybrid GPU cluster.

29. The communication device according to claim 28, characterized in that The communication module is used for: In a case where it is determined based on the communication group information that the communication target of the first GPU is the second GPU in the general sub-communication group, sending the communication data of the first GPU to the first CPU in the first computing node, and the first CPU sending the communication data to the second CPU in the second computing node through the general communication library; The first GPU is any GPU in the first computing node, the second computing node is a computing node where the second GPU is located, and the second CPU is used to receive the communication data through the universal communication library and send the communication data to the second GPU.

30. The communication device according to claim 28, characterized in that The communication module is used for: When it is determined based on the communication group information that the communication target of the third GPU is a fourth GPU in the homogeneous sub-communication group, sending the communication data of the third GPU to the fourth GPU through the homogeneous communication library; The third GPU is any GPU in the first computing node.

31. The communication device according to claim 28, characterized in that The information processing module is used to: In a case where it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and includes a GPU that does not conform to the first computing node type, creating first communication group information corresponding to the communication group; The first communication group information is used to indicate that the communication group is a mixed communication group, the GPUs in the communication group that meet the first computing node type constitute a similar sub-communication group, and the GPUs in the communication group that do not meet the first computing node type constitute a general sub-communication group.

32. The communication device according to claim 28, characterized in that The communication group information is also used to indicate a homogeneous communication group in the hybrid GPU cluster, wherein the homogeneous communication group includes a GPU that conforms to the first computing node type; The communication module is further used to communicate with the same communication group through the same communication library based on the communication group information.

33. The communication device according to claim 32, characterized in that: The communication module is used for: When it is determined based on the communication group information that the communication target of the fifth GPU is a sixth GPU in the homogeneous communication group, sending the communication data of the fifth GPU to the sixth GPU through the homogeneous communication library; The fifth GPU is any GPU in the first computing node.

34. The communication device according to claim 32, characterized in that The information processing module is also used for: When it is determined based on the global hardware information that any communication group in the hybrid GPU cluster includes a GPU that conforms to the first computing node type and does not include a GPU that does not conform to the first computing node type, creating second communication group information corresponding to the communication group; The second communication group information is used to indicate that the communication group is the same type of communication group.

35. The communication device according to claim 28, characterized in that The communication module is used for: In response to the collective communication task for the hybrid communication group, performing a collective communication operation matching the collective communication task between the GPUs belonging to the homogeneous sub-communication group in the first computing node through the homogeneous communication library to obtain first collective communication data; The first collective communication data is sent to a third computing node through the general communication library, and a GPU belonging to the general sub-communication group in the third computing node performs a collective communication operation matching the collective communication task based on the first collective communication data to obtain second collective communication data.

36. The communication device according to any one of claims 28 to 35, characterized in that: In the hybrid GPU cluster, the computing power ratio of any two different types of computing nodes is equal to the load ratio of any two different types of computing nodes.

37. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the communication method according to any one of claims 10 to 18.

38. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the communication method according to any one of claims 10 to 18.

39. A computer program product comprising a computer program, characterized in that The computer program is loaded and executed by a processor to implement the operations performed by the communication method according to any one of claims 10 to 18.

Images