Ensemble communication method and related equipment

Abstract

The invention discloses a set communication method, which is applied to a computing device cluster, the computing device cluster comprises a host and an accelerator cluster, the accelerator cluster comprises a plurality of artificial intelligence (AI) accelerators, and the method comprises the following steps: the host obtains topological positions of available AI accelerators in the accelerator cluster and obtains the number of the AI accelerators participating in set communication; when the number of the AI accelerators included in the maximum continuous topology in the topology of the accelerator cluster is smaller than the number of the AI accelerators participating in the set communication, the host determines a plurality of accelerator groups with continuous topologies according to the topology position and the number of the AI accelerators participating in the set communication; each accelerator group in the plurality of accelerator groups executes the set communication operator to obtain a first execution result; the plurality of accelerator groups perform inter-group communication according to the first execution result to obtain a second execution result. In this way, set communication can be adaptively expanded to the topology discontinuous AI accelerator cluster, and the performance of the AI accelerators is fully utilized.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a collective communication method, a computing device cluster, a computer-readable storage medium, and a computer program product. Background Technology

[0002] As artificial intelligence (AI) technology continues to evolve, the scale of models built based on AI technology (i.e., AI models, or simply models) is constantly growing. Taking examples such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformer (GPT) as illustrations, the parameters of these large AI models can reach hundreds of millions or even billions. Such massive amounts of parameters typically require significant computing resources for training and inference. Therefore, in a distributed computing environment, the host can partition the parameters of a large AI model and then distribute them across multiple AI accelerators. Each AI accelerator is responsible for a portion of the parameters of the large AI model.

[0003] During training or inference, multiple AI accelerators can migrate data through Collective Communications to complete corresponding computational tasks. For distributed computing environments comprising multiple AI accelerators, many hardware vendors provide Collective Communications Libraries designed specifically for such environments. These libraries include Collective Communications operators for efficiently handling data exchange in large-scale parallel computing.

[0004] Current ensemble communication operators are mainly applicable to AI accelerator clusters with continuous and regular topologies, and are difficult to adaptively extend to AI accelerator clusters with discontinuous topologies, thus failing to fully utilize the performance of AI accelerators. Summary of the Invention

[0005] This application provides a cascaded communication method that groups a topologically discontinuous AI accelerator cluster into subgroups, and then performs cascaded communication within each topologically contiguous accelerator subgroup. This adaptively extends cascaded communication to AI accelerator clusters with discontinuous topologies, fully utilizing the performance of the AI ​​accelerators. This application also provides a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the above method.

[0006] Firstly, this application provides a collective communication method. This method is applied to a computing device cluster, which includes a host and an accelerator cluster. The accelerator cluster includes multiple AI accelerators. This method supports adaptive application to any group of AI accelerators within the computing device cluster.

[0007] Specifically, the host obtains the topological locations of available AI accelerators in the accelerator cluster, as well as the number of AI accelerators participating in the aggregated communication. If the number of AI accelerators in the largest contiguous topology of the accelerator cluster is less than the number of AI accelerators participating in the aggregated communication, the host determines multiple accelerator groups based on the topological locations and the number of AI accelerators participating in the aggregated communication. Each accelerator group within these multiple groups has a contiguous topology. Then, each accelerator group executes the aggregated communication operator to obtain a first execution result. Next, the multiple accelerator groups perform inter-group communication based on the first execution result to obtain a second execution result.

[0008] This method supports grouping accelerator clusters into multiple topologically contiguous accelerator groups when the available AI accelerator topologies are discontinuous. Within each accelerator group, aggregate communication can occur first, followed by inter-group communication, thus enabling aggregate communication between any group of AI accelerators. This method expands the number of AI accelerators participating in aggregate communication, fully utilizing the performance of AI accelerators.

[0009] In some possible implementations, each of the multiple accelerator groups has at least one AI accelerator interconnected with other accelerator groups. These interconnected AI accelerators can also be referred to as the interconnection boundaries between accelerator groups. Accordingly, when multiple accelerator groups perform inter-group communication, the AI ​​accelerators in each accelerator group interconnected with other accelerator groups can perform aggregate communication based on a first execution result to obtain a second execution result, and then perform aggregate communication within the group to synchronize the second execution result within the accelerator group.

[0010] This method utilizes ensemble communication to achieve inter-group communication of accelerator groups in one step, further improving the efficiency and performance of ensemble communication.

[0011] In some possible implementations, the host can determine the edges of unavailable AI accelerators in the accelerator cluster based on topological location. For accelerator clusters with dual-axis interconnection (e.g., X-axis and Y-axis interconnection), the edges of unavailable AI accelerators can include at least one of the following: left edge, right edge, top edge, or bottom edge. The host can then divide the available AI accelerators in the cluster into multiple sub-regions based on the edges of the unavailable AI accelerators, with each sub-region having a contiguous topology. The host determines multiple accelerator groups based on the multiple sub-regions and the number of AI accelerators participating in the aggregated communication. Each accelerator group corresponds to a sub-region, either obtained by splitting a sub-region or by merging at least one AI accelerator from adjacent sub-regions. The number of AI accelerators included in the multiple accelerator groups is greater than or equal to the number of AI accelerators participating in the aggregated communication.

[0012] In this method, the host divides the AI ​​accelerators into sub-regions along the edges of unavailable AI accelerators to ensure the topological continuity of AI accelerators within the corresponding accelerator groups of the sub-regions, thereby enabling aggregate communication of accelerator clusters with arbitrary topologies.

[0013] In some possible implementations, the host can generate multiple initialization packets based on multiple sub-regions, with each initialization packet corresponding to one sub-region. Accordingly, the host adjusts the multiple initialization packets based on the number of AI accelerators participating in the aggregated communication, combined with a load balancing strategy, and determines multiple accelerator groups from the adjusted packets.

[0014] This method balances the number of AI accelerators in accelerator groups, enabling different accelerator groups to complete communication in similar timeframes, reducing unnecessary waiting time and improving the efficiency of aggregated communication.

[0015] In some possible implementations, the host obtains the link bandwidth and communication data volume between the available AI accelerators in the accelerator cluster. Accordingly, the host determines a first parallelism parameter based on the link bandwidth and communication data volume between the available AI accelerators in the accelerator cluster. This first parallelism parameter includes the amount of data allocated to different links for multiple accelerator groups. Each of the multiple accelerator groups can execute a set communication operator based on the first parallelism parameter.

[0016] This method considers the link characteristics between AI accelerators to divide the communication data volume, avoiding the waste of link bandwidth and other resources due to uneven distribution of communication tasks, and improving resource utilization.

[0017] In some possible implementations, the host determines the execution time of at least one candidate algorithm based on the number of AI accelerators included in each of the multiple accelerator groups and the amount of communication data. Then, based on the execution time of the at least one candidate algorithm, the host determines a second parallel parameter from among the at least one candidate algorithm. This second parallel parameter includes the target number of steps (the number of steps in the target algorithm). Subsequently, each of the multiple accelerator groups can execute a set communication operator according to the second parallel parameter.

[0018] This method predicts the execution time of different parallel algorithms, selects the algorithm with shorter execution time based on the prediction results, and uses the parameters of the algorithm as the second parallel parameter. The accelerator group executes the set communication operator according to the second parallel parameter, which can further shorten the execution time and improve performance.

[0019] In some possible implementations, the host can obtain the initial topology of the accelerator cluster, which indicates the interconnection of AI accelerators when the accelerator cluster is initialized. If the topology of the available AI accelerators in the accelerator cluster remains unchanged relative to the initial topology, the host determines that the topology of the available AI accelerators in the accelerator cluster is a contiguous topology.

[0020] This method provides a reference for subsequent accelerator grouping by determining whether the topology of available AI accelerators in the accelerator cluster is a contiguous topology.

[0021] Secondly, this application provides a clustered communication method. This method is applied to a computing device cluster. The computing device cluster includes a host and an accelerator cluster, and the accelerator cluster includes multiple AI accelerators.

[0022] In practice, multiple AI accelerators receive the topological locations of available AI accelerators in the accelerator cluster, as well as the number of AI accelerators participating in the aggregated communication. At least one AI accelerator among the multiple AI accelerators determines multiple accelerator groups based on the topological locations and the number of AI accelerators participating in the aggregated communication. Each accelerator group executes the aggregated communication operator to obtain a first execution result. The multiple accelerator groups then perform inter-group communication based on the first execution result to obtain a second execution result.

[0023] In this method, AI accelerators in an accelerator cluster can be grouped into multiple topologically contiguous accelerator groups. Within each accelerator group, aggregate communication occurs first, followed by inter-group communication, thus enabling aggregate communication between any group of AI accelerators. Furthermore, this method can expand the number of AI accelerators participating in aggregate communication, rather than being limited to a small number of linearly allocated AI accelerators, thereby fully utilizing the performance of the AI ​​accelerators.

[0024] Thirdly, this application provides a computing device cluster. The computing device cluster includes a host and an accelerator cluster, the accelerator cluster including multiple artificial intelligence (AI) accelerators;

[0025] The host is used to obtain the topological location of the available AI accelerators in the accelerator cluster, and to obtain the number of AI accelerators participating in the aggregate communication; when the number of AI accelerators included in the largest contiguous topology in the topology of the accelerator cluster is less than the number of AI accelerators participating in the aggregate communication, multiple accelerator groups are determined according to the topological location and the number of AI accelerators participating in the aggregate communication, and the topology of each accelerator group in the multiple accelerator groups is a contiguous topology.

[0026] Each of the plurality of accelerator groups is used to execute the set communication operator to obtain a first execution result;

[0027] The multiple accelerator groups are used to perform inter-group communication based on the first execution result to obtain a second execution result.

[0028] In some possible implementations, each of the plurality of accelerator groups has at least one AI accelerator interconnected with other accelerator groups;

[0029] The AI ​​accelerators in each accelerator group that interconnect with other accelerator groups are specifically used for:

[0030] Based on the first execution result, a set communication is performed to obtain the second execution result, and then a set communication is performed within the group to synchronize the second execution result within the accelerator group.

[0031] In some possible implementations, the host is specifically used for:

[0032] Based on the topological location, determine the edges of unavailable AI accelerators in the accelerator cluster;

[0033] Based on the edges of the unavailable AI accelerators in the accelerator cluster, the available AI accelerators in the accelerator cluster are divided into multiple sub-regions, and the topology of each sub-region in the multiple sub-regions is a continuous topology;

[0034] Based on the multiple sub-regions and the number of AI accelerators participating in the aggregated communication, multiple accelerator groups are determined. Each accelerator group in the multiple accelerator groups corresponds to a sub-region, or is obtained by dividing a sub-region, or by merging at least one AI accelerator in an adjacent sub-region. The number of AI accelerators included in the multiple accelerator groups is greater than or equal to the number of AI accelerators participating in the aggregated communication.

[0035] In some possible implementations, the host is specifically used for:

[0036] Based on the multiple sub-regions, multiple initialization groups are generated, and each initialization group corresponds to one sub-region;

[0037] Based on the number of AI accelerators participating in the aggregated communication, and in conjunction with a balancing strategy, the multiple initialization groups are adjusted, and multiple accelerator groups are determined from the adjusted groups.

[0038] In some possible implementations, the host is also used for:

[0039] Obtain the link bandwidth and communication data volume between available AI accelerators in the accelerator cluster;

[0040] A first parallel parameter is determined based on the link bandwidth between available AI accelerators in the accelerator cluster and the amount of communication data. The first parallel parameter includes the amount of data allocated to different links for the multiple accelerator groups.

[0041] Each of the plurality of accelerator groups is specifically used for:

[0042] The set communication operator is executed based on the first parallel parameter.

[0043] In some possible implementations, the host is also used for:

[0044] The time consumption of at least one candidate algorithm is determined based on the number of AI accelerators and the amount of communication data included in each of the multiple accelerator groups.

[0045] Based on the time consumption of the at least one candidate algorithm, a second parallel parameter is determined from the at least one candidate algorithm, the second parallel parameter including the target number of steps;

[0046] Each of the plurality of accelerator groups is specifically used for:

[0047] The set communication operator is executed according to the second parallel parameter.

[0048] In some possible implementations, the host is also used for:

[0049] Obtain the initial topology of the accelerator cluster;

[0050] When the topology of the available AI accelerators in the accelerator cluster remains unchanged relative to the topology of the available AI accelerators in the initial topology, the topology of the available AI accelerators in the accelerator cluster is determined to be a continuous topology.

[0051] Fourthly, this application provides a computing device cluster. The computing device cluster includes a host and an accelerator cluster, the accelerator cluster including multiple artificial intelligence (AI) accelerators;

[0052] The plurality of AI accelerators are used to receive the topological location of the available AI accelerators in the accelerator cluster, and to obtain the number of AI accelerators participating in the aggregate communication.

[0053] At least one of the plurality of AI accelerators is further configured to determine a plurality of accelerator groups based on the topological location and the number of AI accelerators participating in the aggregated communication.

[0054] Each of the plurality of accelerator groups is used to execute the set communication operator to obtain a first execution result;

[0055] The multiple accelerator groups are also used to perform inter-group communication based on the first execution result to obtain a second execution result.

[0056] Fifthly, this application provides a computer-readable storage medium storing instructions that instruct a computing device or a cluster of computing devices to execute the collective communication method described in the first aspect or any implementation thereof.

[0057] In a sixth aspect, this application provides a computer program product containing instructions that, when run on a computing device or a cluster of computing devices, causes the computing device or cluster of computing devices to execute the collective communication method described in the first aspect or any implementation thereof.

[0058] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0059] To more clearly illustrate the technical methods of this application, the accompanying drawings used will be briefly described below.

[0060] Figure 1 A topology diagram of an accelerator cluster provided in this application;

[0061] Figure 2 A hardware structure diagram of a computing device cluster provided in this application;

[0062] Figure 3 A flowchart of a collection communication method provided in this application;

[0063] Figure 4 A schematic diagram of a hierarchical aggregate communication method for an accelerator cluster provided in this application;

[0064] Figure 5 A flowchart of an accelerator grouping method provided in this application;

[0065] Figure 6 A schematic diagram of an accelerator grouping process is provided for this application;

[0066] Figure 7 A flowchart of a ensemble communication method within an accelerator group provided in this application;

[0067] Figure 8 A schematic diagram illustrating a scenario for parallel implementation of set communication within an accelerator group, as provided in this application;

[0068] Figure 9 A flowchart of a collection communication method provided in this application. Detailed Implementation

[0069] The terms "first" and "second" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0070] First, some technical terms involved in the embodiments of this application will be introduced.

[0071] Artificial intelligence (AI) models are mathematical models with reasoning capabilities built using machine learning (ML) and other methods. AI models can be categorized based on their parameter scale; for example, they can be divided into small AI models and large AI models. Large AI models typically have hundreds of millions or even billions of parameters. For instance, the parameters of a bidirectional encoder-representations-from-transformers (BERT) model are usually in the hundreds of millions, while the parameters of a generative pre-trained transformer (GPT) model are typically in the billions, or even hundreds of billions. Small AI models have a smaller parameter scale than large AI models and can typically be deployed on resource-constrained devices, such as smartphones, computers, or other types of terminals. It should be noted that both large and small AI models can be computed using AI accelerator clusters. For ease of description, AI accelerator clusters can also be simply referred to as accelerator clusters or clusters.

[0072] AI accelerators are microprocessors or computing systems specifically designed for hardware acceleration of artificial intelligence (especially artificial neural networks, machine vision, and machine learning). AI accelerators can include graphics processing units (GPUs), neural network processing units (NPUs), or tensor processing units (TPUs). GPUs, NPUs, and TPUs can be collectively referred to as xPUs. xPUs lack Turing completeness and are difficult to run independently; they typically operate with the assistance of a central processing unit (CPU) on the host machine.

[0073] Multiple AI accelerators can form an accelerator cluster to execute computing tasks in parallel. For example, multiple AI accelerators on a single computing device (such as a standalone AI server) can form an accelerator cluster, or multiple AI accelerators on multiple computing devices within a computing device cluster can form an accelerator cluster. As AI accelerators continue to develop, the number of AI accelerators that can be integrated within computing devices and the number of computing devices that can be integrated into computing device clusters are constantly increasing, leading to a continuous increase in the size of accelerator clusters. For example, as a new type of computing device, a supernode can integrate 64 AI accelerators. A supernode refers to a large-scale server that interconnects multiple accelerators via a bus. Compared to ordinary nodes (such as ordinary AI servers), supernodes have higher computing power or storage capacity and stronger network connectivity.

[0074] Accelerator clusters can achieve distributed parallel computing through Collective Communications. Collective Communications is a global communication operation in which all processes in a process group participate. If each AI accelerator in a group runs one process, Collective Communications can be communication in which all AI accelerators in the group participate. Common Collective Communications operators include Broadcast, Gather, Scatter or Reduce, and AllReduce. Broadcast is a one-to-many data synchronization operation, typically used for parameter initialization during data-parallel training to ensure consistent initial parameters across all AI accelerators. Gather is a many-to-one data operation used to collect data from multiple AI accelerators into a single accelerator. Scatter is the reverse of Gather, specifically a one-to-many data operation used to divide the data from one AI accelerator into multiple parts and distribute them across multiple AI accelerators. Reduce is a many-to-one data operation used to reduce data from multiple AI accelerators to a single AI accelerator. AllReduce is a many-to-many data operation used to perform Reduce operations on all AI accelerators.

[0075] As accelerator clusters grow larger, computational tasks typically don't utilize all AI accelerators, but rather a subset of them. In training scenarios, if some AI accelerators fail, the distribution of AI accelerators can become fragmented. For example... Figure 1 As shown, in an accelerator cluster consisting of 4*4 AI accelerators, if the AI ​​accelerators at coordinates (2,2), (2,3), and (3,2) fail, ensemble communication can utilize the AI ​​accelerators at coordinates (3,3), (3,4), (4,3), and (4,4). The number of AI accelerators participating in ensemble communication (ranksize) decreases significantly, and other compliant AI accelerators are not fully utilized. In inference scenarios,

[0076] Because the services provided by the servers to the multiple users are different, the distribution of available AI accelerators is also fragmented.

[0077] To fully utilize available AI accelerators, distributed algorithms are needed to support collective communication between any group of AI accelerators. Currently, many vendors have designed Collective Communications Libraries (CCLs) for distributed computing environments involving multiple AI accelerators. These libraries provide optimized collective communication algorithms for efficiently handling data exchange in massively parallel computing. For example, some CCLs offer the Ring algorithm. The Ring algorithm is a data transmission method based on a ring topology for efficiently transferring data between multiple AI accelerators. In the Ring algorithm, each AI accelerator is both a receiver and a sender of data, transmitting data in a ring-like order. This method allows each AI accelerator to interact with two other AI accelerators simultaneously (one sending, one receiving), thus effectively utilizing bandwidth and reducing communication latency. In large AI model training or inference scenarios, optimization techniques such as the Ring algorithm support efficient parameter and gradient exchange.

[0078] To address the communication needs of accelerator clusters of varying sizes, the number of AI accelerators participating in ensemble communication (ranksize) can be used as an input parameter to the ensemble communication operator. When the accelerator cluster has a continuous topology, such as a fullmesh topology or a CLOS (Clear-Loop Switched Network) topology, different scales of ensemble communication can be achieved by changing the input parameter. Taking a 4x4 AI accelerator cluster as an example, ranksize can be set to 2, 4, 8, or 16 to allocate the appropriate number of AI accelerators for ensemble communication to computing tasks using a linear allocation method, thereby meeting the communication requirements of ensemble communication at different scales.

[0079] In this context, AI accelerators allocated using a linear distribution method typically have continuous and regular topologies; for example, the topology of AI accelerators allocated for computational tasks can be a regular rectangle. However, the ensemble communication operator struggles to adaptively scale to accelerator clusters with discontinuous topologies, thus failing to fully utilize the performance of AI accelerators.

[0080] In view of this, this application provides a flexible aggregated communication method. This aggregated communication method supports adaptive application to any group of AI accelerators. Specifically, for accelerator clusters with discontinuous topologies, when the number of AI accelerators in the largest contiguous topology of the accelerator cluster is less than the number of AI accelerators participating in aggregated communication (ranksize), it indicates that the largest contiguous topology cannot meet the needs of aggregated communication at the current scale. Multiple AI accelerators in the accelerator cluster can be grouped to determine multiple topologically contiguous accelerator groups. Aggregated communication can first be performed within each accelerator group, and then inter-group communication can be performed between accelerator groups, thereby realizing aggregated communication between any group of AI accelerators. Furthermore, when each accelerator group has at least one AI accelerator interconnected with other accelerator groups, i.e., when there are interconnection boundaries between multiple accelerator groups, multiple accelerator groups can perform aggregated communication at the interconnection boundaries to further improve the efficiency and performance of aggregated communication.

[0081] To make the technical solution of this application clearer and easier to understand, the system architecture of this application is described below with reference to the accompanying drawings. The method of this application can be applied to a computing device cluster. The computing device cluster may include a host and an accelerator cluster. It should be noted that the computing device cluster may adopt a single-machine multi-card architecture or a multi-machine multi-card architecture. For example, a computing device cluster may include a single computing device, such as an AI server, which includes a host and multiple AI accelerators. Another example is that a computing device cluster may include multiple computing devices, each including a host and at least one AI accelerator. The multiple computing devices can be networked using Rails; with this networking method, the link bandwidth converges after passing through a switch (SW).

[0082] Figure 2 A schematic diagram of a multi-machine, multi-GPU computing device cluster is shown. The computing device cluster 20 includes multiple computing devices 200, which can be AI servers. The multiple computing devices 200 can be connected via a switch 100. Each computing device 200 includes a host 202 and multiple AI accelerators 204. The host 202 receives computing tasks, such as computing tasks for AI model training or inference scenarios, and schedules these tasks to at least one of the multiple AI accelerators 204 for execution. The AI ​​accelerators 204 execute the computing tasks scheduled by the host 202.

[0083] The host 202 may include a CPU 2022 and host memory 2024. The CPU 2022 can load computing tasks into the host memory 2024 and determine the number of AI accelerators participating in the aggregate communication for that computing task, i.e., determine the ranksize for that computing task. Then, the CPU 2022 can schedule the computing task to n accelerators, where n is greater than or equal to the ranksize.

[0084] AI accelerator 204 is an independent computing unit responsible for executing AI-related computing tasks. AI accelerator 204 may include AI core 2042 and accelerator memory 2044. AI core 2042 is used to access accelerator memory 2044 to execute AI-related computing tasks. Furthermore, AI accelerator 204 may also include a communication engine 2046. Communication engine 2046 may be software deployed on AI accelerator 204, used to enable communication between AI accelerators 204 or between AI accelerator 204 and host 202.

[0085] exist Figure 2 In the example, computing device 200 can be a server with a single-machine multi-GPU architecture, wherein multiple AI accelerators 204 (multiple GPUs) are interconnected by an interconnect link 206, which is used to achieve fast data exchange and synchronization through an efficient communication mechanism. The communication mechanism between the AI ​​accelerators 204 can be a bus protocol, such as the Cache Coherent System bus protocol.

[0086] AI accelerators 204 on different computing devices 200 can be connected via interconnect links 300. Interconnect links 300 are used to connect AI accelerators 204 to switch 100. Each AI accelerator 204 on a different computing device 200 is connected to switch 100 via interconnect links 300, thereby enabling communication between the AI ​​accelerators 204 on different computing devices 200. Interconnect links 300 can employ either the Remote Direct Memory Access over Converged Ethernet (RoCE) protocol or the InfiniBand (IB) protocol to facilitate communication between the AI ​​accelerators 204 and switch 100.

[0087] RoCE is a network protocol that allows RDMA to be implemented over Ethernet. Specifically, RoCE allows computing devices 200 to directly exchange data in each other's memory, thereby bypassing the operating system, reducing latency and CPU load, and improving network communication efficiency. IB is a high-performance network communication technology primarily used in supercomputers and data centers. IB is designed to support high-bandwidth and low-latency data transmission, enabling efficient inter-server communication by supporting RDMA and other advanced network functions.

[0088] The AI ​​accelerators 204 can form different topologies through different connection methods. For example, if each AI accelerator 204 is connected to all other AI accelerators 204, the topology of the accelerator cluster formed by the AI ​​accelerators 204 is a full-mesh topology. Another example is that the AI ​​accelerators 204 are arranged in a two-dimensional mesh, with each AI accelerator 204 directly connected to its four nearest neighbor nodes (the AI ​​accelerators 204 in the top, bottom, left, and right positions), then the topology of the accelerator cluster formed by the AI ​​accelerators 204 is a 2-dimensional (2D) mesh. In some examples, the AI ​​accelerators 204 are arranged in a three-dimensional torus mesh, with each AI accelerator 204 connected to its adjacent AI accelerators 204, forming a closed loop; correspondingly, the topology of the accelerator cluster formed by the AI ​​accelerators 204 is a 3D torus.

[0089] It should be noted that, Figure 2 The example illustrates that the computing device cluster 20 includes multiple computing devices 200. In actual applications, the computing device cluster 20 may also include a single computing device 200. In other words, the aggregated communication method of this application can also be applied to a single-machine multi-card architecture computing device 200.

[0090] based on Figure 2 The computing device cluster 20 shown in this application also provides a collective communication method. The collective communication method of this application will be described in detail below with reference to the accompanying drawings.

[0091] See Figure 3 The flowchart shown illustrates a collective communication method applied to a computing device cluster 20. The computing device cluster 20 includes a host 202 and an accelerator cluster built based on AI accelerators 204. The accelerator cluster includes multiple AI accelerators 204. The method includes the following steps:

[0092] S302, Host 202 obtains the topology location of the available AI accelerator 204 in the accelerator cluster.

[0093] The AI ​​accelerators 204 in the accelerator cluster can form different topologies depending on the connection method. Different AI accelerators 204 can be characterized by their topological positions. These topological positions can be represented by the coordinates of the AI ​​accelerator 204 in a coordinate system. For example, when the topology of the AI ​​accelerator is a 2D mesh, the horizontal and vertical directions can be the x-axis (or X-axis) and y-axis (or Y-axis) of the coordinate system, respectively. Correspondingly, the topological position of the AI ​​accelerator 204 can be represented as (X, Y). Here, X can take values ​​from 1 to k (inclusive of endpoint values), where k is the number of AI accelerators 204 included in the horizontal direction, and Y can take values ​​from 1 to j (inclusive of endpoint values), where j is the number of AI accelerators 204 included in the vertical direction.

[0094] An available AI accelerator refers to an AI accelerator that is operating normally, capable of receiving and executing new computing tasks. An unavailable AI accelerator refers to an AI accelerator that is not operating normally, such as one that has malfunctioned. The host 202 can periodically send heartbeat signals to multiple AI accelerators 204 to detect whether each AI accelerator 204 is operating normally. If the host 202 does not receive a heartbeat response from an AI accelerator 204 for m consecutive cycles, it determines that the AI ​​accelerator 204 is unavailable. Here, m can be set empirically, for example, m can be set to 3. The host 202 can determine the topology location of available AI accelerators based on the topology location of unavailable AI accelerators 204. In some possible implementations, the host 202 can also directly determine the topology location of available AI accelerators based on the reception of heartbeat responses. For example, if the host 202 receives a heartbeat response from an AI accelerator 204 for m consecutive cycles, it indicates that the AI ​​accelerator 204 is available, and its topology location can be obtained.

[0095] To facilitate understanding, this application also provides an example for illustration. Figure 4The diagram illustrates a hierarchical aggregated communication model. In this example, the AI ​​accelerator cluster includes 6*6 AI accelerators 204. The host 202 determines the topology location of the unavailable AI accelerators 204 as {(3,3),(3,4)} through heartbeat detection. Correspondingly, the topology location of the available AI accelerators 204 can be the topology location of the 6*6 AI accelerators 204 minus the topology location of the unavailable AI accelerators 204 as {(3,3),(3,4)}, which is the remaining topology location. In this example, the topological location of the available AI accelerator 204 could be {(1,1),(1,2),(1,3),(1,4),(1,5),(1,6),(2,1),(2,2),(2,3),(2,4),(2,5),(2,6),(3,1),(3,2),(3,5),(3,6),(4,1),(4,2),(4,3),(4,4),(4,5),(4,6),(5,1),(5,2),(5,3),(5,4),(5,5),(5,6),(6,1),(6,2),(6,3),(6,4),(6,5),6,6)}.

[0096] S304, Host 202 obtains the number of AI accelerators 204 participating in the aggregate communication.

[0097] The number of AI accelerators participating in the aggregated communication (denoted as ranksize) refers to the number of AI accelerators requested by host 202 for the computing task. Specifically, upon receiving a computing task, host 202 can determine the ranksize based on the computational load of the task. Furthermore, host 202 can also consider the load of AI accelerators 204 in the AI ​​accelerator cluster when determining the ranksize. For example, if many AI accelerators 204 in the AI ​​accelerator cluster have a high load, a smaller ranksize can be set.

[0098] S306, Host 202 determines whether the number of AI accelerators in the largest contiguous topology of the accelerator cluster is less than the number of AI accelerators participating in the aggregate communication. If yes, proceed to S308; otherwise, proceed to S314.

[0099] A continuous topology, also known as a homogeneous topology, refers to a topology where nodes (such as AI accelerators 204) have the same function. For example, in a fully interconnected topology, all AI accelerators 204 have the same function; therefore, the fully interconnected topology is a continuous topology. For an accelerator cluster, the largest continuous topology refers to the topology of the subset of the accelerator cluster that is topologically continuous and includes the largest number of AI accelerators 204. The subset of the accelerator cluster can include the accelerator cluster itself and other subsets besides the cluster itself. For example, when the accelerator cluster includes 64 AI accelerators 204, the subset of the accelerator cluster can include the accelerator cluster itself and subsets consisting of fewer than 64 AI accelerators 204.

[0100] In specific implementation, host 202 can first determine whether the topology of the available AI accelerators in the accelerator cluster is a contiguous topology. Host 202 can obtain the initial topology of the accelerator cluster, which indicates the interconnection relationship of the AI ​​accelerators 204 when the accelerator cluster is initialized. When the topology of the available AI accelerators in the accelerator cluster remains unchanged relative to the topology of the available AI accelerators in the initial topology, the host determines that the topology of the available AI accelerators in the accelerator cluster is a contiguous topology. Typically, for an accelerator cluster with a contiguous initial topology, unavailable AI accelerators are equal to one or more homogeneous interconnect domains. For example, if the topology of unavailable AI accelerators is a contiguous topology, then the available AI accelerators maintain their initial interconnection relationship. For example, in a fully interconnected accelerator cluster, if all AI accelerators on one AI accelerator board are unavailable, the interconnection relationship between the remaining available AI accelerators is unaffected. The topology of the remaining available AI accelerators is a contiguous topology. If the unavailable AI accelerators are not equal to one or more homogeneous interconnect domains, then the topology of the remaining available AI accelerators in the accelerator cluster is a non-contiguous topology.

[0101] If the topology of available AI accelerators in the accelerator cluster is a contiguous topology, then the maximum contiguous topology is the topology of available AI accelerators in the accelerator cluster. In this case, the number of AI accelerators 204 included in the maximum contiguous topology is equal to the number of available AI accelerators 204 in the accelerator cluster. When this number is greater than or equal to ranksize, S314 can be executed to directly perform cluster communication within the accelerator cluster. Otherwise, it indicates that the topology of available AI accelerators in the accelerator cluster is a non-contiguous topology, and host 202 can determine the maximum contiguous topology from the topology of the available subset of the accelerator cluster. The available subset includes one or more AI accelerators 204 among the available AI accelerators. For example, Figure 1 The largest contiguous topology of the accelerator cluster in the data includes 2x2 AI accelerators (204). For example, Figure 4The maximum contiguous topology of the accelerator cluster includes 6*3 AI accelerators 204. Then, host 202 can compare the number of AI accelerators included in the maximum contiguous topology with ranksize. When the number is greater than or equal to ranksize, S314 can be executed; when the number is less than ranksize, S308 can be executed.

[0102] S308 and host 202 determine multiple accelerator groups based on topological location and the number of AI accelerators 204 participating in aggregated communication.

[0103] The topology of each accelerator group (also called a subgroup) is a contiguous topology. Specifically, host 202 can group the available AI accelerators 204 in the accelerator cluster according to the topology location and ranksize to determine the multiple accelerator groups used to execute the current computing task. It should be noted that the topology of the multiple accelerator groups used to execute the computing task is a contiguous topology, which can be a regularly shaped topology, such as a rectangular topology. The number of AI accelerators included in the multiple accelerator groups used to execute the current computing task can be greater than or equal to ranksize. When the number of AI accelerators included in the multiple accelerator groups used to execute the current computing task is greater than ranksize, the excess AI accelerators can be used as backup nodes or disaster recovery nodes to achieve rapid fault recovery.

[0104] Furthermore, each of the multiple accelerator groups has at least one AI accelerator 204 interconnected with other accelerator groups. In other words, there are interconnection boundaries between the accelerator groups, which refer to the AI ​​accelerators 204 connecting different accelerator groups. Figure 4 For example, AI accelerator 204 at topological position (2,1) in accelerator group 1 is interconnected with AI accelerator 204 at topological position (3,1) in accelerator group 3; AI accelerator 204 at topological position (2,3) in accelerator group 1 is interconnected with AI accelerator 204 at topological position (4,3) in accelerator group 4; AI accelerator 204 at topological position (4,1) in accelerator group 1 is interconnected with AI accelerator 204 at topological position (5,1) in accelerator group 2; and AI accelerator 204 at topological position (4,2) in accelerator group 3 is interconnected with AI accelerator 204 at topological position (4,3) in accelerator group 4.

[0105] S310. Each of the multiple accelerator groups executes the set communication operator to obtain the first execution result.

[0106] The topology of the accelerator group is a continuous topology, and the set communication operator can be executed directly in the accelerator group. Based on this, the accelerator group can use the number of AI accelerators 204 included in the accelerator group as input parameters to execute the set communication operator and obtain the first execution result.

[0107] by Figure 4 For example, accelerator group 1 can execute the set communication operator with 8 (or 4*2) as input parameters to obtain the first execution result; accelerator group 2 can execute the set communication operator with 6 as input parameters to obtain the first execution result; accelerator group 3 can execute the set communication operator with 8 (or 2*4) as input parameters to obtain the first execution result; and accelerator group 4 can execute the set communication operator with 12 as input parameters to obtain the first execution result. Here, the first execution result refers to the execution result obtained by set communication within the accelerator group, and the first execution result obtained by different accelerator groups may be different.

[0108] S312. Multiple accelerator groups perform inter-group communication based on the first execution result to obtain the second execution result.

[0109] Specifically, multiple accelerator groups can interact with each other's first execution results to perform inter-group communication and obtain a second execution result. For example, when the set communication operator is Broadcast, each accelerator group can send the first execution result obtained from the set communication within its group to other accelerator groups.

[0110] In some possible implementations, when each of multiple accelerator groups has at least one AI accelerator 204 interconnected with other accelerator groups, the AI ​​accelerators interconnected with other accelerator groups in each accelerator group perform collective communication based on a first execution result to obtain a second execution result. Specifically, the AI ​​accelerators 204 interconnected with other accelerator groups in each accelerator group can form a new accelerator cluster / group. The new accelerator cluster / group performs collective communication based on the first execution result to obtain a second execution result. Then, the AI ​​accelerators interconnected with other accelerator groups in each accelerator group (e.g., AI accelerators in the new accelerator cluster) perform intra-group collective communication to synchronize the second execution result within the accelerator group.

[0111] See in some examples Figure 4The AI ​​accelerator 204 at topology position (2,1) in accelerator group 1, the AI ​​accelerator 204 at topology position (3,1) in accelerator group 3, the AI ​​accelerator 204 at topology position (2,3) in accelerator group 1, the AI ​​accelerator 204 at topology position (4,3) in accelerator group 4, the AI ​​accelerator 204 at topology position (4,1) in accelerator group 1, the AI ​​accelerator 204 at topology position (5,1) in accelerator group 2, the AI ​​accelerator 204 at topology position (4,2) in accelerator group 3, and the AI ​​accelerator 204 at topology position (4,3) in accelerator group 4 perform group communication based on the first execution result to obtain the second execution result. Then, AI accelerator 204 at topological position (2,1) in accelerator group 1 can broadcast the second execution result within accelerator group 1; AI accelerator 204 at topological position (5,1) in accelerator group 2 can broadcast the second execution result within accelerator group 2; AI accelerator 204 at topological position (3,1) in accelerator group 3 can broadcast the second execution result within accelerator group 3; and AI accelerator 204 at topological position (4,3) in accelerator group 4 can broadcast the second execution result within accelerator group 4.

[0112] S314. The accelerator cluster executes the set communication operator and obtains the execution result.

[0113] Specifically, the topology of the accelerator cluster is a continuous topology, and the set communication operator can be executed directly on the accelerator cluster. Based on this, the accelerator cluster can execute the set communication operator with the number of available AI accelerators 204 in the accelerator cluster as input parameter and obtain the execution result.

[0114] It should be noted that S306 and S314 above are optional steps in the embodiments of this application. Regardless of whether the topology of the accelerator cluster is continuous, the accelerator cluster can be directly grouped to obtain multiple accelerator groups with continuous topology. In this way, the aggregated communication can be converted into aggregated communication performed on multiple accelerator groups with continuous topology.

[0115] Based on the above description, this application provides a grouped aggregate communication method for arbitrary discontinuous topologies. The topology of available AI accelerators 204 in an accelerator cluster can be continuous or discontinuous. For discontinuous accelerator clusters, when the number of AI accelerators 204 included in the largest continuous topology of the accelerator cluster is less than the ranksize, multiple available AI accelerators 204 in the accelerator cluster can be grouped to determine multiple topologically continuous accelerator groups. Aggregate communication can be performed within each accelerator group first, and then inter-group communication can be performed between accelerator groups, thereby achieving aggregate communication between any group of AI accelerators, offering high flexibility. Moreover, this method can fully utilize the remaining AI accelerators 204 even when there are unavailable AI accelerators 204 in the accelerator cluster, maximizing the ranksize and improving resource utilization. In training scenarios, this method can reduce the fault radius; in inference scenarios, it can improve resource utilization.

[0116] Corresponding to the aforementioned hierarchical aggregate communication method, this application provides a hierarchical aggregate communication operator for arbitrary, non-contiguous topologies. By obtaining the topology of available AI accelerators 204 and the number of AI accelerators 204 participating in aggregate communication (e.g., ranksize), the irregular non-contiguous topology is divided into several regular contiguous topologies (or subgroups). Aggregate communication is run within each subgroup, and then the aggregate communication algorithm is run on the interconnection boundaries between groups, thereby realizing aggregate communication for the entire computing device cluster.

[0117] based on Figure 3 In an embodiment, this application also provides an accelerator grouping method for grouping available AI accelerators in an accelerator cluster.

[0118] See Figure 5 The flowchart shown illustrates an accelerator grouping method, which includes the following steps:

[0119] S502 and host 202 determine the edge of the unavailable AI accelerator 204 in the accelerator cluster based on the topological location of the available AI accelerator 204.

[0120] Specifically, host 202 obtains the topological location of unavailable AI accelerators 204 in the accelerator cluster based on the topological location of available AI accelerators 204. The topological location can be represented by coordinates in a coordinate system. Then, host 202 can determine the edge of the unavailable AI accelerator 204 based on its topological location. For a dual-axis interconnected (e.g., X-axis interconnected in pairs, Y-axis interconnected in pairs) accelerator cluster, the edge of the unavailable AI accelerator 204 can include at least one of the following: left edge, right edge, top edge, or bottom edge.

[0121] S504 and host 202 divide the available AI accelerators 204 in the accelerator cluster into multiple sub-regions based on the edges of the unavailable AI accelerators 204 in the accelerator cluster.

[0122] Within this system, the topology of each of the multiple sub-regions is a continuous topology. Specifically, host 202 can divide the available AI accelerators 204 in the accelerator cluster into multiple sub-regions based on the edges of unavailable AI accelerators 204 and the boundaries of the accelerator cluster. For example, host 202 can extend the edges to intersect with the boundaries of the accelerator cluster, thereby dividing the available AI accelerators into multiple sub-regions.

[0123] The following examples are illustrated with reference to the accompanying drawings. See also... Figure 6 The diagram shown is a schematic of an accelerator grouping, as follows: Figure 6 As shown in (a), the unavailable AI accelerators are located at (3,3) and (3,4). Extending the top and bottom edges of the unavailable AI accelerators to the left and right boundaries of the accelerator cluster, and extending the left and right edges of the unavailable AI accelerators to the top and bottom boundaries of the accelerator set, divides the cluster into 9 sub-regions. The sub-region containing the AI ​​accelerators at topological locations (3,3) and (3,4) is the unavailable sub-region, and the remaining sub-regions are the available sub-regions. The topology of each sub-region is a regular, continuous topology, such as a rectangular topology. Therefore, the topological locations of the top-left and bottom-right corners can be used to identify the sub-region. For example, the available sub-regions are, in order: {(1,1),(2,2)},{(1,3),(2,4)},{(1,5),(2,6)},{(3,1),(3,2)},{(3,5),(3,6)},{(4,1),(6,2)},{(4,3),(6,4)},{(4,5),(6,6)}. Host 202 processes the available sub-regions to determine accelerator groupings.

[0124] To enable aggregated communication among accelerator clusters with arbitrary topologies, the basic grouping principle for AI accelerator groups in host 202 is topological continuity of AI accelerators within an accelerator group. Based on this, host 202 can divide the accelerator group into sub-regions along the edges of unavailable AI accelerators 204 to ensure topological continuity of AI accelerators 204 within the corresponding sub-regions.

[0125] S506 and host 202 determine multiple accelerator groups based on multiple sub-regions and the number of AI accelerators participating in the aggregated communication.

[0126] Each accelerator group in the multiple accelerator groups corresponds to a sub-region, either obtained by dividing a sub-region or by merging at least one AI accelerator from adjacent sub-regions. The number of AI accelerators included in the multiple accelerator groups is greater than or equal to the number of AI accelerators participating in the aggregated communication.

[0127] In some possible implementations, host 202 merges at least two sub-regions from multiple sub-regions, and obtains accelerator packets based on the merged sub-regions. This reduces the number of accelerator packets and enables pairwise interconnection of accelerator packets. For example... Figure 6 As shown in (b), host 202 can merge subregions {(1,1),(2,2)} with subregions {(1,3),(2,4)}, merge subregions {(1,5),(2,6)} with subregions {(3,5),(3,6)}, merge subregions {(3,1),(3,2)} with subregions {(4,1),(6,2)}, and merge subregions {(4,3),(6,4)} with subregions {(4,5),(6,6)}, obtaining four merged subregions. Each merged subregion corresponds to one accelerator group. This determines four accelerator groups. It should be noted that... Figure 6 (b) is merely one example of subregion merging. In other possible implementations of this application, subregion merging can also be performed using other merging methods.

[0128] In other possible implementations, considering load balancing among the accelerator groups, host 202 can also combine a load balancing strategy to determine multiple accelerator groups, thereby making the number of AI accelerators 204 in each accelerator group as balanced as possible. Specifically, host 202 generates multiple initialization groups based on multiple sub-regions, with each initialization group corresponding to one sub-region. Then, host 202 adjusts the multiple initialization groups based on the number of AI accelerators participating in the aggregated communication, combined with a load balancing strategy, and determines multiple accelerator groups from the adjusted groups.

[0129] Still with Figure 6 Example illustration, such as Figure 6As shown in (c), host 202 can merge the AI ​​accelerators in the left column of subregion {(1,1),(2,2)} and subregion {(1,3),(2,4)}, merge the AI ​​accelerators in the right column of subregion {(1,5),(2,6)} and subregion {(1,3),(2,4)}, merge subregion {(3,1),(3,2)} and subregion {(4,1),(6,2)}, and merge subregion {(3,5),(3,6)} and subregion {(4,5),(6,6)}, thereby adjusting multiple initial groups to obtain adjusted groups. The number of AI accelerators 204 in the adjusted groups is close, which can be used as multiple accelerator groups for performing computing tasks.

[0130] It should be noted that host 202 can select accelerator groups for topological edge interconnection from different grouping schemes. This enables inter-group communication to be achieved through aggregated communication at the edge of the two interconnected accelerator groups, thereby completing inter-group communication in one step and improving communication efficiency.

[0131] Furthermore, when the ensemble communication algorithm is used in parallel, it usually divides the data equally according to the ranksize, so that the data can be transmitted in parallel between different AI accelerators 204. Considering that there may be differences in link bandwidth and communication data volume between AI accelerators 204, this application also supports optimizing data grouping by combining link bandwidth and communication data volume to avoid wasting resources such as link bandwidth.

[0132] Specifically, host 202 obtains the link bandwidth and communication data volume between available AI accelerators in the accelerator cluster, and then determines a first parallel parameter based on the link bandwidth and communication data volume between available AI accelerators in the accelerator cluster. The first parallel parameter is a data parallel parameter, including the data volume allocated to different links of multiple accelerator groups. Accordingly, each accelerator group in the multiple accelerator groups can execute a set communication operator according to the first parallel parameter during set communication.

[0133] When performing aggregated communication, the aggregated communication operator can be implemented using different parallel algorithms. Based on this, this application also supports using different parallel algorithms as candidate algorithms, and determining the target algorithm used by the aggregated communication operator in parallel by evaluating the execution time of the candidate algorithms. Specifically, the host determines the execution time of at least one candidate algorithm based on the number of AI accelerators 204 included in each of the multiple accelerator groups and the amount of communication data. Based on the execution time of the at least one candidate algorithm, the host determines a second parallel parameter from the at least one candidate algorithm. The second parallel parameter includes the target algorithm used by the aggregated communication operator in parallel. Accordingly, each of the multiple accelerator groups executes the aggregated communication operator according to the second parallel parameter. Further, when using a pipelined parallel mode, this application also supports evaluating the execution time of different algorithm steps to determine the target number of steps. Based on this, the second parallel parameter determined by this application may also include the target number of steps.

[0134] Based on the above, this application provides a method for aggregated communication within an accelerator group. See also Figure 7 The flowchart shown is a method for aggregated communication within an accelerator group. The method includes the following steps:

[0135] S702 and Host 202 obtain the link bandwidth and communication data volume between available AI accelerators in the accelerator cluster.

[0136] Link bandwidth can include link bandwidth in different directions, such as link bandwidth along the X-axis and link bandwidth along the Y-axis. Specifically, AI accelerators 204 along the X-axis can be interconnected in pairs, and AI accelerators along the Y-axis can also be interconnected in pairs. The amount of communication data can also be referred to as the input length. For example, in a model training scenario, the amount of communication data can be the length of the training set.

[0137] S704 and host 202 determine the first parallel parameter based on the link bandwidth and communication data volume between available AI accelerators in the accelerator cluster.

[0138] Specifically, host 202 can determine the bandwidth ratio of links in different directions based on the link bandwidth between available AI accelerators in the accelerator cluster, and then allocate data according to this bandwidth ratio to obtain a first parallel parameter. The first parallel parameter can include the amount of data allocated to different links in multiple accelerator groups. For example, the first parallel parameter can include the amount of data allocated in the X-axis direction and the amount of data allocated in the Y-axis direction within an accelerator group.

[0139] S706, host 202 determines the time consumption of at least one candidate algorithm based on the number of AI accelerators included in each accelerator group and the amount of communication data in the multiple accelerator groups.

[0140] The host 202, based on the number of AI accelerators and the amount of communication data included in each of the multiple accelerator groups, evaluates the execution time of at least one candidate algorithm using a theoretical communication bandwidth algorithm formula. The candidate algorithms can include parallel algorithms with varying numbers of steps, such as 2-step, 3-step, 4-step, and 6-step parallel algorithms. The host 202 can evaluate the execution time of parallel algorithms with varying numbers of steps.

[0141] S708, host 202 determines the second parallel parameter from at least one candidate algorithm based on the time consumption of at least one candidate algorithm.

[0142] The second parallel parameter includes the target algorithm, and further, the second parallel parameter may also include the target number of steps. Specifically, the host 202 can select the candidate algorithm with the shortest execution time as the target algorithm and select the number of steps with the shortest execution time as the target number of steps. Alternatively, the host 202 can select an algorithm with a execution time less than a threshold as the target algorithm and select the number of steps with a execution time less than the threshold as the target number of steps.

[0143] S710 and host 202 execute the set communication operator according to the first parallel parameter and the second parallel parameter.

[0144] Specifically, the host 202 can use the first parallel parameter and the second parallel parameter as input parameters to execute the set communication operator in the accelerator group, thereby realizing set communication within the accelerator group.

[0145] and Figure 7 Corresponding to the intra-group set communication method, this application also provides a scalable set communication operator for continuous topologies (i.e., isomorphic topologies). The scalable set communication operator can design the optimal grouping scheme based on the interconnection topology between AI accelerators, link type (e.g., characterized by link bandwidth), or communication requirements (e.g., communication data volume), including designing parameters for data parallelism and pipelined parallelism. The parameters for pipelined parallelism may include, but are not limited to, the number of algorithm steps.

[0146] It should be noted that the set communication method of this application can be used not only for the set communication operators listed above, but also for combinations of the set communication operators mentioned above, and thus can be used for all set communication algorithms.

[0147] To make the technical solution of this application clearer and easier to understand, this application also provides an example illustrating intra-group aggregation communication in accelerator packets. See [link to relevant documentation]. Figure 8 The diagram illustrates a scenario where parallel communication is achieved within an accelerator group. This scenario can be a distributed training or inference scenario, where multiple AI accelerators 204 need to synchronize and communicate data. Specifically, gradient synchronization or model parameter synchronization between multiple AI accelerators 204 is achieved through AllReduce.

[0148] In this example, the link bandwidth and communication data volume between the available AI accelerators 204 in the accelerator cluster obtained by host 202 are as follows:

[0149] a) X-axis link bandwidth: 56 gigabytes per second per port, interconnected in pairs;

[0150] b) Y-axis link bandwidth: 112GB / s per port, interconnected in pairs;

[0151] c) Communication data volume or input length, 10 megabytes per second per port.

[0152] Host 202 determines the proportion of data packets based on the bandwidth ratio of links in different directions. For example, if the bandwidth ratio of the links in the X-axis direction and the Y-axis direction is 1:2, in order to make the completion time of one communication step the same or close, host 202 can determine that the data ratio in the X-axis direction and the data ratio in the Y-axis direction is 1:2. Based on this, the amount of data in the vertical parallel direction is S1 = S / 3, and the amount of data in the horizontal parallel direction is S2 = 2*S / 3, where S is the amount of communication data.

[0153] The host 202 can also determine the parallel algorithm and number of steps for executing AllReduce based on the number of AI accelerators 204 in the accelerator group and the amount of communication data S. Candidate algorithms can include parallel algorithms with different numbers of steps, such as 2-step, 3-step, 4-step, or 6-step parallel algorithms. The host 202 can evaluate the time consumption of multiple candidate algorithms based on the number of AI accelerators 204 in the accelerator group and the amount of communication data, using a theoretical communication bandwidth algorithm formula. This time consumption can be the theoretical time consumption. A 2-step parallel algorithm can include ReduceScatter + AllGather, and a 3-step parallel algorithm can include ReduceScatter + Allreduce + AllGather. The following example illustrates the time consumption evaluation process using a 3-step parallel algorithm.

[0154] like Figure 8As shown, the execution process of the 3-step parallel algorithm can be as follows: First, for the first set of data (data size S1), the accelerator group executes ReduceScatter in the X plane, and for the second set of data (data size S2), the accelerator group executes ReduceScatter in the Y plane. The ReduceScatter operations in the X and Y planes can be performed simultaneously. Second, based on the execution result of the first set of data in the previous step, the accelerator group executes AllReduce in the Y plane, and executes AllReduce in the X plane based on the execution result of the second set of data in the previous step. Third, based on the execution result of the first set of data in the previous step, the accelerator group executes AllGather in the X plane, and executes AllGather in the Y plane based on the execution result of the second set of data in the previous step.

[0155] The completion time of the first data set in the X-plane ReduceScatter is S1 / N_X / BW_X, the completion time of the Y-plane Allreduce is S1 / N_X / BW_Y, and the completion time of the X-plane AllGather is S1 / N_X / BW_X. Here, BW_X is the link bandwidth in the X-axis direction, BW_Y is the link bandwidth in the Y-axis direction, N_X is the number of accelerators in the X-axis direction, and N_Y is the number of accelerators in the Y-axis direction. Accordingly, the overall completion time of the first data set can be: T1 = S1 / N_X / BW_X + S1 / N_X / BW_Y + S1 / N_X / BW_X.

[0156] The completion time of the second dataset in the Y-plane ReduceScatter is S2 / N_Y / BW_Y, the completion time of the X-plane AllReduce is S2 / N_Y / BW_X, and the completion time of the Y-plane AllGather is S2 / N_Y / BW_Y. The overall completion time of the second dataset is: T2 = S2 / N_Y / BW_Y + S2 / N_Y / BW_X + S2 / N_Y / BW_Y.

[0157] Based on this, the time taken by the above three-step parallel algorithm can be the maximum value between T1 and T2, i.e., max(T1, T2).

[0158] Then, host 202 can determine the target algorithm based on the time consumption of multiple candidate algorithms. In this example, the accelerator group includes 3*3 AI accelerators 204, and the communication data volume is 10MB. Host 202 determines that the time consumption of the 3-step parallel algorithm is the minimum by using the theoretical communication bandwidth algorithm formula. Therefore, host 202 determines the target algorithm as the above-mentioned 3-step parallel algorithm.

[0159] In the aggregated communication of multi-axis interconnected accelerator clusters, data grouping based on link bandwidth and communication data volume, along with the selection of appropriate parallel algorithms or steps, can fully utilize bandwidth resources, improve parallel efficiency, and significantly reduce communication latency. Related data shows that in some cases, communication latency can be reduced by 20%. For situations where a single AI accelerator in a computing device cluster is unavailable (e.g., 204), the layered communication method described in this application can significantly improve AI accelerator utilization; related experimental data shows that AI accelerator utilization can be increased by 11%.

[0160] It should be noted that this application can automatically modify the number of AI accelerators 204 in the X or Y dimension based on the input parameters, or the accelerator grouping can include only one dimension. In this case, Figure 7 , Figure 8 The ensemble communication method within the accelerator group shown still applies.

[0161] The above provides a detailed explanation of how host 202 and AI accelerator 204 work together to implement hierarchical collection communication. In some possible implementations, hierarchical collection communication can also be implemented by the AI ​​accelerator side.

[0162] See Figure 9 The flowchart shown illustrates a collective communication method applied to a computing device cluster 20, which includes a host 202 and an accelerator cluster. The accelerator cluster includes multiple AI accelerators 204. The method includes:

[0163] S902, Multiple AI Accelerators 204 receive the topology location of available AI accelerators in the accelerator cluster.

[0164] Specifically, multiple AI accelerators 204 can receive the topology locations of available AI accelerators in the accelerator cluster from the host 202. Alternatively, each AI accelerator 204 can send a heartbeat signal to the other AI accelerators 204 to detect whether an AI accelerator 204 is available, thereby obtaining the topology locations of available AI accelerators in the accelerator cluster.

[0165] S904, Multiple AI Accelerators 204: Obtain the number of AI accelerators participating in the aggregate communication.

[0166] Specifically, upon receiving the computation task, host 202 can determine the number of AI accelerators (ranksize) participating in the aggregated communication. The specific implementation of host 202 determining ranksize can be found in [reference needed]. Figure 3 The illustrated embodiment is described below. Multiple AI accelerators 204 can obtain the ranksize sent by the host 202.

[0167] S906. At least one of the multiple AI accelerators determines multiple accelerator groups based on topological location and the number of AI accelerators participating in the aggregated communication.

[0168] In this process, at least one of the multiple AI accelerators 204 can determine the edges of unavailable AI accelerators in the accelerator cluster based on topological location. Then, based on the edges of the unavailable AI accelerators 204, the available AI accelerators 204 in the accelerator cluster are divided into multiple sub-regions, with each sub-region having a contiguous topology. Next, at least one AI accelerator 204 determines multiple accelerator groups based on the multiple sub-regions and the number of AI accelerators participating in the aggregated communication. Each accelerator group corresponds to one sub-region, either obtained by splitting a sub-region or by merging at least one AI accelerator from adjacent sub-regions. The number of AI accelerators included in the multiple accelerator groups is greater than or equal to the number of AI accelerators participating in the aggregated communication, ranksize.

[0169] It should be noted that the specific implementation of at least one AI accelerator 204 determining multiple accelerator groups can refer to the specific implementation of host 202 determining multiple accelerator groups, and will not be repeated here. When an AI accelerator 204 determines multiple accelerator groups, it can broadcast the group information to other AI accelerators 204.

[0170] S908, each of the multiple accelerator groups executes the set communication operator to obtain the first execution result.

[0171] S910, Multiple accelerator groups perform inter-group communication based on the first execution result to obtain a second execution result.

[0172] For specific implementation details of S908 and S910, please refer to [reference needed]. Figure 3 The relevant descriptions of the embodiments shown will not be repeated here.

[0173] Based on the aforementioned aggregated communication method, this application also provides a computing device cluster 20. The computing device cluster 20 of this application will be described below with reference to the accompanying drawings.

[0174] See Figure 2 The diagram shows a structural schematic of a computing device cluster 20, which includes a host 202 and multiple AI accelerators 204. The multiple AI accelerators 204 constitute an accelerator cluster. This accelerator cluster can be composed of multiple AI accelerators 204 in a single-machine, multi-GPU computing device, or multiple AI accelerators 204 in a multi-machine, multi-GPU computing device cluster.

[0175] Host 202 is used to obtain the topological location of available AI accelerators 204 in the accelerator cluster, and to obtain the number of AI accelerators 204 participating in the aggregate communication; when the number of AI accelerators 204 included in the largest contiguous topology in the topology of the accelerator cluster is less than the number of AI accelerators 204 participating in the aggregate communication, multiple accelerator groups are determined based on the topological location and the number of AI accelerators participating in the aggregate communication, and the topology of each accelerator group in the multiple accelerator groups is a contiguous topology;

[0176] Each of the multiple accelerator groups is used to execute the set communication operator to obtain the first execution result;

[0177] Multiple accelerators are grouped together to perform inter-group communication based on the first execution result and obtain a second execution result.

[0178] In some possible implementations, each of the multiple accelerator groups has at least one AI accelerator that is interconnected with other accelerator groups;

[0179] The AI ​​accelerators 204, which interconnect with other accelerator groups within each accelerator group, are specifically used for:

[0180] Based on the first execution result, a set communication is performed to obtain the second execution result, and then a set communication is performed within the group to synchronize the second execution result within the accelerator group.

[0181] In some possible implementations, host 202 is specifically used for:

[0182] Based on the topological location, determine the edge of the unavailable AI accelerator 204 in the accelerator cluster;

[0183] Based on the edge of the unavailable AI accelerator 204 in the accelerator cluster, the available AI accelerator 204 in the accelerator cluster is divided into multiple sub-regions, and the topology of each sub-region in the multiple sub-regions is a continuous topology.

[0184] Based on multiple sub-regions and the number of AI accelerators participating in the aggregated communication, multiple accelerator groups are determined. Each accelerator group in the multiple accelerator groups corresponds to a sub-region, or is obtained by dividing a sub-region, or by merging at least one AI accelerator in an adjacent sub-region. The number of AI accelerators 204 included in the multiple accelerator groups is greater than or equal to the number of AI accelerators 204 participating in the aggregated communication.

[0185] In some possible implementations, host 202 is specifically used for:

[0186] Based on multiple sub-regions, generate multiple initialization groups, each initialization group corresponding to one sub-region;

[0187] Based on the number of AI accelerators participating in the aggregated communication, and combined with a balancing strategy, multiple initialization groups are adjusted, and multiple accelerator groups are determined from the adjusted groups.

[0188] In some possible implementations, host 202 is also used for:

[0189] Obtain the link bandwidth and communication data volume between the 204 available AI accelerators in the accelerator cluster;

[0190] Based on the link bandwidth and communication data volume between the available AI accelerators 204 in the accelerator cluster, a first parallel parameter is determined, which includes the data volume of data allocated to different links for multiple accelerator groups.

[0191] Each of the plurality of accelerator groups is specifically used for:

[0192] The set communication operator is executed based on the first parallel parameter.

[0193] In some possible implementations, host 202 is also used for:

[0194] The time consumption of at least one candidate algorithm is determined based on the number of AI accelerators included in each accelerator group and the amount of communication data.

[0195] Based on the time consumption of at least one candidate algorithm, a second parallel parameter is determined from at least one candidate algorithm, the second parallel parameter including the target number of steps;

[0196] Each accelerator group in the multiple accelerator groups is specifically used for:

[0197] The set communication operator is executed based on the second parallel parameter.

[0198] In some possible implementations, host 202 is also used for:

[0199] Obtain the initial topology of the accelerator cluster;

[0200] When the topology of the available AI accelerators 204 in the accelerator cluster remains unchanged relative to the topology of the available AI accelerators 204 in the initial topology, the topology of the available AI accelerators in the accelerator cluster is determined to be a continuous topology.

[0201] This application also provides a computing device cluster 20. The hardware architecture of the computing device cluster 20 is as follows: Figure 2 As shown, the computing device cluster 20 includes a host 202 and multiple AI accelerators 204. In this embodiment, the grouping of the multiple AI accelerators 204 can be implemented by the AI ​​accelerators themselves. A detailed explanation follows.

[0202] Multiple AI accelerators 204 are used to receive the topological location of available AI accelerators in the accelerator cluster, and to obtain the number of AI accelerators participating in the aggregate communication.

[0203] At least one of the multiple AI accelerators 204 is also used to determine multiple accelerator groups based on topological location and the number of AI accelerators participating in the aggregated communication.

[0204] Each of the multiple accelerator groups is used to execute the set communication operator to obtain the first execution result;

[0205] Multiple accelerator groups are also used to perform inter-group communication based on the first execution result to obtain a second execution result.

[0206] It should be noted that host 202 or AI accelerator 204 can execute cluster communication methods by deploying software. For example, a communication engine can be deployed on the AI ​​accelerator 204 side. The AI ​​accelerator 204 can use the communication engine to perform accelerator grouping, intra-group communication, and inter-group communication, thereby achieving cluster communication in discontinuous topologies. In some examples, the communication engine may not perform accelerator grouping; instead, host 202 can run the corresponding software to implement accelerator grouping, while AI accelerator 204 performs intra-group and inter-group communication.

[0207] Software, exemplified by a communication engine, can be an application running on a computer device. These applications can also be virtualized and provided to users as virtualization services. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, or container services. VM services utilize virtualization technology to create a pool of virtual machine (VM) resources across multiple physical hosts, providing VMs to users on demand. BMS services create a pool of BMS resources across multiple physical hosts, providing BMS services to users on demand. Container services create a pool of container resources across multiple physical hosts, providing containers to users on demand. A VM is a simulated virtual computer, or logically a computer. A BMS is a scalable, high-performance computing service with computing performance indistinguishable from traditional physical machines, featuring secure physical isolation. A container is a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service, and container service mentioned above are merely specific examples. In practical applications, virtualization services can also include other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0208] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a cluster of computing devices to perform a collective communication method.

[0209] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the aforementioned collective communication method.

[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An aggregation communication method characterized by, Applied to a computing device cluster comprising a host and an accelerator cluster, the accelerator cluster comprising a plurality of artificial intelligence (AI) accelerators, the method comprises: The host obtains a topology position of AI accelerators available in the accelerator cluster, and a number of AI accelerators participating in collective communication; When a maximum continuous topology in the topology of the accelerator cluster comprises a number of AI accelerators less than the number of AI accelerators participating in collective communication, the host determines a plurality of accelerator groups according to the topology position and the number of AI accelerators participating in collective communication, each accelerator group in the plurality of accelerator groups having a continuous topology; Each accelerator group in the plurality of accelerator groups executes a collective communication operator to obtain a first execution result; The plurality of accelerator groups performs inter-group communication according to the first execution result to obtain a second execution result.

2. The method of claim 1, wherein, Each accelerator group in the plurality of accelerator groups has at least one AI accelerator interconnected with other accelerator groups; The plurality of accelerator groups performs inter-group communication according to the first execution result to obtain a second execution result, comprising: Each accelerator group in the plurality of accelerator groups has at least one AI accelerator interconnected with other accelerator groups, which performs collective communication according to the first execution result to obtain a second execution result, and then performs intra-group collective communication to synchronize the second execution result within the accelerator group.

3. The method according to claim 1 or 2, characterized in that, The host determines a plurality of accelerator groups according to the topology position and the number of AI accelerators participating in collective communication, comprising: The host determines an edge of AI accelerators unavailable in the accelerator cluster according to the topology position; The host divides the AI accelerators available in the accelerator cluster into a plurality of sub-regions according to the edge of AI accelerators unavailable in the accelerator cluster, each sub-region in the plurality of sub-regions having a continuous topology; The host determines a plurality of accelerator groups according to the plurality of sub-regions and the number of AI accelerators participating in collective communication, each accelerator group in the plurality of accelerator groups corresponding to one sub-region, or being obtained by dividing one sub-region, or being obtained by combining at least one AI accelerator in an adjacent sub-region in one sub-region, the plurality of accelerator groups comprising a number of AI accelerators greater than or equal to the number of AI accelerators participating in collective communication.

4. The method of claim 3, wherein, The host determines a plurality of accelerator groups according to the number of AI accelerators included in each sub-region in the plurality of sub-regions and the number of AI accelerators participating in collective communication, comprising: The host generates a plurality of initialization groups according to the plurality of sub-regions, each initialization group corresponding to one sub-region; The host adjusts the plurality of initialization groups according to the number of AI accelerators participating in collective communication in combination with a balancing strategy, and determines a plurality of accelerator groups from the adjusted groups.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: The host obtains a link bandwidth and a communication data volume between AI accelerators available in the accelerator cluster; The host determines a first parallel parameter according to a link bandwidth between AI accelerators available in the accelerator cluster and the communication data volume, the first parallel parameter including a data volume of data allocated to different links of the plurality of accelerator groups; Each of the plurality of accelerator groups executes a collective communication operator, including: Each of the plurality of accelerator groups executes a collective communication operator according to the first parallel parameter.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The host determines a time consumption of at least one candidate algorithm according to a number of AI accelerators included in each of the plurality of accelerator groups and the communication data volume; The host determines a second parallel parameter from the at least one candidate algorithm according to the time consumption of the at least one candidate algorithm, the second parallel parameter including a target step number; Each of the plurality of accelerator groups executes a collective communication operator, including: Each of the plurality of accelerator groups executes a collective communication operator according to the second parallel parameter.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: The host obtains an initial topology of the accelerator cluster; When a topology of the available AI accelerators in the accelerator cluster remains unchanged relative to a topology of the available AI accelerators in the initial topology, the host determines that the topology of the available AI accelerators in the accelerator cluster is a continuous topology.

8. An aggregation communication method characterized by comprising: Applied to a computing device cluster including a host and an accelerator cluster, the accelerator cluster including a plurality of artificial intelligence (AI) accelerators, the method includes: The plurality of AI accelerators receives a topology position of AI accelerators available in the accelerator cluster and obtains a number of AI accelerators participating in collective communication; At least one of the plurality of AI accelerators determines a plurality of accelerator groups according to the topology position and the number of AI accelerators participating in collective communication; Each of the plurality of accelerator groups executes a collective communication operator to obtain a first execution result; The plurality of accelerator groups performs inter-group communication according to the first execution result to obtain a second execution result.

9. A cluster of computing devices, characterized in that, The computing device cluster includes a host and an accelerator cluster, the accelerator cluster including a plurality of artificial intelligence (AI) accelerators; The host is configured to obtain a topology position of AI accelerators available in the accelerator cluster and obtain a number of AI accelerators participating in collective communication; when a number of AI accelerators included in a maximum continuous topology in a topology of the accelerator cluster is less than the number of AI accelerators participating in collective communication, determine a plurality of accelerator groups according to the topology position and the number of AI accelerators participating in collective communication, each of the plurality of accelerator groups having a continuous topology; Each of the plurality of accelerator groups is configured to execute a collective communication operator to obtain a first execution result; The plurality of accelerator groups is configured to perform inter-group communication according to the first execution result to obtain a second execution result.

10. The computing device cluster of claim 9, wherein, At least one AI accelerator in each of the plurality of accelerator groups is interconnected with other accelerator groups; The AI accelerator in each of the accelerator groups is specifically configured to: According to the first execution result, perform collective communication to obtain a second execution result, and then perform intra-group collective communication to synchronize the second execution result within the accelerator group.

11. The cluster of computing devices of claim 9 or 10, wherein, The host is specifically configured to: According to the topology position, determine the edge of the AI accelerator that is unavailable in the accelerator cluster; According to the edge of the AI accelerator that is unavailable in the accelerator cluster, divide the AI accelerator available in the accelerator cluster into a plurality of sub-regions, and the topology of each of the plurality of sub-regions is continuous topology; According to the plurality of sub-regions and the number of AI accelerators participating in collective communication, determine a plurality of accelerator groups, each of the plurality of accelerator groups corresponds to a sub-region, or is obtained by dividing a sub-region, or is obtained by combining at least one AI accelerator in a neighboring sub-region of a sub-region, and the number of AI accelerators included in the plurality of accelerator groups is greater than or equal to the number of AI accelerators participating in collective communication.

12. The computing device cluster of claim 11, wherein, The host is specifically configured to: According to the plurality of sub-regions, generate a plurality of initialization groups, each of which corresponds to a sub-region; According to the number of AI accelerators participating in collective communication, adjust the plurality of initialization groups in combination with a balancing strategy, and determine a plurality of accelerator groups from the adjusted groups.

13. The cluster of computing devices of any of claims 9 to 12, wherein, The host is further configured to: Obtain the link bandwidth and communication data volume between the AI accelerators available in the accelerator cluster; According to the link bandwidth and the communication data volume between the AI accelerators available in the accelerator cluster, determine a first parallel parameter, the first parallel parameter including the data volume of data allocated to different links of the plurality of accelerator groups; Each of the plurality of accelerator groups is specifically configured to: According to the first parallel parameter, execute a collective communication operator.

14. The cluster of computing devices of any of claims 9 to 12, wherein, The host is further configured to: According to the number of AI accelerators included in each of the plurality of accelerator groups and the communication data volume, determine the time consumption of at least one candidate algorithm; According to the time consumption of the at least one candidate algorithm, determine a second parallel parameter from the at least one candidate algorithm, the second parallel parameter including a target step number; Each of the plurality of accelerator groups is specifically configured to: According to the second parallel parameter, execute a collective communication operator.

15. The cluster of computing devices of any of claims 9 to 14, wherein, The host is further configured to: Obtain an initial topology of the accelerator cluster; When the topology of the AI accelerators available in the accelerator cluster remains unchanged relative to the topology of the AI accelerators available in the initial topology, determine that the topology of the AI accelerators available in the accelerator cluster is continuous topology.

16. A cluster of computing devices, characterized in that, The computing device cluster includes a host and an accelerator cluster, and the accelerator cluster includes a plurality of artificial intelligence (AI) accelerators; The plurality of AI accelerators are configured to receive a topology position of an AI accelerator available in the accelerator cluster, and obtain a number of AI accelerators participating in collective communication. The at least one AI accelerator of the plurality of AI accelerators is further configured to determine a plurality of accelerator groups according to the topology position and a number of AI accelerators participating in the collective communication; Each of the plurality of accelerator groups is configured to execute a collective communication operator to obtain a first execution result. The plurality of accelerator groups is further configured to perform inter-group communication according to the first execution result to obtain a second execution result.

17. A computer-readable storage medium, characterized in that, The computer readable instructions are configured to implement the collective communication method of any one of claims 1-8.

18. A computer program product, characterised in that, The computer readable instructions are configured to implement the collective communication method of any one of claims 1-8.

Images