Simulation method and related equipment

Abstract

The invention provides a simulation method and related equipment. The simulation method and the related equipment are used for improving the accuracy of performance evaluation of an AI task. The method is applied to a simulation system, the simulation system is used for simulating performance of an AI task when the AI task runs on a cluster, a plurality of computing devices included in the cluster are connected through at least one switch, and communication among a plurality of sub-tasks included in the AI task comprises a plurality of communication steps. Input information is received, the input information comprises parameters of an AI model, a parallel strategy of the AI model, a rank table, resource topology and a set communication algorithm used by the AI model, and the rank table comprises a mapping relation between identifiers of a plurality of sub-tasks and address information of a plurality of computing devices; the resource topology indicates a connection relationship between the plurality of computing devices and the at least one switch, and between the at least one switch and each other. A first duration indicating an execution time of each communication step is calculated according to the input information. And outputting the total communication duration of the AI task, wherein the total communication duration is the sum of the first durations of the communication steps.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI), and in particular to the design of a simulation method and related equipment. Background Technology

[0002] With the development of artificial intelligence technology, the scale of AI models is gradually increasing. The number of parameters in AI models, as well as the number of devices required for inference or training, are also gradually increasing. Training or inference on large-scale clusters typically employs a distributed parallel approach. However, the actual performance of an AI model cannot be determined until it is actually running on the cluster.

[0003] In the relevant technical solutions, the parameters of the AI ​​model and the hardware parameters required to run the AI ​​model are input into the simulation device. The simulation device evaluates the running performance of the AI ​​model on the cluster by simulating the training or inference process of the AI ​​model.

[0004] However, in this technical solution, the evaluation of the AI ​​model's performance on the cluster is a rather general result, reflecting the overall performance of the AI ​​model on the cluster, and the accuracy of the evaluation is not high. Summary of the Invention

[0005] This application provides a simulation method and related equipment for simulating the process of an AI task running on a cluster using a simulation system. This includes simulating multiple communication steps between multiple subtasks within the AI ​​task. By calculating the first duration of each communication step, the total communication duration of the AI ​​task is obtained. This achieves accurate prediction of communication duration, improving the accuracy of performance evaluation for the AI ​​task.

[0006] Firstly, this application provides a simulation method executed by a simulation system used to simulate the performance of an AI task running on a cluster. The AI ​​task includes training or inference of an AI model, and comprises multiple subtasks. Communication between the subtasks includes multiple communication steps. The cluster includes multiple computing devices connected via at least one switch.

[0007] The simulation system receives input information and performs simulations based on it. The input information includes the AI ​​model's parameters, parallel strategy, rank table, resource topology, and the collective communication algorithm used by the AI ​​model. The rank table maps the identifiers of multiple subtasks to the address information of multiple computing devices. The resource topology indicates the connections between multiple computing devices and at least one switch, and between at least one switch and each other. The collective communication algorithm defines how many communication steps are included in each subtask, and which communication pairs are included in each step; in other words, it defines how multiple subtasks complete multiple communication steps. Based on the AI ​​model's parameters, parallel strategy, rank table, resource topology, and collective communication algorithm, the simulation system calculates the first duration of each communication step. The first duration is the execution time of the corresponding communication step; that is, the first duration is the actual execution time of each communication step during the simulation of the AI ​​task running on the cluster. The simulation system outputs the total communication duration of the AI ​​task, which is the sum of the first durations of each communication step.

[0008] This application uses a simulation system to simulate the process of an AI task running on a cluster, including simulating multiple communication steps between various subtasks within the AI ​​task. By calculating the first duration of each communication step, the total communication duration of the AI ​​task is obtained. This enables accurate prediction of communication duration and improves the accuracy of performance evaluation for AI tasks.

[0009] In some optional implementations of the first aspect, the simulation system calculates the first duration of each communication step in multiple communication steps, specifically including: decomposing the AI ​​task into multiple subtasks based on the AI ​​model's parameters and parallel strategy. It can be understood that the AI ​​model's parameters indicate the amount of data the AI ​​task needs to process, and the AI ​​model's parallel strategy indicates how the amount of data processed by the AI ​​task is divided. Therefore, the aforementioned process can be understood as dividing the amount of data the AI ​​task needs to process into multiple parts based on the AI ​​model's parallel strategy, with each part corresponding to a subtask. Based on the ranktable and resource topology, the communication topology of the multiple subtasks is determined. The communication topology indicates the communication connection relationship between the multiple subtasks when they run on multiple computing devices. Based on the AI ​​model's parameters, parallel strategy, ensemble communication algorithm, and communication topology, the first duration of each communication step is calculated.

[0010] In this application, the AI ​​task is broken down into multiple subtasks, and then the communication topology of the subtasks is constructed to determine the communication connections between them. Then, the first duration of each communication step in the multiple communication steps of the subtasks is calculated. In other words, the simulation system calculates the duration of communication steps at the subtask level, resulting in a finer granularity for detecting the performance of the AI ​​task and further improving the accuracy of performance evaluation.

[0011] In some alternative implementations of the first aspect, the resource topology further indicates the transmission bandwidth and transmission delay between any two connected devices among multiple computing devices and at least one switch. Correspondingly, the communication topology further indicates the transmission bandwidth and transmission delay between any two connected subtasks among multiple subtasks running on multiple computing devices. The simulation system calculates the first duration of each communication step, specifically by first determining, according to the ensemble communication algorithm, that each communication step includes at least one communication pair, where each communication pair includes a subtask for sending data and a subtask for receiving data. Then, according to the communication topology, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link are determined. Next, according to the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, the total communication volume of at least one communication pair is determined, where the total communication volume of at least one communication pair is the sum of the communication volumes of each communication pair. Finally, based on the total communication volume of at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link, the first duration of each communication step is determined. The first duration can be understood as the duration of the total communication volume of at least one communication pair in a communication step, which is transmitted through the communication link of that communication step.

[0012] In this application, the simulation system can also determine at least one communication pair included in each communication step, and, combining the total communication volume of these at least one communication pair, the transmission link of each communication pair, and the transmission delay, determine the first duration of each communication step. The calculation of the first duration is more precise, further improving the accuracy of performance evaluation for AI tasks.

[0013] In some optional implementations of the first aspect, the simulation system can also determine the transmission time of each communication pair based on the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair. The transmission time of each communication pair here refers to the time for transmitting the communication volume of each communication pair when the link of each communication pair is exclusively used by the communication process of that communication pair; this can be considered the ideal transmission time of each communication pair. A second duration for each communication step is determined from the transmission times of at least one communication pair included in each communication step; the second duration is the transmission time of the communication pair with the longest duration in the corresponding communication step. The average conflict coefficient of each communication step is output; the average conflict coefficient is the quotient of the first duration and the second duration. Therefore, the average conflict coefficient reflects the multiple of the actual time consumption of the communication step compared to the ideal time consumption. The larger the average conflict coefficient, the longer the communication step is considered to be, which can intuitively reflect the performance of the current AI task.

[0014] In this application, the simulation system can also output the average conflict coefficient for each communication step. Since the average conflict coefficient reflects the ratio of the actual time taken for a communication step to the ideal time taken, it can intuitively reflect the performance of the AI ​​task in terms of communication time. The larger the average conflict coefficient, the longer the communication step is considered to take, which can be used as a potential optimization point for the AI ​​task.

[0015] In some alternative implementations of the first aspect, the simulation system may also output the first duration of each communication step.

[0016] In this application, the simulation system, in addition to outputting the total communication duration of the AI ​​task, can also output the first duration of each communication step, enriching the dimensions of performance evaluation for the AI ​​task. Furthermore, based on the first duration of each communication step, adjustments can be made to communication steps that are too long, thereby optimizing the performance of the AI ​​task running on the cluster.

[0017] In some optional implementations of the first aspect, the parallel strategy of the AI ​​model includes multiple communication methods corresponding to multiple communication steps. The simulation system can also statistically analyze the first duration of each communication step from the dimension of communication methods, determine the third duration corresponding to each of the multiple communication methods, and output the third duration corresponding to each communication method. The third duration is the sum of the first durations of communication steps using the same communication method.

[0018] In this application, the simulation system can also output a third duration for each communication method, evaluating the performance of the AI ​​task from more perspectives and enriching the performance evaluation results. Furthermore, the third duration for each communication method can also serve as a reference metric for optimizing the AI ​​task; for example, optimizing time-consuming communication methods can improve the performance of the AI ​​task running on the cluster.

[0019] In some optional implementations of the first aspect, the simulation system can also output the communication link for each communication pair and / or the amount of communication transmitted by each communication pair. This further enriches the dimensions of the simulation system's performance evaluation of AI tasks and can also intuitively reflect the communication process between multiple subtasks in an AI task.

[0020] In some alternative implementations of the first aspect, the communication topology further indicates the set of ports corresponding to the communication links of each communication pair, the port set including ports of the computing devices sending data, ports of the computing devices receiving data, and ports of at least one switch transmitting data. The simulation system determines the communication volume of each port in the port set corresponding to each communication link based on the communication volume of each communication pair. Based on the communication volume of each port in the port set corresponding to each communication link, the system determines the total communication volume of each port of each computing device among the multiple computing devices, and the total communication volume of each port of each switch among the at least one switch, after running the AI ​​task.

[0021] In this application, the simulation system can also determine the communication volume of the ports of each device included in each communication link, thereby determining the total communication volume of the ports of each device after running the AI ​​task, thus reflecting the data transmission situation between multiple subtasks during the operation of the AI ​​task.

[0022] In some optional implementations of the first aspect, the simulation system may also output the total traffic of the first port of the target computing device and / or the total traffic of the second port of the target switch, wherein the target computing device comprises multiple computing devices and the target switch comprises at least one switch. This further enriches the dimensions of the simulation system's performance evaluation of AI tasks and enhances the performance evaluation results.

[0023] Secondly, this application provides a simulation system for simulating the performance of AI tasks running on a cluster. The AI ​​task includes training or inference of an AI model. The cluster includes multiple computing devices connected via at least one switch. The AI ​​task includes multiple subtasks, and communication between the subtasks includes multiple communication steps. The simulation system includes:

[0024] The acquisition unit is used to receive input information, which includes the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the collective communication algorithm used by the AI ​​model. The ranktable includes the mapping relationship between the identifiers of multiple subtasks and the address information of multiple computing devices. The resource topology indicates the connection relationship between multiple computing devices and at least one switch, and between at least one switch and each other.

[0025] The processing unit is used to calculate the first duration of each communication step in multiple communication steps based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology and the collective communication algorithm. The first duration is the execution time of the corresponding communication step.

[0026] The processing unit is also used to output the total communication duration of the AI ​​task, which is the sum of the first duration of each communication step.

[0027] The simulation system is used to implement the method shown in the first aspect above, or any possible implementation of the first aspect, as detailed above, and will not be repeated here.

[0028] Thirdly, this application provides a computing device including a processor and a memory. The processor of the computing device is used to execute instructions stored in the memory, causing the computing device to implement the method shown in the first aspect or any possible implementation of the first aspect.

[0029] Fourthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster implements the method disclosed in the first aspect or any possible implementation of the first aspect.

[0030] Fifthly, this application provides a computer program product containing instructions that, when executed by a cluster of computer devices, cause the cluster of computer devices to implement the method disclosed in the first aspect and any possible implementation of the first aspect.

[0031] In a sixth aspect, this application provides a computer-readable storage medium storing computer program instructions that, when executed by a cluster of computer devices, cause the cluster of computer devices to implement the method disclosed in the first aspect and any possible implementation of the first aspect.

[0032] The beneficial effects shown in any of the second to sixth aspects are similar to those of the first aspect or any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the main architecture of artificial intelligence provided in the embodiments of this application;

[0034] Figure 2 A schematic diagram of a system architecture provided for an embodiment of this application;

[0035] Figure 3 Another system architecture diagram provided for an embodiment of this application;

[0036] Figure 4 Another system architecture diagram provided for an embodiment of this application;

[0037] Figure 5 A flowchart illustrating the simulation method provided in this application embodiment;

[0038] Figure 6 A schematic diagram provided for an embodiment of this application;

[0039] Figure 7 Another schematic flowchart of the simulation method provided in the embodiments of this application;

[0040] Figure 8 A schematic diagram of the communication topology provided in the embodiments of this application;

[0041] Figure 9 A schematic diagram of the simulation output provided in the embodiments of this application;

[0042] Figure 10 Another schematic diagram of the communication topology provided in the embodiments of this application;

[0043] Figure 11 Another schematic flowchart of the simulation method provided in the embodiments of this application;

[0044] Figure 12 Another schematic flowchart of the simulation method provided in the embodiments of this application;

[0045] Figure 13 Another schematic diagram illustrating the output results of the simulation provided in the embodiments of this application;

[0046] Figure 14 A schematic diagram of the simulation system provided in the embodiments of this application;

[0047] Figure 15 Another schematic diagram of the simulation system provided in the embodiments of this application;

[0048] Figure 16 A schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0049] Figure 17 A schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0050] Figure 18 This is another schematic diagram of the computing device cluster provided in an embodiment of this application. Detailed Implementation

[0051] This application provides a simulation method and related equipment for simulating the execution of an AI task on a cluster using a simulation system. This includes simulating multiple communication steps between multiple subtasks within the AI ​​task. By calculating the duration of each communication step, the total communication duration of the AI ​​task is obtained. This achieves accurate prediction of communication duration, improving the accuracy of performance evaluation for the AI ​​task.

[0052] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0053] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses. Additionally, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be expressed as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0054] First, the relevant concepts and proper nouns involved in the embodiments of this application will be explained.

[0055] 1) Distributed parallelism:

[0056] Distributed parallelism is a technique that distributes the training or inference process of an AI model across multiple computing devices. It can also be understood as breaking down the training or inference process into multiple sub-processes and distributing these sub-processes across multiple computing devices for execution. More specifically, distributed parallelism includes distributed computing and parallel computing. This technique allows AI models to leverage the computing power of multiple devices to accelerate large-scale data processing and the execution of complex models. Distributed parallelism includes tensor parallelism (TP), pipeline parallelism (PP), and data parallelism (DP), among other parallelism methods.

[0057] 2) Rank table:

[0058] A ranktable is a file used to manage communication between computing nodes participating in the training or inference of an AI model. It records information about all nodes involved in the training or inference process, including each node's logical identifier (rank ID) and address information. Ranktable ensures that all nodes participating in AI model training or inference can communicate and synchronize effectively, thus achieving high-efficiency parallelism.

[0059] 3) Communication domain:

[0060] A communication domain refers to a set of computing devices that can communicate with each other, and a communication domain is responsible for a specific communication task. Multiple computing devices in a communication domain synchronize information such as AI model parameters and gradients through communication and data exchange. During the training or inference process of an AI model, the multiple computing nodes used by the AI ​​model can include multiple communication domains, and a computing device can belong to one or more communication domains.

[0061] 4) Collective communication algorithms:

[0062] Ensemble communication algorithms are communication algorithms used in distributed and parallel computing for data exchange between multiple processes or threads. In these algorithms, a group of processes jointly participate in communication operations to collect, distribute, or aggregate data. In the training or inference of AI models, ensemble communication algorithms define the sub-processes for communication, as well as communication steps and other communication information related to the training or inference process of the AI ​​model.

[0063] Next, please refer to Figure 1 , Figure 1 This is a schematic diagram of the main architecture of an artificial intelligence system provided in an embodiment of this application. This main framework describes the overall workflow of the artificial intelligence system and is applicable to general artificial intelligence domain needs.

[0064] The above-mentioned artificial intelligence framework will be elaborated from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0065] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it could be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom."

[0066] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (provided and processed by technology) to the industrial ecosystem of systems.

[0067] (1) Infrastructure.

[0068] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, including central processing units (CPUs), neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs) and other hardware acceleration chips. The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0069] (2) Data.

[0070] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0071] (3) Data processing.

[0072] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0073] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0074] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0075] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0076] (4) General ability.

[0077] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0078] (5) Smart products and industry applications.

[0079] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They are the encapsulation of overall artificial intelligence solutions, productizing intelligent information decision-making and realizing practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, smart cities, and intelligent terminals.

[0080] The simulation method provided in this application can be applied to cloud computing. The system architecture of cloud computing is described below with reference to the schematic diagram. Please refer to... Figure 2 , Figure 2 This is a schematic diagram of the system architecture provided for an embodiment of this application.

[0081] like Figure 2 As shown, the tenant logs into the cloud platform 203 via client 201 through the internet 202 using the account and password registered on the cloud platform 203. The cloud platform 203 manages the infrastructure, which includes multiple data centers located in different regions, such as... Figure 2 Region 1, as shown, includes Cloud Data Center 1 and Cloud Data Center 2, while Region 2 includes Cloud Data Center 3 and Cloud Data Center 4. Each cloud data center has multiple servers, on which business instances (including at least one of virtual machines, containers, and dedicated hosts) run.

[0082] In this embodiment, a simulation service is deployed in the business instance to simulate the performance of AI tasks running on a cluster. The tenant purchases cloud services through client 201 on cloud platform 203 and sends a request to cloud platform 203, which requests cloud services from cloud platform 203. In this embodiment, the cloud services specifically include simulating the execution process of AI tasks to evaluate their performance on the cluster. The specific implementation process will be described later.

[0083] In this embodiment, the cloud platform manages multiple computing devices and at least one switch. The multiple computing devices are used for the functions of multiple sub-tasks included in the AI ​​task, and the at least one switch is used to connect the multiple computing devices to enable communication between them. The connection relationships between the multiple computing devices and the at least one switch, as well as the connection relationships between the at least one switch and themselves, are various, and will be explained below with reference to the schematic diagram.

[0084] Please see Figure 3 and Figure 4 , Figure 3 and Figure 4 All of these are schematic diagrams of the system architecture provided in the embodiments of this application.

[0085] exist Figure 3 The illustrated embodiment includes two switching layers; this architecture can also be referred to as a "spine-leaf" architecture. These two switching layers constitute the network topology of the data center. For example, Figure 3 Switch 1 can be considered a spine layer switch, while switches 2 and 3 can be considered leaf layer switches.

[0086] like Figure 3 As shown, computing devices 1 through 3 are all connected to switch 2, switch 2 is connected to switch 1, switch 1 is also connected to switch 3, and switch 3 is also connected to computing devices 4 and 5. Through... Figure 3 The connection shown enables communication between different computing devices.

[0087] It should be noted that this application does not restrict the system architecture for running AI tasks. Figure 3 This is merely an illustration of the system architecture. In practical applications, it may include more or fewer switches and computing devices, and the number of switch layers may also be more or fewer; no specific limitations are made here.

[0088] For example, in Figure 4 The embodiment shown is based on a three-layer network architecture, which includes three layers of switches.

[0089] Please refer to the following. Figure 5 , Figure 5A flowchart illustrating the simulation method provided in this application embodiment includes:

[0090] 501. Receive input information, including the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the ensemble communication algorithm used by the AI ​​model.

[0091] The simulation method provided in this application is executed by a simulation system, simulating the performance of an AI task running on a cluster. An AI task refers to either the training task or the inference task of an AI model. Furthermore, an AI task includes multiple sub-tasks, and communication between these sub-tasks involves multiple communication steps, based on which the complete communication process of the AI ​​task is achieved. The cluster running the AI ​​task includes multiple computing devices connected through at least one switch. These computing devices can be processors, servers, virtual machines, containers, etc., and are not specifically limited here.

[0092] The simulation system receives input information, simulates the execution of AI tasks on a cluster, and outputs the performance of the AI ​​tasks running on the cluster. Therefore, the input information describes the AI ​​task and the cluster running the AI ​​task. Specifically, the input information includes: the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the ensemble communication algorithm used by the AI ​​model.

[0093] The parameters of an AI model define the model and represent its complexity; in simpler terms, they define what kind of model the AI ​​is. For example, AI model parameters may include at least one of the following: input layer parameters, hidden layer parameters, neural network layer parameters, loss function parameters, output layer parameters, optimizer parameters, or regularization parameters. The specific parameters included may vary between different AI models, and are not limited here.

[0094] The parallel strategy of an AI model indicates how the AI ​​model is distributed across multiple computing devices in parallel. It can also be understood as the communication method between multiple subtasks. For example, communication methods include tensor parallelism, pipelined parallelism, and data parallelism. An AI model can employ one or more parallel methods; no specific limitation is made here.

[0095] The ranktable includes a mapping between the identifiers of multiple subtasks and the address information of multiple computing devices; it can be understood as a mapping between logical identifiers and physical identifiers. The address information can be an Internet Protocol (IP) address, a Media Access Control (MAC) address, or other information that can represent the address of a computing device; no specific limitations are made here.

[0096] Resource topology indicates the connectivity between multiple computing devices and at least one switch, and between at least one switch and each other. In other words, resource topology defines the network architecture for running AI tasks.

[0097] The ensemble communication algorithm used by AI models defines how many communication steps are between multiple subtasks included in an AI task, and which communication pairs (data sender and data receiver) are included in each communication step. In other words, the ensemble communication algorithm defines how multiple subtasks complete multiple communication steps.

[0098] 502. Based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the collective communication algorithm, calculate the first duration of each communication step in multiple communication steps. The first duration is the execution time of the corresponding communication step.

[0099] The simulation system acquires input information and simulates multiple communication steps between various subtasks in the AI ​​task, thereby obtaining the initial duration of each communication step. This process is explained in detail below:

[0100] The simulation system breaks down the AI ​​task into multiple sub-tasks based on the AI ​​model's parameters and parallelism strategy. The AI ​​model's parameters indicate the amount of data the AI ​​task needs to process, and the AI ​​model's parallelism strategy indicates how this data is divided. Based on this division method, the simulation system divides the amount of data the AI ​​task needs to process into multiple data sets, with each data set corresponding to a sub-task.

[0101] For example, please refer to Figure 6 , Figure 6 This is a schematic diagram provided for an embodiment of this application. It is assumed that the parallel strategies for AI tasks include data parallelism and pipelined parallelism. Figure 6 In the illustrated embodiment, the AI ​​task is represented by a cube. Based on a pipelined approach, the AI ​​task is broken down into subtask 1 and subtask 2. Then, based on a data parallel approach, it is further broken down into subtask 1-1, subtask 1-2, subtask 2-1, and subtask 2-2.

[0102] The simulation system determines the communication topology of multiple subtasks based on the ranktable and resource topology. This communication topology indicates the communication connections between the subtasks when they run on multiple computing devices. Combined with... Figure 7 For a detailed explanation of this process, please refer to [link / reference]. Figure 7 , Figure 7 This is a flowchart illustrating the simulation method provided in the embodiments of this application.

[0103] exist Figure 7In the illustrated embodiment, rankID represents the identifier of the subtask, and the IP address represents the address information of the computing device. In the ranktable, the IP address corresponding to subtask 1 is 29.59.95.178, the IP address corresponding to subtask 2 is 29.59.115.206, and the IP address corresponding to subtask 3 is 29.59.176.221. These are the IP addresses of computing devices A, B, and C, respectively. This means that subtask 1 runs on computing device A, subtask 2 runs on computing device B, and subtask 3 runs on computing device C. Therefore, combined with... Figure 7 The ranktable and resource topology shown also yield the following results. Figure 7 The communication topology shown.

[0104] It should be noted that the ranktable in the embodiments of this application is not limited to the form of a table, but can also be a key-value pair or other representation that can reflect the mapping relationship. No specific limitation is made here.

[0105] After obtaining the communication topology of multiple subtasks, the simulation system can calculate the first duration of each communication step based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ensemble communication algorithm, and the communication topology. This process is explained below.

[0106] In this embodiment, the resource topology further indicates the transmission bandwidth and transmission latency between any two devices connected to a plurality of computing devices and at least one switch. That is, the resource topology indicates the transmission bandwidth and transmission latency between connected computing devices and switches, and the transmission bandwidth and transmission latency between connected switches.

[0107] Accordingly, the communication topology also indicates the transmission bandwidth and transmission latency between any two connected subtasks when multiple subtasks are running on multiple computing devices. Specifically, if two connected subtasks are connected through at least one switch, then the transmission bandwidth between the two connected subtasks includes the transmission bandwidth between the computing device running the two subtasks and the switch, as well as the transmission bandwidth between the at least one switch and itself. The transmission latency between the two connected subtasks includes the transmission latency between the computing device running the two subtasks and the switch, as well as the transmission latency between the at least one switch and itself.

[0108] For example, please refer to Figure 8 , Figure 8 This is a schematic diagram of the communication topology provided in an embodiment of this application.

[0109] like Figure 8As shown, subtasks 1, 2, and 3 run on computing devices A, B, and C, respectively, which are connected via switches 1 and 2. The resource topology defines the transmission bandwidth and latency between computing device A and switch 1, between switch 1 and switch 2, between computing device B and switch 2, and between computing device B and switch 3.

[0110] Accordingly, the transmission bandwidth and latency of subtask 1 and subtask 2 include the transmission bandwidth and latency between computing device A and switch 1, between switch 1 and switch 2, and between computing device B and switch 2. The transmission bandwidth and latency of subtask 1 and subtask 3 include the transmission bandwidth and latency between computing device A and switch 1, between switch 1 and switch 2, and between computing device C and switch 2.

[0111] In summary, the simulation system, based on the ensemble communication algorithm, determines at least one communication pair included in each communication step, where each communication pair includes a subtask of sending data and a subtask of receiving data. Based on the communication topology, it determines the communication link, transmission bandwidth, and transmission delay of each communication link. Based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, it determines the total communication volume of at least one communication pair, which is the sum of the communication volumes of each communication pair. Based on the total communication volume of at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link, it determines the first duration of each communication step.

[0112] The following is based on Figure 8 Taking an example, the aforementioned process will be explained in detail. In Figure 8 In the illustrated embodiment, it is assumed that a communication step includes two communication pairs, namely <subtask 1-subtask 2> and <subtask 1-subtask 3>. Based on the aforementioned pairs... Figure 8 The example illustrates how the simulation system can determine the transmission delay and bandwidth of each of the two communication pairs. Then, based on the parameters and parallel strategy of the AI ​​model, it determines the total communication volume and communication domain for each communication method. Within the same communication domain, different subtasks communicate using the same communication method. Furthermore, subtasks cannot communicate across communication domains. Combining this with the ensemble communication algorithm, the communication volume of each communication pair is determined, i.e., the amount of data transmitted by each pair. Finally, the first duration of each communication step is determined. The first duration is the actual execution time of each communication step simulated by the simulation system, or the time it takes for each communication step to complete. In a scheme where a communication step includes multiple communication pairs, the first duration is the longest transmission time among these multiple communication pairs.

[0113] exist Figure 8 In the illustrated embodiment, the data transmitted in the communication pair <Subtask 1-Subtask 2> is data 1, and the communication link is "Computing Device A-Switch 1-Switch 2-Computing Device B", which includes three sub-links: "Computing Device A-Switch 1", "Switch 1-Switch 2", and "Switch 2-Computing Device B". The communication pair <Subtask 1-Subtask 3> is similar and will not be described again here. Therefore, two sets of data are transmitted on the sub-links "Computing Device A-Switch 1" and "Switch 1-Switch 2". The sum of the data amounts of these two sets of data is the total communication volume of this communication step.

[0114] The transmission time of each communication pair is the longest among the multiple sub-links. The transmission time of each sub-link is based on the communication volume, transmission bandwidth, and transmission delay of each sub-link, and their relationship satisfies: Transmission Time = Communication Volume / Transmission Bandwidth + Transmission Delay.

[0115] For example, the transmission time of the sub-link "Computing Device A - Switch 1" is the quotient of the communication volume of data 1 and data 2 and the transmission bandwidth of the sub-link, plus the transmission delay of the sub-link.

[0116] For example, the transmission time of the sub-link "switch 2-computing device B" is the quotient of the communication volume of data 1 and the transmission bandwidth of the sub-link, plus the transmission delay of the sub-link.

[0117] The simulation system can then calculate the transmission time for each communication pair. It then selects the longest transmission time as the first duration for the communication step involving these multiple communication pairs.

[0118] As described above, in this embodiment, the AI ​​task is broken down into multiple sub-tasks, and then the communication topology of the sub-tasks is constructed to determine the communication connection relationships between the multiple sub-tasks. Then, the first duration of each communication step in the multiple communication steps of the multiple sub-tasks is calculated. That is, the simulation system calculates the duration of the communication steps at the sub-task granularity, providing a finer granularity for detecting the performance of the AI ​​task and further improving the accuracy of the performance evaluation of the AI ​​task. In addition, the simulation system can also determine at least one communication pair included in each communication step, and combine the total communication volume of these at least one communication pair, the transmission link of each communication pair, and the transmission delay to determine the first duration of each communication step. The calculation of the first duration is more refined, further improving the accuracy of the performance evaluation of the AI ​​task.

[0119] 503. Output the total communication duration of the AI ​​task. The total communication duration is the sum of the first duration of each communication step.

[0120] After obtaining the first duration of each communication step, the simulation system sums the first durations of each communication step to obtain the total communication duration of the AI ​​task and outputs the total communication duration.

[0121] This application uses a simulation system to simulate the process of an AI task running on a cluster, including simulating multiple communication steps between various subtasks within the AI ​​task. By calculating the first duration of each communication step, the total communication duration of the AI ​​task is obtained. This enables accurate prediction of communication duration and improves the accuracy of performance evaluation for AI tasks.

[0122] In this embodiment, the simulation system can also evaluate and output other performance metrics of AI tasks running on a cluster. Possible implementation methods are described below.

[0123] In some optional implementations, the simulation system can output the average conflict coefficient for each communication step. The average conflict coefficient is the quotient of the actual execution time and the ideal execution time for each step, thus intuitively reflecting the performance of the AI ​​task in terms of communication time. The process by which the simulation system determines and outputs the average conflict coefficient for each communication step is explained below.

[0124] In summary, the simulation system determines the transmission time of each communication pair based on its traffic volume, communication link, transmission bandwidth, and transmission delay. It then determines a second duration for each communication step, which is the transmission time of the longest communication pair within that step. This second duration can also be referred to as the collision-free duration of each step. Finally, it outputs the average collision coefficient for each communication step, which is the quotient of the first and second durations.

[0125] The transmission time of each communication pair is the ideal transmission time of each communication pair, which can be understood as the time during which each communication pair exclusively enjoys its communication link to transmit data.

[0126] For example, in Figure 8In the illustrated embodiment, the transmission time of the communication pair <Subtask 1-Subtask 2> is the time that the communication link "Computing Device A-Switch 1-Switch 2-Computing Device B" is fully used for transmitting data 1. Specifically, it is the longest transmission time among the sub-links "Computing Device A-Switch 1", "Switch 1-Switch 2", and "Switch 2-Computing Device B". The transmission time of data 1 for each sub-link is the quotient of the data volume of data 1 and the transmission bandwidth of the sub-link, plus the transmission delay of the sub-link. Based on a similar method, the simulation system also calculates the transmission time of the communication pair <Subtask 1-Subtask 3>. Then, from the transmission times of these two communication pairs, the longest time is selected as the second duration. Finally, based on the first and second durations, the average collision coefficient is obtained.

[0127] Optionally, the average conflict coefficient can satisfy the following formula:

[0128]

[0129] Among them, data step For the amount of communication transmitted in a single communication pair during a single communication step, This represents the maximum number of communication pairs in a single communication link within a communication step.

[0130] In this embodiment, the simulation system can also output the average conflict coefficient for each communication step. Since the average conflict coefficient reflects the ratio of the actual time taken for a communication step to the ideal time taken, it can intuitively reflect the performance of the AI ​​task in terms of communication time. The larger the average conflict coefficient, the longer the communication step is considered to take, which can be used as a potential optimization point for the AI ​​task.

[0131] In some alternative implementations, the simulation system may also output the first duration of each communication step.

[0132] In this embodiment, the simulation system outputs not only the total communication duration of the AI ​​task but also the first duration of each communication step, enriching the dimensions for evaluating the performance of the AI ​​task. Furthermore, based on the first duration of each communication step, excessively long communication steps can be adjusted, thereby optimizing the performance of the AI ​​task running on the cluster.

[0133] In the foregoing embodiments, the durations described are calculated from the perspective of communication steps or AI tasks. In practical applications, the simulation system can also determine and output other durations.

[0134] In some optional implementations, the parallel strategy of the AI ​​model includes multiple communication methods corresponding to multiple communication steps. The simulation system can also determine and output a third duration for each of the multiple communication methods based on a first duration for each communication step, where the third duration is the sum of the first durations of communication steps using the same communication method.

[0135] Specifically, a communication step includes at least one communication pair, and the communication method of the communication step is the communication method between the at least one communication pair (i.e., the subtask of sending data and the subtask of receiving data) included in the communication step. Therefore, one communication step can correspond to one or more communication methods.

[0136] In layman's terms, the communication method between different subtasks is the parallel processing method used to break down these different subtasks. For example, Figure 6 In the illustrated embodiment, the communication methods between subtask 1-1 and subtask 1-2, and between subtask 2-1 and subtask 2-2, are all data parallel. The communication methods between subtask 1-1 and subtask 2-1, between subtask 1-1 and subtask 2-2, between subtask 1-2 and subtask 2-1, and between subtask 1-2 and subtask 2-2 are all pipeline parallel.

[0137] For example, suppose Figure 8 In the illustrated embodiment, the communication steps include communication pairs <subtask 1-subtask 2> and <subtask 1-subtask 3> using the same communication method, which is pipelined parallelism. Therefore, the first duration of this communication step is the duration of the pipelined parallel communication method.

[0138] For example, suppose Figure 8 In the illustrated embodiment, the communication steps include communication steps for the communication pairs <subtask 1-subtask 2> and <subtask 1-subtask 3> using different communication methods: pipelined parallelism and tensor parallelism, respectively. Therefore, the first duration of this communication step is both the duration of the pipelined parallelism communication method and the duration of the tensor parallelism communication method.

[0139] In this embodiment, the simulation system can also output a third duration corresponding to each communication method, evaluating the performance of the AI ​​task from more perspectives and enriching the performance evaluation results. Furthermore, the third duration corresponding to each communication method can also serve as a reference indicator for optimizing the AI ​​task; for example, optimizing time-consuming communication methods can improve the performance of the AI ​​task running on the cluster.

[0140] In some alternative implementations, the simulation system may also output the communication link for each communication pair and / or the amount of communication transmitted for each communication pair.

[0141] For example, such as Figure 9 As shown, the simulation system can output the communication link and / or the amount of communication transmitted by each communication pair in the form of code. Optionally, the simulation system can also output the aforementioned information in other forms, such as images, tables, etc., which are not limited here.

[0142] This further enriches the dimensions of the simulation system for evaluating the performance of AI tasks, and can also intuitively reflect the communication process between multiple subtasks in an AI task.

[0143] In some optional implementations, the communication topology further indicates the set of ports corresponding to the communication links of each communication pair. The port set includes ports of the computing device sending data, ports of the computing device receiving data, and ports of at least one switch transmitting data. The computing device sending data is the computing device that runs the subtask of sending data. The computing device receiving data is the computing device that runs the subtask of receiving data.

[0144] For example, in Figure 8 In the illustrated embodiment, computing device A is a computing device that sends data, computing device B is a computing device that receives data 1, and computing device C is a computing device that receives data 2.

[0145] The simulation system determines the communication volume of each port in the port set corresponding to each communication link based on the communication volume of each communication pair. Then, based on the communication volume of each port in the port set corresponding to each communication link, it determines the total communication volume of each port of each computing device in multiple computing devices and the total communication volume of each port of each switch in at least one switch after running the AI ​​task. The process is further illustrated below with reference to the diagram.

[0146] Please see Figure 10 , Figure 10 This is a schematic diagram of the communication topology provided in an embodiment of this application.

[0147] exist Figure 10 In the above embodiment, it is assumed that the communication volume of data 1 is 1GB and the communication volume of data 2 is 2GB. In one communication step, if computing device A sends data 1 through port 1 and data 2 through port 2, the switch receives data 1 and data 2 through port 3, sends data 1 to computing device B through port 4, and sends data 2 to computing device C through port 5. Computing device B receives data 1 through port 6, and computing device C receives data 2 through port 7.

[0148] Therefore, the communication volume of port 1 is 1GB, the communication volume of port 2 is 2GB, the communication volume of port 3 is 3GB, the communication volume of port 4 is 1GB, the communication volume of port 5 is 2GB, the communication volume of port 6 is 1GB, and the communication volume of port 7 is 2GB.

[0149] The simulation system sums up the communication volume of the same port of the same device in each communication step to obtain the total communication volume of that port after the AI ​​task is completed.

[0150] In this application, the simulation system can also determine the communication volume of the ports of each device included in each communication link, thereby determining the total communication volume of the ports of each device after running the AI ​​task, thus reflecting the data transmission situation between multiple subtasks during the operation of the AI ​​task.

[0151] In some alternative implementations, the simulation system can also output the communication traffic of any one or more ports of each device in each communication step.

[0152] In some optional implementations, the simulation system can also output the total traffic of the first port of the target computing device and / or the total traffic of the second port of the target switch. This further enriches the dimensions of the simulation system's performance evaluation of AI tasks and enhances the performance evaluation results.

[0153] The target computing device is comprised of multiple computing devices. The first port is some or all of the ports of the target computing device. In other words, the simulation system can output the total communication volume of any one or more ports on the target computing device after the AI ​​task is completed.

[0154] The target switch is contained in at least one switch, and the second port is some or all of the ports of the target switch. In other words, the simulation system can output the total traffic of any one or more ports on the target switch after the AI ​​task is completed.

[0155] In some optional implementations, the simulation system can also sum the communication traffic of each port of each device in each communication step to obtain the total communication traffic of each device in each communication step, and output the total communication traffic of one or more devices in each communication step.

[0156] In some optional implementations, the simulation system can also sum the total communication traffic of the same devices in each communication step to obtain the total communication traffic of each device after the AI ​​task is completed. It can then output the total communication traffic of one or more devices after the AI ​​task is completed.

[0157] Please refer to the following. Figure 11 , Figure 11 This is a flowchart illustrating the simulation method provided in the embodiments of this application.

[0158] like Figure 11 As shown, the simulation system acquires input information, including AI model parameters, parallel strategy, resource topology, ranktable, and ensemble communication algorithm. Based on the AI ​​model parameters and parallel strategy, the simulation system calculates the communication volume for different parallel methods. It then calculates the communication domains for different parallel methods based on the AI ​​model's parallel strategy. Using the ensemble communication algorithm, it determines the communication pairs included in each communication step. Based on the ranktable and resource topology, it constructs a communication topology for multiple communication pairs, i.e., constructs the communication topology for multiple subtasks. According to the communication pairs and topology of each communication topology, it adds communication volume to the communication links corresponding to each communication step. The simulation system can also calculate the first and second durations of each communication step. The simulation system processes the input information in multiple ways to obtain output results. The output results reflect the performance of the AI ​​task in terms of duration, communication volume, and average conflict coefficient.

[0159] The duration includes the first duration of each communication step, the total communication duration of the AI ​​task, the third duration corresponding to each communication method, and other durations described in the preceding embodiments. The communication volume includes the communication volume of communication steps, port communication volume, device communication volume, and other communication volumes described in the preceding embodiments. The average collision coefficient includes the average collision coefficient of each communication step.

[0160] Furthermore, the simulation method provided in this application embodiment follows the following process: Figure 12 As shown.

[0161] In this embodiment of the application, the simulation system output results have various forms. For example, the output results can be shown in Table 1 below:

[0162] Table 1

[0163]

[0164] As shown in the first row of Table 1, the communication pair in the first communication step of the AI ​​task uses the DP communication method, passing through a switch with IP address 26.71.102.210. Port 16 of this switch is the data receiving port, and the communication volume through this port is 1.1GB. The average collision coefficient of the first communication step is 2, and the transmission time is 0.051 seconds. The transmission time here refers to the first duration of the communication step in the aforementioned embodiment.

[0165] For example, the output results can also be shown in Table 2 below:

[0166] Table 2

[0167] switch port number direction Single iteration communication volume 26.71.102.210 16 In 1.1GB 26.71.102.233 8 out 0.75GB 26.71.103.45 26 0.75GB …

[0168] The output results shown in Table 2 include switch-related information for a single iteration during the AI ​​model training process. As shown in the first row of Table 2, the switch with IP address 26.71.102.210 received 1.1GB of data on port 16 in a single iteration.

[0169] For example, the output results can also be shown in Table 3 below:

[0170] Table 3

[0171] Communication methods Communication volume per iteration Estimated time for a single iteration (s) TP 336GB 1.53 DP 13.08GB 0.19 PP 1.5GB 0.11

[0172] The output results shown in Table 2 include relevant information for different communication methods in a single iteration during the training process of the AI ​​model. As shown in the first row of Table 2, the TP communication method processed 336GB of data in a single iteration, with an estimated processing time of 1.53 seconds.

[0173] For example, the simulation output can also be as follows: Figure 13 The flowchart shown is in the form of a diagram. Please refer to [link / reference]. Figure 13 , Figure 13 This is a schematic diagram of the output results provided in the embodiments of this application.

[0174] exist Figure 13 The illustrated embodiment uses a "spine-leaf" switch architecture as an example. Spine5, Spine6, and Spine12 represent the switches in the Spine layer. The communication topology also includes switches with IP addresses 26.71.102.224 and 26.71.102.227, which, as leaf layer switches, can also be called Tor switches. The bottom-most rectangle represents the computing devices running subtasks.

[0175] The simulation system simulates the performance of AI tasks running on a cluster, and can calculate the actual amount of communication transmitted in each sub-link, which can be compared with the theoretical amount of communication.

[0176] It should also be noted that the embodiments of this application do not limit the number of AI tasks simulated by the simulation system; there can be one or more, and no specific limitation is made here.

[0177] Next, the relevant devices provided in the embodiments of this application will be described.

[0178] Please see Figure 14 , Figure 14 This is a schematic diagram of the simulation system provided in an embodiment of this application. Figure 14As shown, the simulation system 1400 includes a simulation module 1401, a calculation module 1402, and a visualization module 1403.

[0179] Simulation module 1401 is used to simulate the process of AI tasks running on a cluster based on input information.

[0180] The computing module 1402 is used to calculate the performance of the simulated AI task running on the cluster, including the duration, communication volume, and average collision coefficient described in the aforementioned embodiments. These performance metrics can be used as the output results of the simulation.

[0181] The visualization module 1403 is used to display the simulation output results in a visual manner.

[0182] The simulation system 1400 is used to implement the simulation method provided in the embodiments of this application, as detailed above, and will not be repeated here.

[0183] Please see Figure 15 , Figure 15 This is a schematic diagram of the simulation system provided in an embodiment of this application. In this embodiment, the simulation system 1500 is used to simulate the performance of AI tasks running on a cluster. The AI ​​task includes training or inference of an AI model. The cluster includes multiple computing devices connected through at least one switch. The AI ​​task includes multiple subtasks, and communication between the subtasks includes multiple communication steps.

[0184] like Figure 15 As shown, the simulation system 1500 includes an acquisition unit 1501 and a processing unit 1502.

[0185] In some optional implementations, the acquisition unit 1501 is used to receive input information, including the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the collective communication algorithm used by the AI ​​model. The ranktable includes the mapping relationship between the identifiers of multiple subtasks and the address information of multiple computing devices. The resource topology indicates the connection relationship between multiple computing devices and at least one switch, and between at least one switch and each other.

[0186] Processing unit 1502 is used to calculate the first duration of each communication step in multiple communication steps based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the ensemble communication algorithm. The first duration is the execution time of the corresponding communication step. It outputs the total communication duration of the AI ​​task, which is the sum of the first durations of each communication step.

[0187] In some optional implementations, processing unit 1502 is specifically configured to: decompose the AI ​​task into multiple subtasks based on the parameters and parallel strategy of the AI ​​model; determine the communication topology of the multiple subtasks based on the ranktable and resource topology, wherein the communication topology indicates the communication connection relationship between the multiple subtasks when they run on multiple computing devices; and calculate the first duration of each communication step based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the aggregated communication algorithm, and the communication topology.

[0188] In some alternative implementations, the resource topology also indicates the transmission bandwidth and latency between any two connected devices among multiple computing devices and at least one switch. The communication topology also indicates the transmission bandwidth and latency between any two linked subtasks among multiple subtasks running on multiple computing devices.

[0189] Processing unit 1502 is specifically configured to: determine, according to the ensemble communication algorithm, at least one communication pair included in each communication step, wherein each communication pair includes a subtask of sending data and a subtask of receiving data; determine, according to the communication topology, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link; determine, according to the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, the total communication volume of at least one communication pair, wherein the total communication volume of at least one communication pair is the sum of the communication volumes of each communication pair; and determine the first duration of each communication step based on the total communication volume of at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link.

[0190] In some optional implementations, the processing unit 1502 is further configured to determine the transmission time of each communication pair based on the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair. A second duration for each communication step is determined, where the second duration is the transmission time of the communication pair with the longest duration in the corresponding communication step. The average collision coefficient for each communication step is output, where the average collision coefficient is the quotient of the first duration and the second duration.

[0191] In some alternative implementations, the processing unit 1502 is also configured to output a first duration for each communication step.

[0192] In some alternative implementations, the parallel strategy of the AI ​​model includes multiple communication methods corresponding to multiple communication steps.

[0193] The processing unit 1502 is further configured to: determine a third duration corresponding to each of the multiple communication methods based on a first duration of each communication step, wherein the third duration is the sum of the first durations of the communication steps using the same communication method; and output the third duration corresponding to each communication method.

[0194] In some alternative implementations, the processing unit 1502 is also configured to output the communication link of each communication pair and / or the amount of communication transmitted by each communication pair.

[0195] In some alternative implementations, the communication topology also indicates the set of ports corresponding to the communication links of each communication pair, the set of ports including the ports of the computing device that sends data, the ports of the computing device that receives data, and the ports of at least one switch that transmits data.

[0196] The processing unit 1502 is further configured to: determine the communication volume of each port in the port set corresponding to each communication link based on the communication volume of each communication pair; and determine the total communication volume of each port of each computing device in the multiple computing devices and the total communication volume of each port of each switch in at least one switch after running the AI ​​task, based on the communication volume of each port in the port set corresponding to each communication link.

[0197] In some alternative implementations, the processing unit 1502 is further configured to: output the total traffic of a first port of a target computing device, and / or the total traffic of a second port of a target switch, wherein the target computing device comprises a plurality of computing devices and the target switch comprises at least one switch.

[0198] Both the acquisition unit 1501 and the processing unit 1502 can be implemented in software or in hardware. For example, the implementation of the processing unit 1502 will be described below. Similarly, the implementation of the acquisition unit 1501 can be referenced to the implementation of the processing unit 1502.

[0199] As an example of a software functional unit, processing unit 1502 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device, processor, server), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, processing unit 1502 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0200] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0201] As an example of a hardware functional unit, the processing unit 1502 may include at least one computing device, such as a server. Alternatively, the processing unit 1502 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0202] The processing unit 1502 includes multiple computing devices that can be distributed in the same region or in different regions. Similarly, the processing unit 1502 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the processing unit 1502 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0203] It should be noted that the simulation system 1500 achieves all its functions by implementing different steps in the simulation method through the acquisition unit 1501 and the processing unit 1502. The simulation system 1500 is used to execute the aforementioned... Figure 2 The operations performed by the cloud platform in the illustrated embodiments, or Figures 5 to 13 The operations performed by the simulation system in the illustrated embodiment are to implement the simulation method provided in the embodiments of this application, as detailed above, and will not be repeated here.

[0204] Please see Figure 16 , Figure 16 This is a schematic diagram of a computing device provided in an embodiment of this application. The computing device 1600 includes a processor 1601, a communication interface 1602, a bus 1603, and a memory 1604. The processor 1601, the communication interface 1602, and the memory 1604 communicate with each other via the bus 1603. In practical applications, communication can also be achieved through other means such as wireless transmission; the specific method is not limited here.

[0205] The computing device 1600 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memory in the computing device 1600.

[0206] Processor 1601 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0207] The communication interface 1602 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1600 and other devices or communication networks.

[0208] The 1603 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 16 The bus 1603 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1603 may include a path for transmitting information between various components of the computing device 1600 (e.g., memory 1604, processor 1601, communication interface 1602).

[0209] Memory 1604 may include volatile memory, such as random access memory (RAM). Memory 1604 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0210] The memory 1604 stores executable program code, and the processor 1601 executes the executable program code to implement the functions of the aforementioned acquisition unit 1501 and processing unit 1502, thereby realizing the simulation method. That is, the memory 1604 stores instructions for executing the simulation method.

[0211] This application also provides a computing device cluster, which includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some optional embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0212] Please see Figure 17 and Figure 18 , Figure 17 and Figure 18 These are all schematic diagrams of the computing device clusters provided in the embodiments of this application.

[0213] like Figure 17 As shown, the computing device cluster includes at least one computing device 1600. The memory 1604 of one or more computing devices 1600 in the computing device cluster may store the same instructions for executing the simulation method provided in the embodiments of this application.

[0214] In some possible implementations, the memory 1604 of one or more computing devices 1600 in the computing device cluster may also store partial instructions for executing the simulation method. In other words, a combination of one or more computing devices 1604 can jointly execute the instructions for executing the simulation method.

[0215] It should be noted that the memory 1604 in different computing devices 1600 within the computing device cluster can store different instructions, which are used to execute parts of the cloud platform or simulation system. That is, the instructions stored in the memory 1604 of different computing devices 1600 can realize the functions of one or more units among the acquisition unit 1501 and processing unit 1502, so as to realize the simulation method provided in the embodiments of this application.

[0216] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 18 One possible implementation is shown. For example... Figure 18 As shown, the two computing devices 1600A and 1600B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 1604 in computing device 1600A stores instructions for executing the functions of the acquisition unit 1501. Simultaneously, the memory 1604 in computing device 1600B stores instructions for executing the functions of the processing unit 1502.

[0217] Figure 18 The connection method between the computing device clusters shown can be based on the simulation method provided in this application, in which the processing operation and the operation other than the processing operation are executed separately. That is, the function of the acquisition unit 1501 is considered to be executed by the computing device 1600A, and the function of the processing unit 1502 is considered to be executed by the computing device 1600B.

[0218] It should be understood that Figure 18 The functions of computing device 1600A shown can also be performed by multiple computing devices 1600. Similarly, the functions of computing device 1600B can also be performed by multiple computing devices 1600.

[0219] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 17 and Figure 18 The connection method of the computing device cluster will not be described in detail here.

[0220] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computer device, it causes the at least one computer device to execute the simulation method provided in this application.

[0221] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the simulation method provided in this application.

[0222] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A simulation method, characterized in that, The method is executed by a simulation system used to simulate the performance of an AI task running on a cluster. The AI ​​task includes training or inference of an AI model. The cluster includes multiple computing devices connected through at least one switch. The AI ​​task includes multiple subtasks, and communication between the multiple subtasks includes multiple communication steps. The method includes: The system receives input information, which includes the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the rank table, the resource topology, and the collective communication algorithm used by the AI ​​model. The rank table includes the mapping relationship between the identifiers of the multiple subtasks and the address information of the multiple computing devices. The resource topology indicates the connection relationship between the multiple computing devices and the at least one switch, and between the at least one switch and each other. Based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the aggregate communication algorithm, calculate the first duration of each communication step in the plurality of communication steps, where the first duration is the execution time of the corresponding communication step; Output the total communication duration of the AI ​​task, where the total communication duration is the sum of the first durations of each communication step.

2. The method according to claim 1, characterized in that, The step of calculating the first duration of each communication step in the plurality of communication steps based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the ensemble communication algorithm includes: Based on the parameters of the AI ​​model and the parallel strategy of the AI ​​model, the AI ​​task is decomposed into the multiple sub-tasks; Based on the ranktable and the resource topology, the communication topology of the plurality of subtasks is determined, wherein the communication topology indicates the communication connection relationship between the plurality of subtasks when the plurality of subtasks are running on the plurality of computing devices; The first duration of each communication step is calculated based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ensemble communication algorithm, and the communication topology.

3. The method according to claim 2, characterized in that, The resource topology also indicates the transmission bandwidth and transmission latency between any two devices that are connected to the plurality of computing devices and the at least one switch; The communication topology also indicates the transmission bandwidth and transmission latency between any two connected subtasks when the plurality of subtasks are running on the plurality of computing devices; The step of calculating the first duration of each communication step based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ensemble communication algorithm, and the communication topology includes: According to the set communication algorithm, at least one communication pair is determined for each communication step, wherein each communication pair includes a subtask for sending data and a subtask for receiving data; Based on the communication topology, determine the communication link for each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link; Based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, the total communication volume of the at least one communication pair is determined, and the total communication volume of the at least one communication pair is the sum of the communication volumes of each communication pair. The first duration of each communication step is determined based on the total communication volume of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link.

4. The method according to claim 3, characterized in that, The method further includes: The transmission time of each communication pair is determined based on the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair. Determine a second duration for each communication step, where the second duration is the transmission time of the longest communication pair in the corresponding communication step; Output the average collision coefficient for each communication step, where the average collision coefficient is the quotient of the first duration and the second duration.

5. The method according to claim 3 or 4, characterized in that, The method further includes: outputting the first duration of each communication step.

6. The method according to any one of claims 2 to 5, characterized in that, The parallel strategy of the AI ​​model includes multiple communication methods corresponding to the multiple communication steps; the methods further include: Based on the first duration of each communication step, a third duration is determined for each of the plurality of communication methods, wherein the third duration is the sum of the first durations of communication steps using the same communication method; Output the third duration corresponding to each communication method.

7. The method according to any one of claims 4 to 6, characterized in that, The method further includes: outputting the communication link of each communication pair and / or the amount of communication transmitted by each communication pair.

8. The method according to any one of claims 4 to 7, characterized in that, The communication topology also indicates the set of ports corresponding to the communication links of each communication pair, the set of ports including the ports of the computing device that sends data, the ports of the computing device that receives data, and the ports of at least one switch that transmits data; The method further includes: Based on the communication volume of each communication pair, determine the communication volume of each port in the port set corresponding to each communication link; Based on the communication volume of each port in the port set corresponding to each communication link, determine the total communication volume of each port of each of the multiple computing devices and the total communication volume of each port of each of the at least one switch after running the AI ​​task.

9. The method according to claim 8, characterized in that, The method further includes: Output the total traffic of a first port of the target computing device, and / or the total traffic of a second port of the target switch, wherein the target computing device is included in the plurality of computing devices and the target switch is included in the at least one switch.

10. A simulation system, characterized in that, The simulation system is used to simulate the performance of AI tasks running on a cluster. The AI ​​tasks include training or inference of AI models. The cluster includes multiple computing devices connected via at least one switch. Each AI task includes multiple subtasks, and communication between these subtasks includes multiple communication steps. The simulation system includes: An acquisition unit is used to receive input information, which includes the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the collective communication algorithm used by the AI ​​model. The ranktable includes the mapping relationship between the identifiers of the multiple subtasks and the address information of the multiple computing devices. The resource topology indicates the connection relationship between the multiple computing devices and the at least one switch, and between the at least one switch and each other. The processing unit is configured to calculate the first duration of each communication step in the plurality of communication steps based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology and the collective communication algorithm, wherein the first duration is the execution time of the corresponding communication step; The processing unit is also configured to output the total communication duration of the AI ​​task, wherein the total communication duration is the sum of the first durations of each communication step.

11. The simulation system according to claim 10, characterized in that, The processing unit is specifically used for: Based on the parameters of the AI ​​model and the parallel strategy of the AI ​​model, the AI ​​task is decomposed into the multiple sub-tasks; Based on the ranktable and the resource topology, the communication topology of the plurality of subtasks is determined, wherein the communication topology indicates the communication connection relationship between the plurality of subtasks when the plurality of subtasks are running on the plurality of computing devices; The first duration of each communication step is calculated based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ensemble communication algorithm, and the communication topology.

12. The simulation system according to claim 11, characterized in that, The resource topology also indicates the transmission bandwidth and transmission latency between any two devices that are connected to the plurality of computing devices and the at least one switch; The communication topology also indicates the transmission bandwidth and transmission latency between any two connected subtasks when the plurality of subtasks are running on the plurality of computing devices; The processing unit is specifically used for: According to the set communication algorithm, at least one communication pair is determined for each communication step, wherein each communication pair includes a subtask for sending data and a subtask for receiving data; Based on the communication topology, determine the communication link for each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link; Based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, the total communication volume of the at least one communication pair is determined, and the total communication volume of the at least one communication pair is the sum of the communication volumes of each communication pair. The first duration of each communication step is determined based on the total communication volume of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link.

13. The simulation system according to claim 12, characterized in that, The processing unit is further configured to: The transmission time of each communication pair is determined based on the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair. Determine a second duration for each communication step, where the second duration is the transmission time of the longest communication pair in the corresponding communication step; Output the average collision coefficient for each communication step, where the average collision coefficient is the quotient of the first duration and the second duration.

14. The simulation system according to claim 12 or 13, characterized in that, The processing unit is also used to output the first duration of each communication step.

15. The simulation system according to any one of claims 11 to 14, characterized in that, The parallel strategy of the AI ​​model includes multiple communication methods corresponding to the multiple communication steps; the processing unit is further configured to: Based on the first duration of each communication step, a third duration is determined for each of the plurality of communication methods, wherein the third duration is the sum of the first durations of communication steps using the same communication method; Output the third duration corresponding to each communication method.

16. The simulation system according to any one of claims 13 to 15, characterized in that, The processing unit is also configured to output the communication link of each communication pair and / or the amount of communication transmitted by each communication pair.

17. The simulation system according to any one of claims 13 to 16, characterized in that, The communication topology further indicates the port set corresponding to the communication link of each communication pair, the port set including the port of the computing device sending data, the port of the computing device receiving data, and the port of at least one switch transmitting data; the processing unit is further configured to: Based on the communication volume of each communication pair, determine the communication volume of each port in the port set corresponding to each communication link; Based on the communication volume of each port in the port set corresponding to each communication link, determine the total communication volume of each port of each of the multiple computing devices and the total communication volume of each port of each of the at least one switch after running the AI ​​task.

18. The simulation system according to claim 17, characterized in that, The processing unit is further configured to: Output the total traffic of a first port of the target computing device, and / or the total traffic of a second port of the target switch, wherein the target computing device is included in the plurality of computing devices and the target switch is included in the at least one switch.

19. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 9.

20. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster performs the method as described in any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1 to 9.

Images