Simulation method and related device

The simulation system simulates the process of AI tasks running on a cluster, calculates the duration of each communication step, solves the problem of inaccurate performance evaluation of AI models running on a cluster, and achieves more accurate performance evaluation.

WO2026036655A9PCT designated stage Publication Date: 2026-11-12HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/072440
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-08-16
Filing Date
2025-01-15
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

In existing technologies, the performance evaluation of AI models running on clusters is not accurate enough, resulting in inaccurate evaluation results.

Method used

The simulation system simulates the process of AI tasks running on a cluster, including simulating the communication steps between multiple subtasks, calculating the first duration of each communication step, and outputting the total communication duration of the AI ​​task, thereby improving the accuracy of performance evaluation.

Benefits of technology

It enables accurate prediction of AI task communication duration, improves the precision of performance evaluation, and allows for more refined detection of AI task performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025072440_12112026_PF_FP_ABST
    Figure CN2025072440_12112026_PF_FP_ABST
Patent Text Reader

Abstract

The present application provides a simulation method and a related device, for use in improving the accuracy of performance evaluation for AI tasks. The method is applied to a simulation system, the simulation system is used for simulating the performance of an AI task when running on a cluster, a plurality of computing devices comprised in the cluster are connected by means of at least one switch, and the communication between a plurality of subtasks comprised in the AI task comprises a plurality of communication steps. The method comprises: receiving input information, wherein the input information comprises parameters of an AI model, a parallel strategy of the AI model, a ranktable, a resource topology, and a collective communication algorithm used by the AI model, the ranktable comprises mapping relationships between identifiers of a plurality of subtasks and address information of a plurality of computing devices, and the resource topology indicates connection relationships between the plurality of computing devices and at least one switch, and between the at least one switch; on the basis of the input information, calculating a first duration indicating the execution time of each communication step; and outputting a total communication duration of an AI task, the total communication duration being the sum of the first durations of the communication steps.
Need to check novelty before this filing date? Find Prior Art

Description

A simulation method and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202411135093.9, filed with the State Intellectual Property Office of China on August 16, 2024, entitled "A Simulation Method and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI), and in particular to the design of a simulation method and related equipment. Background Technology

[0003] With the development of artificial intelligence technology, the scale of AI models is gradually increasing. The number of parameters in AI models, as well as the number of devices required for inference or training, are also gradually increasing. Training or inference on large-scale clusters typically employs a distributed parallel approach. However, the actual performance of an AI model cannot be determined until it is actually running on the cluster.

[0004] In the relevant technical solutions, the parameters of the AI ​​model and the hardware parameters required to run the AI ​​model are input into the simulation device. The simulation device evaluates the running performance of the AI ​​model on the cluster by simulating the training or inference process of the AI ​​model.

[0005] However, in this technical solution, the evaluation of the AI ​​model's performance on the cluster is a rather general result, reflecting the overall performance of the AI ​​model on the cluster, and the accuracy of the evaluation is not high. Summary of the Invention

[0006] This application provides a simulation method and related equipment for simulating the process of an AI task running on a cluster using a simulation system. This includes simulating multiple communication steps between multiple subtasks within the AI ​​task. By calculating the first duration of each communication step, the total communication duration of the AI ​​task is obtained. This achieves accurate prediction of communication duration, improving the accuracy of performance evaluation for the AI ​​task.

[0007] Firstly, this application provides a simulation method executed by a simulation system used to simulate the performance of an AI task running on a cluster. The AI ​​task includes training or inference of an AI model, and comprises multiple subtasks. Communication between the subtasks includes multiple communication steps. The cluster includes multiple computing devices connected via at least one switch.

[0008] The simulation system receives input information and performs simulations based on it. The input information includes the AI ​​model's parameters, parallel strategy, rank table, resource topology, and the collective communication algorithm used by the AI ​​model. The rank table maps the identifiers of multiple subtasks to the address information of multiple computing devices. The resource topology indicates the connections between multiple computing devices and at least one switch, and between at least one switch and each other. The collective communication algorithm defines how many communication steps are included in each subtask, and which communication pairs are included in each step; in other words, it defines how multiple subtasks complete multiple communication steps. Based on the AI ​​model's parameters, parallel strategy, rank table, resource topology, and collective communication algorithm, the simulation system calculates the first duration of each communication step. The first duration is the execution time of the corresponding communication step; that is, the first duration is the actual execution time of each communication step during the simulation of the AI ​​task running on the cluster. The simulation system outputs the total communication duration of the AI ​​task, which is the sum of the first durations of each communication step.

[0009] This application uses a simulation system to simulate the process of an AI task running on a cluster, including simulating multiple communication steps between various subtasks within the AI ​​task. By calculating the first duration of each communication step, the total communication duration of the AI ​​task is obtained. This enables accurate prediction of communication duration and improves the accuracy of performance evaluation for the AI ​​task.

[0010] In some optional implementations of the first aspect, the simulation system calculates the first duration of each communication step in multiple communication steps, specifically including: decomposing the AI ​​task into multiple subtasks based on the AI ​​model's parameters and parallel strategy. It can be understood that the AI ​​model's parameters indicate the amount of data the AI ​​task needs to process, and the AI ​​model's parallel strategy indicates how the amount of data processed by the AI ​​task is divided. Therefore, the aforementioned process can be understood as dividing the amount of data the AI ​​task needs to process into multiple parts based on the AI ​​model's parallel strategy, with each part corresponding to a subtask. Based on the ranktable and resource topology, the communication topology of the multiple subtasks is determined. The communication topology indicates the communication connection relationship between the multiple subtasks when they run on multiple computing devices. Based on the AI ​​model's parameters, parallel strategy, ensemble communication algorithm, and communication topology, the first duration of each communication step is calculated.

[0011] In this application, the AI ​​task is broken down into multiple subtasks, and then the communication topology of the subtasks is constructed to determine the communication connections between them. Then, the first duration of each communication step in the multiple communication steps of the subtasks is calculated. In other words, the simulation system calculates the duration of communication steps at the subtask level, resulting in a finer granularity for detecting the performance of the AI ​​task and further improving the accuracy of performance evaluation.

[0012] In some alternative implementations of the first aspect, the resource topology further indicates the transmission bandwidth and transmission delay between any two connected devices among multiple computing devices and at least one switch. Correspondingly, the communication topology further indicates the transmission bandwidth and transmission delay between any two connected subtasks among multiple subtasks running on multiple computing devices. The simulation system calculates the first duration of each communication step, specifically by first determining, according to the ensemble communication algorithm, that each communication step includes at least one communication pair, where each communication pair includes a subtask for sending data and a subtask for receiving data. Then, according to the communication topology, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link are determined. Next, according to the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, the total communication volume of at least one communication pair is determined, where the total communication volume of at least one communication pair is the sum of the communication volumes of each communication pair. Finally, based on the total communication volume of at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link, the first duration of each communication step is determined. The first duration can be understood as the duration of the total communication volume of at least one communication pair in a communication step, which is transmitted through the communication link of that communication step.

[0013] In this application, the simulation system can also determine at least one communication pair included in each communication step, and, combining the total communication volume of these at least one communication pair, the transmission link of each communication pair, and the transmission delay, determine the first duration of each communication step. The calculation of the first duration is more precise, further improving the accuracy of performance evaluation for AI tasks.

[0014] In some optional implementations of the first aspect, the simulation system can also determine the transmission time of each communication pair based on the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair. The transmission time of each communication pair here refers to the time for transmitting the communication volume of each communication pair when the link of each communication pair is exclusively used by the communication process of that communication pair; this can be considered the ideal transmission time of each communication pair. A second duration for each communication step is determined from the transmission times of at least one communication pair included in each communication step; the second duration is the transmission time of the communication pair with the longest duration in the corresponding communication step. The average conflict coefficient of each communication step is output; the average conflict coefficient is the quotient of the first duration and the second duration. Therefore, the average conflict coefficient reflects the multiple of the actual time consumption of the communication step compared to the ideal time consumption. The larger the average conflict coefficient, the longer the communication step is considered to be, which can intuitively reflect the performance of the current AI task.

[0015] In this application, the simulation system can also output the average conflict coefficient for each communication step. Since the average conflict coefficient reflects the ratio of the actual time taken for a communication step to the ideal time taken, it can intuitively reflect the performance of the AI ​​task in terms of communication time. The larger the average conflict coefficient, the longer the communication step is considered to take, which can be used as a potential optimization point for the AI ​​task.

[0016] In some alternative implementations of the first aspect, the simulation system may also output the first duration of each communication step.

[0017] In this application, the simulation system, in addition to outputting the total communication duration of the AI ​​task, can also output the first duration of each communication step, enriching the dimensions of performance evaluation for the AI ​​task. Furthermore, based on the first duration of each communication step, adjustments can be made to communication steps that are too long, thereby optimizing the performance of the AI ​​task running on the cluster.

[0018] In some optional implementations of the first aspect, the parallel strategy of the AI ​​model includes multiple communication methods corresponding to multiple communication steps. The simulation system can also statistically analyze the first duration of each communication step from the dimension of communication methods, determine the third duration corresponding to each of the multiple communication methods, and output the third duration corresponding to each communication method. The third duration is the sum of the first durations of communication steps using the same communication method.

[0019] In this application, the simulation system can also output a third duration for each communication method, evaluating the performance of the AI ​​task from more perspectives and enriching the performance evaluation results. Furthermore, the third duration for each communication method can also serve as a reference metric for optimizing the AI ​​task; for example, optimizing time-consuming communication methods can improve the performance of the AI ​​task running on the cluster.

[0020] In some optional implementations of the first aspect, the simulation system can also output the communication link for each communication pair and / or the amount of communication transmitted by each communication pair. This further enriches the dimensions of the simulation system's performance evaluation of AI tasks and can also intuitively reflect the communication process between multiple subtasks in an AI task.

[0021] In some alternative implementations of the first aspect, the communication topology further indicates the set of ports corresponding to the communication links of each communication pair, the port set including ports of the computing devices sending data, ports of the computing devices receiving data, and ports of at least one switch transmitting data. The simulation system determines the communication volume of each port in the port set corresponding to each communication link based on the communication volume of each communication pair. Based on the communication volume of each port in the port set corresponding to each communication link, the system determines the total communication volume of each port of each computing device among the multiple computing devices, and the total communication volume of each port of each switch among the at least one switch, after running the AI ​​task.

[0022] In this application, the simulation system can also determine the communication volume of the ports of each device included in each communication link, thereby determining the total communication volume of the ports of each device after running the AI ​​task, thus reflecting the data transmission situation between multiple subtasks during the operation of the AI ​​task.

[0023] In some optional implementations of the first aspect, the simulation system may also output the total traffic of the first port of the target computing device and / or the total traffic of the second port of the target switch, wherein the target computing device comprises multiple computing devices and the target switch comprises at least one switch. This further enriches the dimensions of the simulation system's performance evaluation of AI tasks and enhances the performance evaluation results.

[0024] Secondly, this application provides a simulation system for simulating the performance of AI tasks running on a cluster. The AI ​​task includes training or inference of an AI model. The cluster includes multiple computing devices connected via at least one switch. The AI ​​task includes multiple subtasks, and communication between the subtasks includes multiple communication steps. The simulation system includes:

[0025] The acquisition unit is used to receive input information, which includes the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the collective communication algorithm used by the AI ​​model. The ranktable includes the mapping relationship between the identifiers of multiple subtasks and the address information of multiple computing devices. The resource topology indicates the connection relationship between multiple computing devices and at least one switch, and between at least one switch and each other.

[0026] The processing unit is used to calculate the first duration of each communication step in multiple communication steps based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology and the collective communication algorithm. The first duration is the execution time of the corresponding communication step.

[0027] The processing unit is also used to output the total communication duration of the AI ​​task, which is the sum of the first duration of each communication step.

[0028] The simulation system is used to implement the method shown in the first aspect above, or any possible implementation of the first aspect, as detailed above, and will not be repeated here.

[0029] Thirdly, this application provides a computing device including a processor and a memory. The processor of the computing device is used to execute instructions stored in the memory, causing the computing device to implement the method shown in the first aspect or any possible implementation of the first aspect.

[0030] Fourthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster implements the method disclosed in the first aspect or any possible implementation of the first aspect.

[0031] Fifthly, this application provides a computer program product containing instructions that, when executed by a cluster of computer devices, cause the cluster of computer devices to implement the method disclosed in the first aspect and any possible implementation of the first aspect.

[0032] In a sixth aspect, this application provides a computer-readable storage medium storing computer program instructions that, when executed by a cluster of computer devices, cause the cluster of computer devices to implement the method disclosed in the first aspect and any possible implementation of the first aspect.

[0033] The beneficial effects shown in any of the second to sixth aspects are similar to those of the first aspect or any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0034] Figure 1 is a schematic diagram of the main architecture of artificial intelligence provided in an embodiment of this application;

[0035] Figure 2 is a schematic diagram of a system architecture provided in an embodiment of this application;

[0036] Figure 3 is a schematic diagram of another system architecture provided in an embodiment of this application;

[0037] Figure 4 is a schematic diagram of another system architecture provided in an embodiment of this application;

[0038] Figure 5 is a flowchart illustrating a simulation method provided in an embodiment of this application.

[0039] Figure 6 is a schematic diagram provided in an embodiment of this application;

[0040] Figure 7 is another flowchart illustrating the simulation method provided in an embodiment of this application;

[0041] Figure 8 is a schematic diagram of a communication topology provided in an embodiment of this application;

[0042] Figure 9 is a schematic diagram of the simulation output results provided in the embodiment of this application;

[0043] Figure 10 is another schematic diagram of the communication topology provided in an embodiment of this application;

[0044] Figure 11 is another flowchart illustrating the simulation method provided in an embodiment of this application;

[0045] Figure 12 is another flowchart illustrating the simulation method provided in an embodiment of this application;

[0046] Figure 13 is another schematic diagram of the simulation output results provided in the embodiment of this application;

[0047] Figure 14 is a schematic diagram of the simulation system provided in an embodiment of this application;

[0048] Figure 15 is another structural schematic diagram of the simulation system provided in the embodiment of this application;

[0049] Figure 16 is a schematic diagram of a computing device provided in an embodiment of this application;

[0050] Figure 17 is a schematic diagram of a computing device cluster provided in an embodiment of this application;

[0051] Figure 18 is another schematic diagram of the computing device cluster provided in the embodiment of this application. Detailed Implementation

[0052] This application provides a simulation method and related equipment for simulating the execution of an AI task on a cluster using a simulation system. This includes simulating multiple communication steps between multiple subtasks within the AI ​​task. By calculating the duration of each communication step, the total communication duration of the AI ​​task is obtained. This achieves accurate prediction of communication duration, improving the accuracy of performance evaluation for the AI ​​task.

[0053] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0054] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses. Additionally, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be expressed as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0055] First, the relevant concepts and proper nouns involved in the embodiments of this application will be explained.

[0056] 1) Distributed parallelism:

[0057] Distributed parallelism is a technique that distributes the training or inference process of an AI model across multiple computing devices. It can also be understood as breaking down the training or inference process into multiple sub-processes and distributing these sub-processes across various computing devices for execution. More specifically, distributed parallelism includes distributed computing and parallel computing. This technique allows AI models to leverage the computing power of multiple devices to accelerate large-scale data processing and the execution of complex models. Distributed parallelism includes tensor parallelism (TP), pipeline parallelism (PP), and data parallelism (DP), among other parallelism methods.

[0058] 2) Rank table:

[0059] A ranktable is a file used to manage communication between computing nodes participating in the training or inference of an AI model. It records information about all nodes involved in the training or inference process, including each node's logical identifier (rank ID) and address information. Ranktable ensures that all nodes participating in AI model training or inference can communicate and synchronize effectively, thus achieving high-efficiency parallelism.

[0060] 3) Communication domain:

[0061] A communication domain refers to a set of computing devices that can communicate with each other, and a communication domain is responsible for a specific communication task. Multiple computing devices in a communication domain synchronize information such as AI model parameters and gradients through communication and data exchange. During the training or inference process of an AI model, the multiple computing nodes used by the AI ​​model can include multiple communication domains, and a computing device can belong to one or more communication domains.

[0062] 4) Collective communication algorithms:

[0063] Ensemble communication algorithms are communication algorithms used in distributed and parallel computing for data exchange between multiple processes or threads. In these algorithms, a group of processes jointly participate in communication operations to collect, distribute, or aggregate data. In the training or inference of AI models, ensemble communication algorithms define the sub-processes for communication, as well as communication steps and other communication information related to the training or inference process of the AI ​​model.

[0064] Next, please refer to Figure 1, which is a schematic diagram of the main architecture of artificial intelligence provided in an embodiment of this application. This main framework describes the overall workflow of the artificial intelligence system and is applicable to general artificial intelligence domain needs.

[0065] The above-mentioned artificial intelligence framework will be elaborated from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0066] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it could be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom."

[0067] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (provided and processed by technology) to the industrial ecosystem of systems.

[0068] (1) Infrastructure.

[0069] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, including central processing units (CPUs), neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs) and other hardware acceleration chips. The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0070] (2) Data.

[0071] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0072] (3) Data processing.

[0073] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0074] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0075] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0076] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0077] (4) General ability.

[0078] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0079] (5) Smart products and industry applications.

[0080] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They are the encapsulation of overall artificial intelligence solutions, productizing intelligent information decision-making and realizing practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, smart cities, and intelligent terminals.

[0081] The simulation method provided in this application embodiment can be applied to cloud computing. The system architecture of cloud computing is described below with reference to the schematic diagram. Please refer to Figure 2, which is a schematic diagram of the system architecture provided in this application embodiment.

[0082] As shown in Figure 2, a tenant logs into the cloud platform 203 via client 201 through the Internet 202 using the account and password registered on the cloud platform 203. The cloud platform 203 manages the infrastructure, which includes multiple data centers located in different regions. For example, region 1 in Figure 2 includes cloud data center 1 and cloud data center 2, and region 2 includes cloud data center 3 and cloud data center 4. Each cloud data center has multiple servers running business instances (including at least one of virtual machines, containers, and dedicated servers).

[0083] In this embodiment, a simulation service is deployed in the business instance to simulate the performance of AI tasks running on a cluster. The tenant purchases cloud services through client 201 on cloud platform 203 and sends a request to cloud platform 203, which requests cloud services from cloud platform 203. In this embodiment, the cloud services specifically include simulating the execution process of AI tasks to evaluate their performance on the cluster. The specific implementation process will be described later.

[0084] In this embodiment, the cloud platform manages multiple computing devices and at least one switch. The multiple computing devices are used for the functions of multiple sub-tasks included in the AI ​​task, and the at least one switch is used to connect the multiple computing devices to enable communication between them. The connection relationships between the multiple computing devices and the at least one switch, as well as the connection relationships between the at least one switch and themselves, are various, and will be explained below with reference to the schematic diagram.

[0085] Please refer to Figures 3 and 4, which are schematic diagrams of the system architecture provided in the embodiments of this application.

[0086] The embodiment shown in Figure 3 includes two switching layers, an architecture also known as a "spine-leaf" architecture. These two switching layers form the network topology of the data center. For example, switch 1 in Figure 3 can be considered a switch in the spine layer, while switches 2 and 3 can be considered switches in the leaf layer.

[0087] As shown in Figure 3, computing devices 1 through 3 are all connected to switch 2, switch 2 is connected to switch 1, switch 1 is also connected to switch 3, and switch 3 is also connected to computing devices 4 and 5. Through the connections shown in Figure 3, communication between the different computing devices is achieved.

[0088] It should be noted that this application does not limit the system architecture for running AI tasks; Figure 3 is merely a schematic diagram of the system architecture. In practical applications, it may include more or fewer switches and computing devices, and the number of switch layers may also be more or fewer; specific details are not limited here.

[0089] For example, in the embodiment shown in Figure 4, a three-layer network architecture is used, which includes three layers of switches.

[0090] Please refer to Figure 5 below. Figure 5 is a flowchart illustrating the simulation method provided in the embodiments of this application, including:

[0091] 501. Receive input information, including the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the set communication algorithm used by the AI ​​model.

[0092] The simulation method provided in this application is executed by a simulation system, simulating the performance of an AI task running on a cluster. An AI task refers to either the training task or the inference task of an AI model. Furthermore, an AI task includes multiple sub-tasks, and communication between these sub-tasks involves multiple communication steps, based on which the complete communication process of the AI ​​task is achieved. The cluster running the AI ​​task includes multiple computing devices connected through at least one switch. These computing devices can be processors, servers, virtual machines, containers, etc., and are not specifically limited here.

[0093] The simulation system receives input information, simulates the execution of AI tasks on a cluster, and outputs the performance of the AI ​​tasks running on the cluster. Therefore, the input information describes the AI ​​task and the cluster running the AI ​​task. Specifically, the input information includes: the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the ensemble communication algorithm used by the AI ​​model.

[0094] The parameters of an AI model define the model and represent its complexity; in simpler terms, they define what kind of model the AI ​​is. For example, AI model parameters may include at least one of the following: input layer parameters, hidden layer parameters, neural network layer parameters, loss function parameters, output layer parameters, optimizer parameters, or regularization parameters. The specific parameters included may vary between different AI models, and are not limited here.

[0095] The parallel strategy of an AI model indicates how the AI ​​model is distributed across multiple computing devices in parallel. It can also be understood as the communication method between multiple subtasks. For example, communication methods include tensor parallelism, pipelined parallelism, and data parallelism. An AI model can employ one or more parallel methods; no specific limitation is made here.

[0096] The ranktable includes a mapping between the identifiers of multiple subtasks and the address information of multiple computing devices; it can be understood as a mapping between logical identifiers and physical identifiers. The address information can be an Internet Protocol (IP) address, a Media Access Control (MAC) address, or other information that can represent the address of a computing device; no specific limitations are made here.

[0097] Resource topology indicates the connectivity between multiple computing devices and at least one switch, and between at least one switch and each other. In other words, resource topology defines the network architecture for running AI tasks.

[0098] The ensemble communication algorithm used by AI models defines how many communication steps are between multiple subtasks included in an AI task, and which communication pairs (data sender and data receiver) are included in each communication step. In other words, the ensemble communication algorithm defines how multiple subtasks complete multiple communication steps.

[0099] 502. Based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the collective communication algorithm, calculate the first duration of each communication step in multiple communication steps. The first duration is the execution time of the corresponding communication step.

[0100] The simulation system acquires input information and simulates multiple communication steps between various subtasks in the AI ​​task, thereby obtaining the initial duration of each communication step. This process is explained in detail below:

[0101] The simulation system breaks down the AI ​​task into multiple sub-tasks based on the AI ​​model's parameters and parallelism strategy. The AI ​​model's parameters indicate the amount of data the AI ​​task needs to process, and the AI ​​model's parallelism strategy indicates how this data is divided. Based on this division method, the simulation system divides the amount of data the AI ​​task needs to process into multiple data sets, with each data set corresponding to a sub-task.

[0102] For example, please refer to Figure 6, which is a schematic diagram provided in an embodiment of this application. Assume that the parallel strategy for the AI ​​task includes data parallelism and pipelined parallelism. In the embodiment shown in Figure 6, the AI ​​task is represented by a cube. Based on pipelined parallelism, the AI ​​task is decomposed into subtask 1 and subtask 2. Then, based on data parallelism, it is further decomposed into subtask 1-1, subtask 1-2, subtask 2-1, and subtask 2-2.

[0103] The simulation system determines the communication topology of multiple subtasks based on the ranktable and resource topology. The communication topology indicates the communication connection relationships between the multiple subtasks when they run on multiple computing devices. This process is explained in detail with reference to Figure 7, which is a flowchart illustrating the simulation method provided in this embodiment.

[0104] In the embodiment shown in Figure 7, rankID represents the identifier of the subtask, and the IP address represents the address information of the computing device. In the ranktable, the IP address corresponding to subtask 1 is 29.59.95.178, the IP address corresponding to subtask 2 is 29.59.115.206, and the IP address corresponding to subtask 3 is 29.59.176.221. These are the IP addresses of computing devices A, B, and C, respectively. This means that subtask 1 runs on computing device A, subtask 2 runs on computing device B, and subtask 3 runs on computing device C. Therefore, combining the ranktable and resource topology shown in Figure 7, we obtain the communication topology shown in Figure 7.

[0105] It should be noted that the ranktable in the embodiments of this application is not limited to the form of a table, but can also be a key-value pair or other representation that can reflect the mapping relationship. No specific limitation is made here.

[0106] After obtaining the communication topology of multiple subtasks, the simulation system can calculate the first duration of each communication step based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ensemble communication algorithm, and the communication topology. This process is explained below.

[0107] In this embodiment, the resource topology further indicates the transmission bandwidth and transmission latency between any two devices connected to a plurality of computing devices and at least one switch. That is, the resource topology indicates the transmission bandwidth and transmission latency between connected computing devices and switches, and the transmission bandwidth and transmission latency between connected switches.

[0108] Accordingly, the communication topology also indicates the transmission bandwidth and transmission latency between any two connected subtasks when multiple subtasks are running on multiple computing devices. Specifically, if two connected subtasks are connected through at least one switch, then the transmission bandwidth between the two connected subtasks includes the transmission bandwidth between the computing device running the two subtasks and the switch, as well as the transmission bandwidth between the at least one switch and itself. The transmission latency between the two connected subtasks includes the transmission latency between the computing device running the two subtasks and the switch, as well as the transmission latency between the at least one switch and itself.

[0109] For example, please refer to Figure 8, which is a schematic diagram of the communication topology provided in an embodiment of this application.

[0110] As shown in Figure 8, subtasks 1, 2, and 3 run on computing devices A, B, and C, respectively, which are connected via switches 1 and 2. The resource topology defines the transmission bandwidth and latency between computing device A and switch 1, between switch 1 and switch 2, between computing device B and switch 2, and between computing device B and switch 3.

[0111] Accordingly, the transmission bandwidth and latency of subtask 1 and subtask 2 include the transmission bandwidth and latency between computing device A and switch 1, between switch 1 and switch 2, and between computing device B and switch 2. The transmission bandwidth and latency of subtask 1 and subtask 3 include the transmission bandwidth and latency between computing device A and switch 1, between switch 1 and switch 2, and between computing device C and switch 2.

[0112] In summary, the simulation system, based on the ensemble communication algorithm, determines at least one communication pair included in each communication step, where each communication pair includes a subtask of sending data and a subtask of receiving data. Based on the communication topology, it determines the communication link, transmission bandwidth, and transmission delay of each communication link. Based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, it determines the total communication volume of at least one communication pair, which is the sum of the communication volumes of each communication pair. Based on the total communication volume of at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link, it determines the first duration of each communication step.

[0113] The process described above will be explained in detail below using Figure 8 as an example. In the embodiment shown in Figure 8, it is assumed that a communication step includes two communication pairs, namely <Subtask 1-Subtask 2> and <Subtask 1-Subtask 3>. Based on the above description of the example in Figure 8, the simulation system can determine the transmission delay and transmission bandwidth of each of the two communication pairs. Then, based on the parameters and parallel strategy of the AI ​​model, the total communication volume and communication domain of each communication method are determined. Within the same communication domain, different subtasks communicate using the same communication method. Furthermore, subtasks cannot communicate across communication domains. Combined with the ensemble communication algorithm, the communication volume of each communication pair is determined, that is, the amount of data transmitted by each communication pair. Finally, the first duration of each communication step is determined. The first duration is the actual execution time of each communication step simulated by the simulation system, or it can be understood as the time when each communication step is completed. In a scheme where a communication step includes multiple communication pairs, the first duration is the longest transmission time among these multiple communication pairs.

[0114] In the embodiment shown in Figure 8, the data transmitted in the communication pair <Subtask 1-Subtask 2> is data 1, and the communication link is "Computing Device A-Switch 1-Switch 2-Computing Device B", which includes three sub-links: "Computing Device A-Switch 1", "Switch 1-Switch 2", and "Switch 2-Computing Device B". The communication pair <Subtask 1-Subtask 3> is similar and will not be described again here. Therefore, two sets of data are transmitted on the sub-links "Computing Device A-Switch 1" and "Switch 1-Switch 2". The sum of the data amounts of these two sets of data is the total communication volume of this communication step.

[0115] The transmission time of each communication pair is the longest among the multiple sub-links. The transmission time of each sub-link is based on the communication volume, transmission bandwidth, and transmission delay of each sub-link, and their relationship satisfies: Transmission Time = Communication Volume / Transmission Bandwidth + Transmission Delay.

[0116] For example, the transmission time of the sub-link "Computing Device A - Switch 1" is the quotient of the communication volume of data 1 and data 2 and the transmission bandwidth of the sub-link, plus the transmission delay of the sub-link.

[0117] For example, the transmission time of the sub-link "switch 2-computing device B" is the quotient of the communication volume of data 1 and the transmission bandwidth of the sub-link, plus the transmission delay of the sub-link.

[0118] The simulation system can then calculate the transmission time for each communication pair. It then selects the longest transmission time as the first duration for the communication step involving these multiple communication pairs.

[0119] As described above, in this embodiment, the AI ​​task is broken down into multiple sub-tasks, and then the communication topology of the sub-tasks is constructed to determine the communication connection relationships between the multiple sub-tasks. Then, the first duration of each communication step in the multiple communication steps of the multiple sub-tasks is calculated. That is, the simulation system calculates the duration of the communication steps at the sub-task granularity, providing a finer granularity for detecting the performance of the AI ​​task and further improving the accuracy of the performance evaluation of the AI ​​task. In addition, the simulation system can also determine at least one communication pair included in each communication step, and combine the total communication volume of these at least one communication pair, the transmission link of each communication pair, and the transmission delay to determine the first duration of each communication step. The calculation of the first duration is more refined, further improving the accuracy of the performance evaluation of the AI ​​task.

[0120] 503. Output the total communication duration of the AI ​​task. The total communication duration is the sum of the first duration of each communication step.

[0121] After obtaining the first duration of each communication step, the simulation system sums the first durations of each communication step to obtain the total communication duration of the AI ​​task and outputs the total communication duration.

[0122] This application uses a simulation system to simulate the process of an AI task running on a cluster, including simulating multiple communication steps between various subtasks within the AI ​​task. By calculating the first duration of each communication step, the total communication duration of the AI ​​task is obtained. This enables accurate prediction of communication duration and improves the accuracy of performance evaluation for the AI ​​task.

[0123] In this embodiment, the simulation system can also evaluate and output other performance metrics of AI tasks running on a cluster. Possible implementation methods are described below.

[0124] In some optional implementations, the simulation system can output the average conflict coefficient for each communication step. The average conflict coefficient is the quotient of the actual execution time and the ideal execution time for each step, thus intuitively reflecting the performance of the AI ​​task in terms of communication time. The process by which the simulation system determines and outputs the average conflict coefficient for each communication step is explained below.

[0125] In summary, the simulation system determines the transmission time of each communication pair based on its traffic volume, communication link, transmission bandwidth, and transmission delay. It then determines a second duration for each communication step, which is the transmission time of the longest communication pair within that step. This second duration can also be referred to as the collision-free duration of each step. Finally, it outputs the average collision coefficient for each communication step, which is the quotient of the first and second durations.

[0126] The transmission time of each communication pair is the ideal transmission time of each communication pair, which can be understood as the time during which each communication pair exclusively enjoys its communication link to transmit data.

[0127] For example, in the embodiment shown in Figure 8, the transmission time of the communication pair <Subtask 1-Subtask 2> is the time that the communication link "Computing Device A-Switch 1-Switch 2-Computing Device B" is fully used for transmitting data 1. Specifically, it is the longest time among the transmission times of data 1 in sublinks "Computing Device A-Switch 1", "Switch 1-Switch 2", and "Switch 2-Computing Device B". The transmission time of data 1 in each sublink is the quotient of the data volume of data 1 and the transmission bandwidth of the sublink, plus the transmission delay of the sublink. Based on a similar method, the simulation system also calculates the transmission time of the communication pair <Subtask 1-Subtask 3>. Then, from the transmission times of these two communication pairs, the longest time is selected as the second duration. Finally, based on the first and second durations, the average collision coefficient is obtained.

[0128] Optionally, the average conflict coefficient can satisfy the following formula:

[0129] Among them, data step For the amount of communication transmitted by a single communication pair in a single communication step, This represents the maximum number of communication pairs in a single communication link within a communication step.

[0130] In this embodiment, the simulation system can also output the average conflict coefficient for each communication step. Since the average conflict coefficient reflects the ratio of the actual time taken for a communication step to the ideal time taken, it can intuitively reflect the performance of the AI ​​task in terms of communication time. The larger the average conflict coefficient, the longer the communication step is considered to take, which can be used as a potential optimization point for the AI ​​task.

[0131] In some alternative implementations, the simulation system may also output the first duration of each communication step.

[0132] In this embodiment, the simulation system outputs not only the total communication duration of the AI ​​task but also the first duration of each communication step, enriching the dimensions for evaluating the performance of the AI ​​task. Furthermore, based on the first duration of each communication step, excessively long communication steps can be adjusted, thereby optimizing the performance of the AI ​​task running on the cluster.

[0133] In the foregoing embodiments, the durations described are calculated from the perspective of communication steps or AI tasks. In practical applications, the simulation system can also determine and output other durations.

[0134] In some optional implementations, the parallel strategy of the AI ​​model includes multiple communication methods corresponding to multiple communication steps. The simulation system can also determine and output a third duration for each of the multiple communication methods based on a first duration for each communication step, where the third duration is the sum of the first durations of communication steps using the same communication method.

[0135] Specifically, a communication step includes at least one communication pair, and the communication method of the communication step is the communication method between the at least one communication pair (i.e., the subtask of sending data and the subtask of receiving data) included in the communication step. Therefore, one communication step can correspond to one or more communication methods.

[0136] In layman's terms, the communication method between different subtasks is the parallelism of these different subtasks. For example, in the embodiment shown in Figure 6, the communication methods between subtask 1-1 and subtask 1-2, and between subtask 2-1 and subtask 2-2, are all data parallelism. The communication methods between subtask 1-1 and subtask 2-1, between subtask 1-1 and subtask 2-2, between subtask 1-2 and subtask 2-1, and between subtask 1-2 and subtask 2-2 are all pipeline parallelism.

[0137] For example, assuming that in the embodiment shown in Figure 8, the communication method for the communication pair <subtask 1-subtask 2> and the communication pair <subtask 1-subtask 3> included in the communication step is the same, both being pipelined parallel. Then, the first duration of this communication step is the duration of the pipelined parallel communication method.

[0138] For example, assuming that in the embodiment shown in Figure 8, the communication steps include communication steps for communication pairs <subtask 1-subtask 2> and <subtask 1-subtask 3> using different communication methods, namely pipelined parallelism and tensor parallelism respectively. Then, the first duration of this communication step is both the duration of the pipelined parallelism communication method and the duration of the tensor parallelism communication method.

[0139] In this embodiment, the simulation system can also output a third duration corresponding to each communication method, evaluating the performance of the AI ​​task from more perspectives and enriching the performance evaluation results. Furthermore, the third duration corresponding to each communication method can also serve as a reference indicator for optimizing the AI ​​task; for example, optimizing time-consuming communication methods can improve the performance of the AI ​​task running on the cluster.

[0140] In some alternative implementations, the simulation system may also output the communication link for each communication pair and / or the amount of communication transmitted for each communication pair.

[0141] For example, as shown in Figure 9, the simulation system can output the communication link of each communication pair and / or the amount of communication transmitted by each communication pair in the form of code. Optionally, the simulation system can also output the aforementioned information in other forms, such as images, tables, etc., which are not limited here.

[0142] This further enriches the dimensions of the simulation system for evaluating the performance of AI tasks, and can also intuitively reflect the communication process between multiple subtasks in an AI task.

[0143] In some optional implementations, the communication topology further indicates the set of ports corresponding to the communication links of each communication pair. The port set includes ports of the computing device sending data, ports of the computing device receiving data, and ports of at least one switch transmitting data. The computing device sending data is the computing device that runs the subtask of sending data. The computing device receiving data is the computing device that runs the subtask of receiving data.

[0144] For example, in the embodiment shown in FIG8, computing device A is a computing device for sending data, computing device B is a computing device for receiving data 1, and computing device C is a computing device for receiving data 2.

[0145] The simulation system determines the communication volume of each port in the port set corresponding to each communication link based on the communication volume of each communication pair. Then, based on the communication volume of each port in the port set corresponding to each communication link, it determines the total communication volume of each port of each computing device in multiple computing devices and the total communication volume of each port of each switch in at least one switch after running the AI ​​task. The process is further illustrated below with reference to the diagram.

[0146] Please refer to Figure 10, which is a schematic diagram of the communication topology provided in the embodiments of this application.

[0147] In the embodiment shown in Figure 10, it is assumed that the communication volume of data 1 is 1GB and the communication volume of data 2 is 2GB. In one communication step, if computing device A sends data 1 through port 1 and data 2 through port 2, the switch receives data 1 and data 2 through port 3, sends data 1 to computing device B through port 4, and sends data 2 to computing device C through port 5. Computing device B receives data 1 through port 6, and computing device C receives data 2 through port 7.

[0148] Therefore, the communication volume of port 1 is 1GB, the communication volume of port 2 is 2GB, the communication volume of port 3 is 3GB, the communication volume of port 4 is 1GB, the communication volume of port 5 is 2GB, the communication volume of port 6 is 1GB, and the communication volume of port 7 is 2GB.

[0149] The simulation system sums up the communication volume of the same port of the same device in each communication step to obtain the total communication volume of that port after the AI ​​task is completed.

[0150] In this application, the simulation system can also determine the communication volume of the ports of each device included in each communication link, thereby determining the total communication volume of the ports of each device after running the AI ​​task, thus reflecting the data transmission situation between multiple subtasks during the operation of the AI ​​task.

[0151] In some alternative implementations, the simulation system can also output the communication traffic of any one or more ports of each device in each communication step.

[0152] In some optional implementations, the simulation system can also output the total traffic of the first port of the target computing device and / or the total traffic of the second port of the target switch. This further enriches the dimensions of the simulation system's performance evaluation of AI tasks and enhances the performance evaluation results.

[0153] The target computing device is comprised of multiple computing devices. The first port is some or all of the ports of the target computing device. In other words, the simulation system can output the total communication volume of any one or more ports on the target computing device after the AI ​​task is completed.

[0154] The target switch is contained in at least one switch, and the second port is some or all of the ports of the target switch. In other words, the simulation system can output the total traffic of any one or more ports on the target switch after the AI ​​task is completed.

[0155] In some optional implementations, the simulation system can also sum the communication traffic of each port of each device in each communication step to obtain the total communication traffic of each device in each communication step, and output the total communication traffic of one or more devices in each communication step.

[0156] In some optional implementations, the simulation system can also sum the total communication traffic of the same devices in each communication step to obtain the total communication traffic of each device after the AI ​​task is completed. It can then output the total communication traffic of one or more devices after the AI ​​task is completed.

[0157] Please refer to Figure 11 below, which is a flowchart illustrating the simulation method provided in the embodiments of this application.

[0158] As shown in Figure 11, the simulation system acquires input information, including the AI ​​model's parameters, parallel strategy, resource topology, ranktable, and ensemble communication algorithm. Based on the AI ​​model's parameters and parallel strategy, the simulation system calculates the communication volume for different parallel methods. It then calculates the communication domains for different parallel methods based on the AI ​​model's parallel strategy. Using the ensemble communication algorithm, it determines the communication pairs included in each communication step. Based on the ranktable and resource topology, it constructs a communication topology for multiple communication pairs, i.e., constructs the communication topology for multiple subtasks. According to the communication pairs and topology of each communication topology, it adds communication volume to the communication link corresponding to each communication step. The simulation system can also calculate the first and second durations of each communication step. The simulation system processes the input information in multiple ways to obtain the output results. The output results can reflect the performance of the AI ​​task in terms of duration, communication volume, and average conflict coefficient.

[0159] The duration includes the first duration of each communication step, the total communication duration of the AI ​​task, the third duration corresponding to each communication method, and other durations described in the preceding embodiments. The communication volume includes the communication volume of communication steps, port communication volume, device communication volume, and other communication volumes described in the preceding embodiments. The average collision coefficient includes the average collision coefficient of each communication step.

[0160] Furthermore, the flowchart of the simulation method provided in this application embodiment is shown in Figure 12.

[0161] In this embodiment of the application, the simulation system output results have various forms. For example, the output results can be shown in Table 1 below:

[0162] Table 1

[0163] As shown in the first row of Table 1, the communication pair in the first communication step of the AI ​​task uses the DP communication method, passing through a switch with IP address 26.71.102.210. Port 16 of this switch is the data receiving port, and the communication volume through this port is 1.1GB. The average collision coefficient of the first communication step is 2, and the transmission time is 0.051 seconds. The transmission time here refers to the first duration of the communication step in the aforementioned embodiment.

[0164] For example, the output results can also be shown in Table 2 below:

[0165] Table 2

[0166] The output results shown in Table 2 include switch-related information for a single iteration during the AI ​​model training process. As shown in the first row of Table 2, the switch with IP address 26.71.102.210 received 1.1GB of data on port 16 in a single iteration.

[0167] For example, the output results can also be shown in Table 3 below:

[0168] Table 3

[0169] The output results shown in Table 2 include relevant information for different communication methods in a single iteration during the training process of the AI ​​model. As shown in the first row of Table 2, the TP communication method processed 336GB of data in a single iteration, with an estimated processing time of 1.53 seconds.

[0170] For example, the simulation output can also be in the form of a flowchart as shown in Figure 13. Please refer to Figure 13, which is a schematic diagram of the output provided in an embodiment of this application.

[0171] In the embodiment shown in Figure 13, a "spine-leaf" switch architecture is used as an example. Spine5, Spine6, and Spine12 represent the switches in the spine layer. The communication topology also includes switches with IP addresses 26.71.102.224 and 26.71.102.227, which, as leaf layer switches, can also be called Tor switches. The bottom-most rectangle represents the computing devices running subtasks.

[0172] The simulation system simulates the performance of AI tasks running on a cluster, and can calculate the actual amount of communication transmitted in each sub-link, which can be compared with the theoretical amount of communication.

[0173] It should also be noted that the embodiments of this application do not limit the number of AI tasks simulated by the simulation system; there can be one or more, and no specific limitation is made here.

[0174] Next, the relevant devices provided in the embodiments of this application will be described.

[0175] Please refer to Figure 14, which is a schematic diagram of the structure of the simulation system provided in the embodiment of this application. As shown in Figure 14, the simulation system 1400 includes a simulation module 1401, a calculation module 1402, and a visualization module 1403.

[0176] Simulation module 1401 is used to simulate the process of AI tasks running on a cluster based on input information.

[0177] The computing module 1402 is used to calculate the performance of the simulated AI task running on the cluster, including the duration, communication volume, and average collision coefficient described in the aforementioned embodiments. These performance metrics can be used as the output results of the simulation.

[0178] The visualization module 1403 is used to display the simulation output results in a visual manner.

[0179] The simulation system 1400 is used to implement the simulation method provided in the embodiments of this application, as detailed above, and will not be repeated here.

[0180] Please refer to Figure 15, which is a schematic diagram of the simulation system provided in this embodiment. In this embodiment, the simulation system 1500 is used to simulate the performance of AI tasks running on a cluster. The AI ​​task includes training or inference of an AI model. The cluster includes multiple computing devices connected through at least one switch. The AI ​​task includes multiple subtasks, and communication between the subtasks includes multiple communication steps.

[0181] As shown in Figure 15, the simulation system 1500 includes an acquisition unit 1501 and a processing unit 1502.

[0182] In some optional implementations, the acquisition unit 1501 is used to receive input information, including the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the collective communication algorithm used by the AI ​​model. The ranktable includes the mapping relationship between the identifiers of multiple subtasks and the address information of multiple computing devices. The resource topology indicates the connection relationship between multiple computing devices and at least one switch, and between at least one switch and each other.

[0183] Processing unit 1502 is used to calculate the first duration of each communication step in multiple communication steps based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the ranktable, the resource topology, and the ensemble communication algorithm. The first duration is the execution time of the corresponding communication step. It outputs the total communication duration of the AI ​​task, which is the sum of the first durations of each communication step.

[0184] In some optional implementations, processing unit 1502 is specifically configured to: decompose the AI ​​task into multiple subtasks based on the parameters and parallel strategy of the AI ​​model; determine the communication topology of the multiple subtasks based on the ranktable and resource topology, wherein the communication topology indicates the communication connection relationship between the multiple subtasks when they run on multiple computing devices; and calculate the first duration of each communication step based on the parameters of the AI ​​model, the parallel strategy of the AI ​​model, the aggregated communication algorithm, and the communication topology.

[0185] In some alternative implementations, the resource topology also indicates the transmission bandwidth and latency between any two connected devices among multiple computing devices and at least one switch. The communication topology also indicates the transmission bandwidth and latency between any two linked subtasks among multiple subtasks running on multiple computing devices.

[0186] Processing unit 1502 is specifically configured to: determine, according to the ensemble communication algorithm, at least one communication pair included in each communication step, wherein each communication pair includes a subtask of sending data and a subtask of receiving data; determine, according to the communication topology, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link; determine, according to the parameters of the AI ​​model, the parallel strategy of the AI ​​model, and the ensemble communication algorithm, the total communication volume of at least one communication pair, wherein the total communication volume of at least one communication pair is the sum of the communication volumes of each communication pair; and determine the first duration of each communication step based on the total communication volume of at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link.

[0187] In some optional implementations, the processing unit 1502 is further configured to determine the transmission time of each communication pair based on the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair. A second duration for each communication step is determined, where the second duration is the transmission time of the communication pair with the longest duration in the corresponding communication step. The average collision coefficient for each communication step is output, where the average collision coefficient is the quotient of the first duration and the second duration.

[0188] In some alternative implementations, the processing unit 1502 is also configured to output a first duration for each communication step.

[0189] In some alternative implementations, the parallel strategy of the AI ​​model includes multiple communication methods corresponding to multiple communication steps.

[0190] The processing unit 1502 is further configured to: determine a third duration corresponding to each of the multiple communication methods based on a first duration of each communication step, wherein the third duration is the sum of the first durations of the communication steps using the same communication method; and output the third duration corresponding to each communication method.

[0191] In some alternative implementations, the processing unit 1502 is also configured to output the communication link of each communication pair and / or the amount of communication transmitted by each communication pair.

[0192] In some alternative implementations, the communication topology also indicates the set of ports corresponding to the communication links of each communication pair, the set of ports including the ports of the computing device that sends data, the ports of the computing device that receives data, and the ports of at least one switch that transmits data.

[0193] The processing unit 1502 is further configured to: determine the communication volume of each port in the port set corresponding to each communication link based on the communication volume of each communication pair; and determine the total communication volume of each port of each computing device in the multiple computing devices and the total communication volume of each port of each switch in at least one switch after running the AI ​​task, based on the communication volume of each port in the port set corresponding to each communication link.

[0194] In some alternative implementations, the processing unit 1502 is further configured to: output the total traffic of a first port of a target computing device, and / or the total traffic of a second port of a target switch, wherein the target computing device comprises a plurality of computing devices and the target switch comprises at least one switch.

[0195] Both the acquisition unit 1501 and the processing unit 1502 can be implemented in software or in hardware. For example, the implementation of the processing unit 1502 will be described below. Similarly, the implementation of the acquisition unit 1501 can be referenced to the implementation of the processing unit 1502.

[0196] As an example of a software functional unit, processing unit 1502 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device, processor, server), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, processing unit 1502 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0197] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0198] As an example of a hardware functional unit, the processing unit 1502 may include at least one computing device, such as a server. Alternatively, the processing unit 1502 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0199] The processing unit 1502 includes multiple computing devices that can be distributed in the same region or in different regions. Similarly, the processing unit 1502 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the processing unit 1502 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0200] It should be noted that the simulation system 1500 achieves all its functions by implementing different steps in the simulation method through the acquisition unit 1501 and the processing unit 1502. The simulation system 1500 is used to execute the operations performed by the cloud platform in the embodiment shown in Figure 2 above, or the operations performed by the simulation system in the embodiments shown in Figures 5 to 13, to implement the simulation method provided in this application embodiment, as detailed above, and will not be repeated here.

[0201] Please refer to Figure 16, which is a schematic diagram of a computing device provided in an embodiment of this application. The computing device 1600 includes a processor 1601, a communication interface 1602, a bus 1603, and a memory 1604. The processor 1601, communication interface 1602, and memory 1604 communicate via the bus 1603. In practical applications, communication can also be achieved through other means such as wireless transmission; specific methods are not limited here.

[0202] The computing device 1600 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memory in the computing device 1600.

[0203] Processor 1601 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0204] The communication interface 1602 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 1600 and other devices or communication networks.

[0205] Bus 1603 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 16, but this does not imply that there is only one bus or one type of bus. Bus 1603 can include pathways for transmitting information between various components of computing device 1600 (e.g., memory 1604, processor 1601, communication interface 1602).

[0206] Memory 1604 may include volatile memory, such as random access memory (RAM). Memory 1604 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0207] The memory 1604 stores executable program code, and the processor 1601 executes the executable program code to implement the functions of the aforementioned acquisition unit 1501 and processing unit 1502, thereby realizing the simulation method. That is, the memory 1604 stores instructions for executing the simulation method.

[0208] This application also provides a computing device cluster, which includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some optional embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0209] Please refer to Figures 17 and 18, which are schematic diagrams of the structure of the computing device cluster provided in the embodiments of this application.

[0210] As shown in Figure 17, the computing device cluster includes at least one computing device 1600. The memory 1604 of one or more computing devices 1600 in the computing device cluster may store the same instructions for executing the simulation method provided in the embodiments of this application.

[0211] In some possible implementations, the memory 1604 of one or more computing devices 1600 in the computing device cluster may also store partial instructions for executing the simulation method. In other words, a combination of one or more computing devices 1604 can jointly execute the instructions for executing the simulation method.

[0212] It should be noted that the memory 1604 in different computing devices 1600 within the computing device cluster can store different instructions, which are used to execute parts of the cloud platform or simulation system. That is, the instructions stored in the memory 1604 of different computing devices 1600 can realize the functions of one or more units among the acquisition unit 1501 and processing unit 1502, so as to realize the simulation method provided in the embodiments of this application.

[0213] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 18 illustrates one possible implementation. As shown in Figure 18, two computing devices 1600A and 1600B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1604 in computing device 1600A stores instructions for executing the functions of the acquisition unit 1501. Simultaneously, the memory 1604 in computing device 1600B stores instructions for executing the functions of the processing unit 1502.

[0214] The connection method between the computing device clusters shown in Figure 18 can be based on the simulation method provided in this application, which separates the processing operation from the operation outside the processing operation. That is, it is considered that the function of the acquisition unit 1501 is executed by the computing device 1600A, and the function of the processing unit 1502 is executed by the computing device 1600B.

[0215] It should be understood that the functions of computing device 1600A shown in Figure 18 can also be performed by multiple computing devices 1600. Similarly, the functions of computing device 1600B can also be performed by multiple computing devices 1600.

[0216] This application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster described in Figures 17 and 18, and will not be repeated here.

[0217] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computer device, it causes the at least one computer device to execute the simulation method provided in this application.

[0218] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the simulation method provided in this application.

[0219] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A simulation method, characterized by, The method is performed by a simulation system for simulating performance of an AI task running on a cluster, the AI task including training or inference of an AI model, the cluster including a plurality of computing devices connected by at least one switch; the AI task including a plurality of subtasks, communication among the plurality of subtasks including a plurality of communication steps; The method includes: receiving input information including parameters of the AI model, a parallel strategy of the AI model, a rank table, a resource topology, and a collective communication algorithm used by the AI model, the rank table including a mapping relationship between identifications of the plurality of subtasks and address information of the plurality of computing devices, the resource topology indicating connection relationships among the plurality of computing devices and the at least one switch, among the at least one switch and each other; calculating a first duration of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the rank table, the resource topology, and the collective communication algorithm, the first duration being an execution time of the corresponding communication step; outputting a total communication duration of the AI task, the total communication duration being a sum of the first durations of the plurality of communication steps.

2. The method of claim 1, wherein, The calculation of the first duration of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the rank table, the resource topology, and the collective communication algorithm includes: decomposing the AI task into the plurality of subtasks according to the parameters of the AI model and the parallel strategy of the AI model; determining a communication topology of the plurality of subtasks according to the rank table and the resource topology, the communication topology indicating a communication connection relationship among the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; calculating the first duration of each communication step according to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology.

3. The method of claim 2, wherein, The resource topology further indicates a transmission bandwidth and a transmission latency between any two devices having a connection relationship in the plurality of computing devices and the at least one switch; The communication topology further indicates a transmission bandwidth and a transmission latency between any two subtasks having a connection relationship in the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; The calculation of the first duration of each communication step according to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology includes: determining at least one communication pair included in each communication step according to the collective communication algorithm, wherein each communication pair includes a subtask sending data and a subtask receiving data; determining a communication link of each communication pair, a transmission bandwidth of each communication link, and a transmission latency of each communication link according to the communication topology; determining, according to the parameters of the AI model, the parallel strategy of the AI model, and the collective communication algorithm, a total traffic of the at least one communication pair, the total traffic of the at least one communication pair being a sum of the traffic of each communication pair; determining, according to the total traffic of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission latency of each communication link, a first duration of each communication step.

4. The method of claim 3, wherein, The method further comprises: determining, according to the traffic of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission latency of the communication link of each communication pair, a transmission time of each communication pair; determining a second duration of each communication step, the second duration being the transmission time of the communication pair with the longest duration in the corresponding communication step; outputting an average conflict coefficient of each communication step, the average conflict coefficient being a quotient of the first duration and the second duration.

5. The method according to claim 3 or 4, characterized in that, The method further comprises: outputting the first duration of each communication step.

6. The method according to any one of claims 2 to 5, characterized in that, The parallel strategy of the AI model comprises a plurality of communication methods corresponding to the plurality of communication steps; the method further comprises: determining, based on the first duration of each communication step, a third duration corresponding to each communication method in the plurality of communication methods, the third duration being a sum of the first duration of the communication steps using the same communication method; outputting the third duration corresponding to each communication method.

7. The method according to any one of claims 4 to 6, characterized in that, The method further comprises: outputting the communication link of each communication pair and / or the traffic transmitted by each communication pair.

8. The method according to any one of claims 4 to 7, characterized in that, The communication topology further indicates a port set corresponding to the communication link of each communication pair, the port set comprising a port of a computing device sending data, a port of a computing device receiving data, and a port of at least one switch transmitting data; The method further comprises: determining, according to the traffic of each communication pair, the traffic of each port in the port set corresponding to each communication link; determining, according to the traffic of each port in the port set corresponding to each communication link, a total traffic of each port of each computing device in the plurality of computing devices and a total traffic of each port of each switch in the at least one switch after running the AI task.

9. The method of claim 8, wherein, The method further comprises: outputting the total traffic of the first port of the target computing device and / or the total traffic of the second port of the target switch, the target computing device being included in the plurality of computing devices, and the target switch being included in the at least one switch.

10. An emulation system, comprising: The simulation system is used to simulate the performance of an AI task running on a cluster, the AI task comprising training or inference of an AI model, the cluster comprising a plurality of computing devices connected through at least one switch; the AI task comprising a plurality of subtasks, communication between the plurality of subtasks comprising a plurality of communication steps; the simulation system comprising: The acquisition unit is configured to receive input information, the input information including parameters of the AI model, a parallel strategy of the AI model, a ranktable, a resource topology, and a collective communication algorithm used by the AI model, the ranktable including a mapping relationship between identifications of the plurality of subtasks and address information of the plurality of computing devices, and the resource topology indicating a connection relationship between the plurality of computing devices and the at least one switch, and between the at least one switch and each other; The processing unit is configured to calculate a first duration of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the ranktable, the resource topology, and the collective communication algorithm, the first duration being an execution time of the corresponding communication step. The processing unit is further configured to output a total communication duration of the AI task, the total communication duration being a sum of the first durations of the plurality of communication steps.

11. The simulation system of claim 10, wherein, The processing unit is specifically configured to: According to the parameters of the AI model and the parallel strategy of the AI model, the AI task is decomposed into the plurality of subtasks; According to the ranktable and the resource topology, a communication topology of the plurality of subtasks is determined, the communication topology indicating a communication connection relationship between the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; According to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology, the first duration of each communication step is calculated.

12. The simulation system of claim 11, wherein, The resource topology further indicates a transmission bandwidth and a transmission delay between any two devices having a connection relationship in the plurality of computing devices and the at least one switch; The communication topology further indicates a transmission bandwidth and a transmission delay between any two subtasks having a connection relationship in the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; The processing unit is specifically configured to: According to the collective communication algorithm, at least one communication pair included in each communication step is determined, wherein each communication pair includes a subtask sending data and a subtask receiving data; According to the communication topology, a communication link of each communication pair, a transmission bandwidth of each communication link, and a transmission delay of each communication link are determined; According to the parameters of the AI model, the parallel strategy of the AI model, and the collective communication algorithm, a total communication amount of the at least one communication pair is determined, the total communication amount of the at least one communication pair being a sum of communication amounts of each communication pair; According to the total communication amount of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link, the first duration of each communication step is determined.

13. The simulation system of claim 12, wherein, The processing unit is further configured to: According to the communication amount of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair, a transmission time of each communication pair is determined respectively. determine a second duration of each of the communication steps, the second duration being a transmission time of a communication pair with a longest duration in a corresponding communication step; output an average conflict coefficient of each of the communication steps, the average conflict coefficient being a quotient of the first duration and the second duration.

14. The simulation system of claim 12 or 13, wherein, The processing unit is further configured to output the first duration of each of the communication steps.

15. The simulation system of any one of claims 11 to 14, wherein, The parallel strategy of the AI model comprises a plurality of communication methods corresponding to the plurality of communication steps; the processing unit is further configured to: determine, based on the first duration of each of the communication steps, a third duration corresponding to each of the plurality of communication methods, the third duration being a sum of the first duration of the communication steps using a same communication method; and output the third duration corresponding to each of the plurality of communication methods.

16. The simulation system of any one of claims 13 to 15, wherein, The processing unit is further configured to output a communication link of each of the communication pairs and / or a communication volume transmitted by each of the communication pairs.

17. The simulation system of any one of claims 13 to 16, wherein, The communication topology further indicates a port set corresponding to a communication link of each of the communication pairs, the port set comprising a port of a computing device sending data, a port of a computing device receiving data, and a port of at least one switch transmitting data; the processing unit is further configured to: determine, according to the communication volume of each of the communication pairs, a communication volume of each port in the port set corresponding to each of the communication links; and determine, according to the communication volume of each port in the port set corresponding to each of the communication links, a total communication volume of each port of each of the plurality of computing devices and a total communication volume of each port of each of the at least one switch after running the AI task.

18. The simulation system of claim 17, wherein, The processing unit is further configured to: output the total communication volume of the first port of the target computing device and / or the total communication volume of the second port of the target switch, the target computing device being included in the plurality of computing devices, and the target switch being included in the at least one switch.

19. A cluster of computing devices, characterized in that, The at least one computing device comprises a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method of any one of claims 1 to 9.

20. A computer program product comprising instructions, characterized in that, The instructions, when executed by the computing device cluster, cause the computing device cluster to perform the method of any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises computer program instructions, which, when executed by the computing device cluster, cause the computing device cluster to perform the method of any one of claims 1 to 9.