Simulation method and related device

Abstract

The present application provides a simulation method and a related device, for use in improving the accuracy of performance evaluation for AI tasks. The method is applied to a simulation system, the simulation system is used for simulating the performance of an AI task when running on a cluster, a plurality of computing devices comprised in the cluster are connected by means of at least one switch, and the communication between a plurality of subtasks comprised in the AI task comprises a plurality of communication steps. The method comprises: receiving input information, wherein the input information comprises parameters of an AI model, a parallel strategy of the AI model, a ranktable, a resource topology, and a collective communication algorithm used by the AI model, the ranktable comprises mapping relationships between identifiers of a plurality of subtasks and address information of a plurality of computing devices, and the resource topology indicates connection relationships between the plurality of computing devices and at least one switch, and between the at least one switch; on the basis of the input information, calculating a first duration indicating the execution time of each communication step; and outputting a total communication duration of an AI task, the total communication duration being the sum of the first durations of the communication steps.

Description

An emulation method and related device

[0001] The present application claims priority to the Chinese Patent Application No. 202411135093.9, filed on August 16, 2024, and entitled "An Emulation Method and Related Device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence (AI), and in particular to an emulation method and related device. BACKGROUND

[0003] With the development of artificial intelligence technology, the size of AI models gradually increases. The number of parameters of AI models and the number of devices required to implement inference or training of AI models also gradually increase. Training or inference of AI models on a large-scale cluster is usually performed in a distributed parallel manner. However, before the AI model is actually run on the cluster, the actual performance thereof cannot be determined.

[0004] In a related technical solution, parameters of an AI model and hardware parameters required to run the AI model are input to an emulation device. The emulation device evaluates the running performance of the AI model on the cluster by simulating the training or inference process of the AI model.

[0005] However, in this technical solution, the evaluation of the running performance of the AI model on the cluster is a relatively general result, which reflects the overall performance of the AI model running on the cluster, and the accuracy of the evaluation is not high. SUMMARY

[0006] The present application provides an emulation method and related device for simulating the process of running an AI task on a cluster by an emulation system, including simulating a plurality of communication steps between a plurality of subtasks included in the AI task. A first duration of each communication step is calculated to obtain a total communication duration of the AI task. The accuracy of the communication duration is achieved, and the accuracy of the performance evaluation of the AI task is improved.

[0007] In a first aspect, the present application provides an emulation method, which is performed by an emulation system for simulating the performance of an AI task running on a cluster. The AI task includes training or inference of an AI model, the AI task includes a plurality of subtasks, and the communication between the plurality of subtasks includes a plurality of communication steps. The cluster includes a plurality of computing devices connected through at least one switch.

[0008] The simulation system receives input information and performs simulation based on the input information. The input information includes parameters of the AI model, a parallel strategy of the AI model, a rank table, a resource topology, and a collective communication algorithm used by the AI model. The rank table includes a mapping relationship between identifiers of a plurality of sub-tasks and address information of a plurality of computing devices. The resource topology indicates a connection relationship between the plurality of computing devices and at least one switch, and between the at least one switch and each other. The collective communication algorithm defines how many communication steps are included in the plurality of sub-tasks, and which communication pairs are included in each communication step, that is, the collective communication algorithm defines how the plurality of sub-tasks complete the plurality of communication steps. The simulation system calculates a first duration of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the rank table, the resource topology, and the collective communication algorithm. The first duration is an execution time of the corresponding communication step, that is, the first duration is an actual execution time of each communication step in the process of simulating the running of the AI task on the cluster by the simulation system. The simulation system outputs a total communication duration of the AI task, which is a sum of the first durations of each communication step.

[0009] In this application, the process of running the AI task on the cluster is simulated by the simulation system, including simulating a plurality of communication steps between a plurality of sub-tasks included in the AI task. By calculating the first duration of each communication step, the total communication duration of the AI task is obtained. The accurate prediction of the communication duration is realized, and the accuracy of the performance evaluation of the AI task is improved.

[0010] In some optional implementation forms of the first aspect, the simulation system calculates the first duration of each communication step in the plurality of communication steps, specifically including: according to the parameters of the AI model and the parallel strategy of the AI model, the AI task is decomposed into a plurality of sub-tasks. It can be understood that the parameters of the AI model indicate the amount of data to be processed by the AI task, and the parallel strategy of the AI model indicates the splitting manner of the amount of data processed by the AI task, so the foregoing process can also be understood as splitting the amount of data to be processed by the AI task into a plurality of data according to the parallel strategy of the AI model, and each piece of data corresponds to a sub-task. According to the rank table and the resource topology, a communication topology of the plurality of sub-tasks is determined, which indicates a communication connection relationship between the plurality of sub-tasks when the plurality of sub-tasks run on the plurality of computing devices. According to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology, the first duration of each communication step is calculated.

[0011] In the present application, the AI task is disassembled into multiple sub-tasks, and a communication topology of the sub-tasks is constructed to determine the communication connection relationship between the multiple sub-tasks. Then, a first duration of each communication step in multiple communication steps of the multiple sub-tasks is calculated. That is, the simulation system calculates the duration of the communication step from the granularity of the sub-tasks, and the detection granularity of the running performance of the AI task is more fine, which further improves the accuracy of the performance evaluation of the AI task.

[0012] In some optional implementations of the first aspect, the resource topology further indicates a transmission bandwidth and a transmission delay between any two devices having a connection relationship in the multiple computing devices and the at least one switch. Correspondingly, the communication topology further indicates a transmission bandwidth and a transmission delay between any two sub-tasks having a connection relationship in the multiple sub-tasks when the multiple sub-tasks run on the multiple computing devices. The simulation system calculates the first duration of each communication step, specifically, determines at least one communication pair included in each communication step according to the collective communication algorithm, wherein each communication pair includes a sub-task sending data and a sub-task receiving data. Then, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link are determined according to the communication topology. Then, the total communication amount of the at least one communication pair is determined according to the parameters of the AI model, the parallel strategy of the AI model, and the collective communication algorithm, and the total communication amount of the at least one communication pair is the sum of the communication amounts of each communication pair. Then, the first duration of each communication step is determined according to the total communication amount of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link. The first duration can be understood as the duration of the communication link of the at least one communication pair included in a communication step to transmit the total communication amount of the at least one communication pair included in the communication step.

[0013] In the present application, the simulation system can also determine at least one communication pair included in each communication step, and determine the first duration of each communication step in combination with the total communication amount of the at least one communication pair, the transmission link of each communication pair, and the transmission delay. The calculation of the first duration is more fine, which further improves the accuracy of the performance evaluation of the AI task.

[0014] In some optional implementation of the first aspect, the simulation system can further determine a transmission time of each communication pair according to the traffic of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair, respectively. The transmission time of each communication pair refers to the time for transmitting the traffic of each communication pair when the link of each communication pair is exclusively used by the communication process of each communication pair, which can be regarded as the ideal transmission time of each communication pair. The second duration of each communication step is determined from the transmission time of at least one communication pair included in each communication step, and the second duration is the transmission time of the communication pair with the longest duration in the corresponding communication step. The average conflict coefficient of each communication step is output, and the average conflict coefficient is the quotient of the first duration and the second duration. Therefore, the average conflict coefficient can reflect the multiple of the actual time consumption of the communication step compared with the ideal time consumption, and the larger the average conflict coefficient is, the longer the time consumption of the communication step is, which can intuitively reflect the performance of the current AI task.

[0015] In the present application, the simulation system can output the average conflict coefficient of each communication step, since the average conflict coefficient reflects the multiple of the actual time consumption of the communication step compared with the ideal time consumption, which can intuitively reflect the performance of the AI task in the communication time. The larger the average conflict coefficient is, the longer the time consumption of the communication step is, which can be used as a potential optimization point of the AI task.

[0016] In some optional implementation of the first aspect, the simulation system can further output the first duration of each communication step.

[0017] In the present application, in addition to outputting the total communication duration of the AI task, the simulation system can also output the first duration of each communication step, which enriches the dimensions of performance evaluation of the AI task. In addition, according to the first duration of each communication step, the communication step with too long duration can be adjusted, so as to optimize the performance of the AI task running on the cluster.

[0018] In some optional implementation of the first aspect, the parallel strategy of the AI model includes a plurality of communication methods corresponding to a plurality of communication steps. The simulation system can further statistically determine the first duration of each communication step from the dimension of the communication method, determine a third duration corresponding to each communication method in the plurality of communication methods, and output the third duration corresponding to each communication method. The third duration is the sum of the first durations of the communication steps using the same communication method.

[0019] In the present application, the simulation system can also output a third time length corresponding to each communication method, to evaluate the performance of the AI task from more angles and enrich the performance evaluation results. In addition, the third time length corresponding to each communication method can also be used as a reference index for optimizing the AI task, for example, optimizing the communication method with long time consumption, so as to optimize the performance of the AI task running on the cluster.

[0020] In some optional implementations of the first aspect, the simulation system can also output the communication link of each communication pair and / or the communication volume transmitted by each communication pair. This further enriches the dimensions of the performance evaluation of the AI task by the simulation system and can also intuitively reflect the communication process between the multiple sub-tasks in the AI task.

[0021] In some optional implementations of the first aspect, the communication topology also indicates a port set corresponding to the communication link of each communication pair, and the port set includes a port of the computing device sending data, a port of the computing device receiving data, and a port of the at least one switch transmitting data. The simulation system determines the communication volume of each port in the port set corresponding to each communication link according to the communication volume of each communication pair. According to the communication volume of each port in the port set corresponding to each communication link, the simulation system determines the total communication volume of each port of each computing device in the multiple computing devices and the total communication volume of each port of each switch in the at least one switch after running the AI task.

[0022] In the present application, the simulation system can also determine the communication volume of the ports of each device included in each communication link, so as to determine the total communication volume of the ports of each device after running the AI task, thereby reflecting the data transmission between the multiple sub-tasks when the AI task is running.

[0023] In some optional implementations of the first aspect, the simulation system can also output the total communication volume of the first port of the target computing device and / or the total communication volume of the second port of the target switch, and the target computing device is included in the multiple computing devices and the target switch is included in the at least one switch. This further enriches the dimensions of the performance evaluation of the AI task by the simulation system and enriches the performance evaluation results.

[0024] In the second aspect, the present application provides a simulation system for simulating the performance of an AI task running on a cluster, the AI task including training or inference of an AI model, the cluster including multiple computing devices connected through at least one switch; the AI task including multiple sub-tasks, and the communication between the multiple sub-tasks including multiple communication steps. The simulation system includes:

[0025] The acquisition unit is configured to receive input information, the input information including parameters of the AI model, a parallel strategy of the AI model, a ranktable, a resource topology, and a collective communication algorithm used by the AI model, the ranktable including a mapping relationship between identifiers of a plurality of subtasks and address information of a plurality of computing devices, and the resource topology indicating a connection relationship between the plurality of computing devices and at least one switch and between the at least one switch and each other;

[0026] The processing unit is configured to calculate, according to the parameters of the AI model, the parallel strategy of the AI model, the ranktable, the resource topology, and the collective communication algorithm, a first time length of each communication step in a plurality of communication steps, the first time length being an execution time of the corresponding communication step.

[0027] The processing unit is further configured to output a total communication time length of the AI task, the total communication time length being a sum of the first time lengths of each communication step.

[0028] The simulation system is configured to implement the method according to the first aspect or any possible implementation manner of the first aspect, and details are shown in the foregoing description and will not be repeated here.

[0029] In a third aspect, the present application provides a computing device, including a processor and a memory. The processor of the computing device is configured to execute instructions stored in the memory, so that the computing device implements the method according to the first aspect or any possible implementation manner of the first aspect.

[0030] In a fourth aspect, the present application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster implements the method according to the first aspect or any possible implementation manner of the first aspect.

[0031] In a fifth aspect, the present application provides a computer program product including instructions, which, when executed by a computing device cluster, cause the computing device cluster to implement the method according to the first aspect or any possible implementation manner of the first aspect.

[0032] In a sixth aspect, the present application provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster implements the method according to the first aspect or any possible implementation manner of the first aspect.

[0033] The beneficial effects of any one of the second aspect to the sixth aspect are similar to those of the first aspect or any possible implementation manner of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0034] Fig. 1 is a schematic diagram of an artificial intelligence subject architecture according to an embodiment of the present application;

[0035] Fig. 2 is a schematic diagram of a system architecture according to an embodiment of the present application;

[0036] Fig. 3 is a schematic diagram of another system architecture according to an embodiment of the present application;

[0037] Fig. 4 is a schematic diagram of another system architecture according to an embodiment of the present application;

[0038] Fig. 5 is a schematic diagram of a flow of a simulation method according to an embodiment of the present application;

[0039] Fig. 6 is a schematic diagram according to an embodiment of the present application;

[0040] Fig. 7 is a schematic diagram of another flow of a simulation method according to an embodiment of the present application;

[0041] Fig. 8 is a schematic diagram of a communication topology according to an embodiment of the present application;

[0042] Fig. 9 is a schematic diagram of an output result of a simulation according to an embodiment of the present application;

[0043] Fig. 10 is a schematic diagram of another communication topology according to an embodiment of the present application;

[0044] Fig. 11 is a schematic diagram of another flow of a simulation method according to an embodiment of the present application;

[0045] Fig. 12 is a schematic diagram of another flow of a simulation method according to an embodiment of the present application;

[0046] Fig. 13 is a schematic diagram of another output result of a simulation according to an embodiment of the present application;

[0047] Fig. 14 is a schematic diagram of a structure of a simulation system according to an embodiment of the present application;

[0048] Fig. 15 is a schematic diagram of another structure of a simulation system according to an embodiment of the present application;

[0049] Fig. 16 is a schematic diagram of a structure of a computing device according to an embodiment of the present application;

[0050] Fig. 17 is a schematic diagram of a structure of a computing device cluster according to an embodiment of the present application;

[0051] Fig. 18 is a schematic diagram of another structure of a computing device cluster according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiment of the application provides a simulation method and related equipment, which are used for simulating the process of an AI task running on a cluster through a simulation system, and simulating a plurality of communication steps between a plurality of subtasks included in the AI task. The total communication time length of the AI task is obtained by calculating the time length of each communication step. The communication time length is accurately predicted, and the accuracy of performance evaluation of the AI task is improved.

[0053] The embodiments of the application are described below with reference to the drawings. Those skilled in the art can know that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the application are also applicable to similar technical problems.

[0054] The terms "first", "second", and the like in the specification and claims of the application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, which is only a distinguishing way adopted in the description of the embodiments of the application for the objects with the same attribute in the description. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the process, method, system, product or equipment including a series of units does not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or equipment. In addition, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0055] Firstly, the related concepts and proper nouns involved in the embodiments of the application are described.

[0056] 1) Distributed parallelism:

[0057] Distributed parallelism is a technique that distributes the training process or inference process of an AI model across multiple computing devices, or can be understood as splitting the training process or inference process of an AI model into multiple sub-processes and distributing the multiple sub-processes to multiple computing devices for execution. Further, distributed parallelism includes distributed computing and parallel computing. This technique allows an AI model to utilize the computing power of multiple computing devices to accelerate large-scale data processing and complex model running. Distributed parallelism includes parallelism modes such as tensor parallelism (TP), pipeline parallelism (PP), and data parallelism (DP).

[0058] 2) Rank table:

[0059] A rank table is a file used to manage communication between computing nodes participating in the training or inference of an AI model, and is used to record information of all nodes participating in the training or inference of an AI model. It includes the logical identifier (rank ID) of each node, address information of the node, etc. The rank table is used to ensure that all nodes participating in the training or inference of an AI model can effectively communicate and synchronize, thereby achieving efficient parallelism.

[0060] 3) Communication domain:

[0061] A communication domain indicates a set of computing devices that can communicate with each other, and a communication domain is responsible for a specific communication task. The multiple computing devices in a communication domain synchronize AI model parameters and gradient information through communication and data exchange. In the training or inference process of an AI model, the multiple computing nodes used by the AI model can include multiple communication domains, and a computing device can belong to one or more communication domains.

[0062] 4) Collective communication algorithms:

[0063] Collective communication algorithms are communication algorithms used for data exchange between multiple processes or multiple threads in distributed computing and parallel computing. In these algorithms, a group of processes jointly participate in communication operations to achieve data collection, distribution, or aggregation. In the training or inference of an AI model, collective communication algorithms define sub-processes for communication, as well as communication steps and other communication information related to the training or inference process of the AI model.

[0064] Next, please refer to FIG. 1, which is a schematic diagram of an artificial intelligence main body framework provided by an embodiment of the present application. The main body framework describes the overall workflow of an artificial intelligence system and is suitable for general artificial intelligence field requirements.

[0065] The above-mentioned artificial intelligence theme framework is described below from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0066] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensation process of "data-information-knowledge-wisdom".

[0067] The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

[0068] (1) Infrastructure.

[0069] The infrastructure provides computing power support for the artificial intelligence system, realizes communication with the external world, and realizes support through the underlying platform. Communication with the outside world through sensors; computing power is provided by intelligent chips, including central processing unit (CPU), embedded neural network processing unit (NPU), graphics processing unit (GPU), application specific integrated circuit (ASIC), or field programmable gate array (FPGA) and other hardware acceleration chips; the underlying platform includes distributed computing framework and related platform guarantees and support such as network, which can include cloud storage and computing, interconnection network, etc. For example, sensors and external communication acquire data, which are provided to intelligent chips in the distributed computing system provided by the underlying platform for calculation.

[0070] (2) Data.

[0071] The data on the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence. Data involves graphics, images, speech, text, and also involves Internet of Things data of traditional devices, including business data of existing systems and sensing data such as force, displacement, liquid level, temperature, and humidity.

[0072] (3) Data processing.

[0073] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision, etc.

[0074] Among them, machine learning and deep learning can model, extract, preprocess, train, etc. of symbolic and formalized intelligent information on data.

[0075] Reasoning refers to the process of simulating human intelligent reasoning methods in a computer or intelligent system, using formalized information to solve problems according to reasoning control strategies, and the typical function is search and matching.

[0076] Decision refers to the process of decision-making after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0077] (4) General capabilities.

[0078] After data is processed as mentioned above, some general capabilities can be formed based on the results of data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0079] (5) Intelligent products and industry applications.

[0080] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields, which are encapsulations of overall solutions of artificial intelligence, and realize application landing by productizing intelligent information decision-making. The application fields mainly include intelligent manufacturing, intelligent transportation, smart home, intelligent medical treatment, intelligent security, autonomous driving, smart city, intelligent terminal, etc.

[0081] The simulation method provided by the embodiments of the present application can be applied in cloud computing. The system architecture of cloud computing will be described below in conjunction with the schematic diagram. Please refer to FIG. 2, which is a system architecture schematic diagram provided by the embodiments of the present application.

[0082] As shown in FIG. 2, the tenant logs in to the cloud platform 203 through the client 201 via the Internet 202 by using the account and password registered in the cloud platform 203. The cloud platform 203 manages the infrastructure, which includes multiple data centers set in different regions, for example, region 1 includes cloud data center 1 and cloud data center 2, and region 2 includes cloud data center 3 and cloud data center 4 as shown in FIG. 2. Each cloud data center is provided with multiple servers, and the servers run business instances (including at least one of virtual machines, containers, and dedicated hosts).

[0083] In the embodiments of the present application, the simulation service for simulating the performance of the AI task running on the cluster is deployed in the service instance, the tenant purchases the cloud service through the client 201 in the cloud platform 203, and sends a calling request to the cloud platform 203, which is used to request the cloud service from the cloud platform 203. In the embodiments of the present application, the specific content of the cloud service includes simulating the running process of the AI task, so as to evaluate the performance of the AI task running on the cluster. The specific implementation process is described below.

[0084] In the embodiments of the present application, the cloud platform manages a plurality of computing devices and at least one switch, the plurality of computing devices are used for the functions of a plurality of sub-tasks included in the AI task, and the at least one switch is used for connecting the plurality of computing devices and realizing the communication between the plurality of computing devices. The connection relationship between the plurality of computing devices and the at least one switch, and the connection relationship between the at least one switch have multiple possibilities, which are described below in conjunction with the schematic diagram.

[0085] Please refer to FIG. 3 and FIG. 4, both of which are schematic diagrams of system architecture provided by the embodiments of the present application.

[0086] In the embodiment shown in FIG. 3, two switching layers are included, and this architecture can also be referred to as a "spine-leaf" architecture. The two switching layers constitute the network topology of the data center. For example, the switch 1 in FIG. 3 can be considered as a switch in the spine layer, and the switch 2 and the switch 3 can be considered as switches in the leaf layer.

[0087] As shown in FIG. 3, the computing device 1 to the computing device 3 are all connected to the switch 2, the switch 2 is connected to the switch 1, the switch 1 is further connected to the switch 3, and the switch 3 is further connected to the computing device 4 and the computing device 5. Through the connection relationship shown in FIG. 3, the communication between different computing devices is realized.

[0088] It should be noted that the present application does not limit the system architecture for running the AI task, and FIG. 3 is only a schematic diagram of the system architecture. In actual application, more or fewer switches and computing devices can be included, and the number of switching layers can also be more or fewer, which are not limited here.

[0089] For example, in the embodiment shown in FIG. 4, a three-layer network architecture is taken as an example, that is, three levels of switches are included.

[0090] Next, please refer to FIG. 5, which is a flowchart of the simulation method provided by the embodiments of the present application, including:

[0091] 501. Receive input information, the input information including the parameters of the AI model, the parallel strategy of the AI model, the ranktable, the resource topology and the collective communication algorithm used by the AI model.

[0092] The simulation method provided by the embodiments of the present application is executed by a simulation system, and the performance of an AI task when running on a cluster is simulated. The AI task refers to a training task of an AI model or an inference task of an AI model. Moreover, the AI task includes multiple subtasks, and the communication between the multiple subtasks includes multiple communication steps, and the complete communication process of the AI task is implemented based on the multiple communication steps. The cluster running the AI task includes multiple computing devices, and the multiple computing devices are connected through at least one switch. The computing device can be a processor, a server, a virtual machine, a container, etc., and the specific embodiments are not limited here.

[0093] The simulation system simulates the running process of the AI task on the cluster by receiving input information, and outputs the performance of the AI task when running on the cluster. Therefore, the input information is used to describe the AI task and the cluster running the AI task. Specifically, the input information includes: parameters of an AI model, a parallel strategy of the AI model, a ranktable, a resource topology, and a collective communication algorithm used by the AI model.

[0094] Among them, the parameters of the AI model are used to define the AI model, which can represent the complexity of the AI model, and in a popular way, it is defined that the AI model is what kind of model. Exemplarily, the parameters of the AI model include at least one of the following: input layer parameters, hidden layer parameters, neural network layer parameters, loss function parameters, output layer parameters, optimizer parameters, or regularization parameters, etc. Different AI models include different parameters, and the specific embodiments are not limited here.

[0095] The parallel strategy of the AI model is used to indicate the way of distributing the AI model to multiple computing devices in parallel, which can also be understood as the communication mode between multiple subtasks. Exemplarily, the communication mode includes tensor parallelism, pipeline parallelism, data parallelism, etc. One AI model can adopt one or more parallel modes, and the specific embodiments are not limited here.

[0096] The ranktable includes the mapping relationship between the identifiers of the multiple subtasks and the address information of the multiple computing devices, which can be understood as the mapping relationship between the logical identifiers and the physical identifiers. Among them, the address information can be an internet protocol (IP) address, a media access control (MAC) address, or other information capable of representing the address of the computing device, and the specific embodiments are not limited here.

[0097] The resource topology indicates the connection relationship between the multiple computing devices and the at least one switch, and the connection relationship between the at least one switch and each other. That is, the resource topology defines the networking architecture of running the AI task.

[0098] The collective communication algorithm used by the AI model defines how many communication steps there are between the plurality of sub-tasks included in the AI task, and which communication pairs (data sender and data receiver) are included in each communication step, that is, the collective communication algorithm defines how the plurality of sub-tasks complete the plurality of communication steps.

[0099] 502. According to the parameters of the AI model, the parallel strategy of the AI model, the ranktable, the resource topology, and the collective communication algorithm, a first duration of each communication step in the plurality of communication steps is calculated, the first duration being an execution time of the corresponding communication step.

[0100] The simulation system obtains input information and simulates the plurality of communication steps between the plurality of sub-tasks in the AI task, thereby obtaining the first duration of each communication step. The following describes this process in detail:

[0101] The simulation system decomposes the AI task into a plurality of sub-tasks based on the parameters of the AI model and the parallel strategy of the AI model. The parameters of the AI model indicate the amount of data to be processed by the AI task, and the parallel strategy of the AI model indicates the splitting manner of the amount of data. The simulation system splits the amount of data to be processed by the AI task into a plurality of data based on the splitting manner, each piece of data corresponding to a sub-task.

[0102] For example, refer to FIG. 6, which is a schematic diagram provided by an embodiment of the present application. It is assumed that the parallel strategy of the AI task includes data parallelism and pipeline parallelism. In the embodiment shown in FIG. 6, the AI task is represented by a cube. Based on the pipeline parallelism manner, the AI task is decomposed into sub-task 1 and sub-task 2. Based on the data parallelism manner, sub-task 1 is further decomposed into sub-task 1-1 and sub-task 1-2, and sub-task 2 is further decomposed into sub-task 2-1 and sub-task 2-2.

[0103] The simulation system determines the communication topology of the plurality of sub-tasks according to the ranktable and the resource topology, the communication topology indicating the communication connection relationship between the plurality of sub-tasks when the plurality of sub-tasks run on the plurality of computing devices. This process is described in detail in combination with FIG. 7, refer to FIG. 7, which is a flowchart of the simulation method provided by an embodiment of the present application.

[0104] In the embodiment shown in FIG. 7, rankID represents the identification of the subtask, and the IP address represents the address information of the computing device. In the ranktable, the IP address corresponding to the subtask 1 is 29.59.95.178, the IP address corresponding to the subtask 2 is 29.59.115.206, and the IP address corresponding to the subtask 3 is 29.59.176.221. The IP addresses of the computing device A, the computing device B, and the computing device C are respectively 29.59.95.178, 29.59.115.206, and 29.59.176.221. That is, the subtask 1 runs on the computing device A, the subtask 2 runs on the computing device B, and the subtask 3 runs on the computing device C. Then, in combination with the ranktable and the resource topology shown in FIG. 7, the communication topology shown in FIG. 7 is obtained.

[0105] It should be noted that the ranktable in the embodiments of the present application is not limited to the form of a table, but can also be a key-value pair or other forms that can reflect the mapping relationship, and the specific form is not limited herein.

[0106] After the simulation system obtains the communication topology of the plurality of subtasks, the first time length of each communication step can be calculated according to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology. The following describes this process.

[0107] In the embodiments of the present application, the resource topology further indicates the transmission bandwidth and the transmission delay between any two devices having a connection relationship in the plurality of computing devices and the at least one switch. That is, the resource topology indicates the transmission bandwidth and the transmission delay between the computing devices and the switches having a connection relationship, and the transmission bandwidth and the transmission delay between the switches having a connection relationship.

[0108] Correspondingly, the communication topology further indicates the transmission bandwidth and the transmission delay between any two subtasks having a connection relationship in the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices. Specifically, the two subtasks having a connection relationship are connected through the at least one switch, and the transmission bandwidth between the two subtasks having a connection relationship includes the transmission bandwidth between the computing devices running the two subtasks and the switches and the transmission bandwidth between the at least one switch. The transmission delay between the two subtasks having a connection relationship includes the transmission delay between the computing devices running the two subtasks and the switches and the transmission delay between the at least one switch.

[0109] For example, refer to FIG. 8, which is a schematic diagram of a communication topology provided by the embodiments of the present application.

[0110] As shown in FIG. 8, sub-tasks 1, 2, 3 run on computing devices A, B, C respectively, and computing devices A, B, C are connected through switches 1, 2. The resource topology defines the transmission bandwidth and transmission latency between computing device A and switch 1, the transmission bandwidth and transmission latency between switch 1 and switch 2, the transmission bandwidth and transmission latency between computing device B and switch 2, and the transmission bandwidth and transmission latency between computing device B and switch 3.

[0111] Correspondingly, the transmission bandwidth and transmission latency between sub-task 1 and sub-task 2 include the transmission bandwidth and transmission latency between computing device A and switch 1, the transmission bandwidth and transmission latency between switch 1 and switch 2, and the transmission bandwidth and transmission latency between computing device B and switch 2. The transmission bandwidth and transmission latency between sub-task 1 and sub-task 3 include the transmission bandwidth and transmission latency between computing device A and switch 1, the transmission bandwidth and transmission latency between switch 1 and switch 2, and the transmission bandwidth and transmission latency between computing device C and switch 2.

[0112] In summary, the simulation system determines, according to the collective communication algorithm, at least one communication pair included in each communication step, wherein each communication pair includes a sub-task sending data and a sub-task receiving data. According to the communication topology, the simulation system determines a communication link of each communication pair, a transmission bandwidth of each communication link, and a transmission latency of each communication link. According to the parameters of the AI model, the parallel strategy of the AI model, and the collective communication algorithm, the simulation system determines a total communication volume of the at least one communication pair, the total communication volume of the at least one communication pair being a sum of communication volumes of each communication pair. According to the total communication volume of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission latency of each communication link, the simulation system determines a first duration of each communication step.

[0113] The foregoing process is described in detail below with reference to FIG. 8. In the embodiment shown in FIG. 8, it is assumed that a communication step includes two communication pairs, <sub-task 1-sub-task 2> and <sub-task 1-sub-task 3>. Based on the foregoing description of the example of FIG. 8, the simulation system can determine the transmission latency and transmission bandwidth of each of the two communication pairs. Then, based on the parameters of the AI model and the parallel strategy of the AI model, the simulation system determines the total communication volume of each communication mode and the communication domain of each communication mode. In the same communication domain, different sub-tasks communicate based on the same communication mode. Moreover, sub-tasks cannot communicate across communication domains. In combination with the collective communication algorithm, the simulation system determines the communication volume of each communication pair, i.e., the amount of data transmitted by each communication pair. Finally, the simulation system determines the first duration of each communication step, which is the actual execution time of each communication step simulated by the simulation system, or can be understood as the time at which each communication step is completed. In a scheme in which a communication step includes multiple communication pairs, the first duration is the longest duration of transmission among the multiple communication pairs.

[0114] In the embodiment shown in FIG. 8, the communication pair <subtask 1-subtask 2> transmits data 1, and the communication link is “computing device A-switch 1-switch 2-computing device B”, which includes three sub-links, i.e., “computing device A-switch 1”, “switch 1-switch 2”, and “switch 2-computing device B”. The communication pair <subtask 1-subtask 3> is similar, and will not be described here. Then, on the sub-link “computing device A-switch 1” and the sub-link “switch 1-switch 2”, two copies of data are transmitted. The sum of the data amounts of the two copies of data is the total communication amount of this communication step.

[0115] The transmission time of each communication pair is the longest among the multiple sub-links. The transmission time of each sub-link is obtained based on the communication amount, the transmission bandwidth, and the transmission delay of each sub-link, and the relationship satisfies: transmission time = communication amount / transmission bandwidth + transmission delay.

[0116] For example, the transmission time of the sub-link “computing device A-switch 1” is the quotient of the communication amounts of data 1 and data 2 and the transmission bandwidth of the sub-link, plus the transmission delay of the sub-link.

[0117] For example, the transmission time of the sub-link “switch 2-computing device B” is the quotient of the communication amount of data 1 and the transmission bandwidth of the sub-link, plus the transmission delay of the sub-link.

[0118] Then, the simulation system can calculate the transmission time of each communication pair. Then, the longest transmission time is selected as the first duration of the communication step including the multiple communication pairs.

[0119] Based on the foregoing description, in the embodiments of the present application, the AI task is divided into multiple sub-tasks, and the communication topology of the sub-tasks is constructed to determine the communication connection relationship among the multiple sub-tasks. Then, the first duration of each communication step in the multiple communication steps of the multiple sub-tasks is calculated. That is, the simulation system calculates the duration of the communication step from the granularity of the sub-tasks, and the detection granularity of the running performance of the AI task is more fine, and the accuracy of the performance evaluation of the AI task is further improved. In addition, the simulation system can also determine at least one communication pair included in each communication step, and determine the first duration of each communication step in combination with the total communication amount of the at least one communication pair, the transmission link, and the transmission delay of each communication pair. The calculation of the first duration is more fine, and the accuracy of the performance evaluation of the AI task is further improved.

[0120] 503. Output the total communication duration of the AI task, and the total communication duration is the sum of the first durations of each communication step.

[0121] After obtaining the first time length of each communication step by the simulation system, the first time length of each communication step is summed to obtain the total communication time length of the AI task, and the total communication time length is output.

[0122] In the present application, the simulation system simulates the process of the AI task running on the cluster, including simulating multiple communication steps between multiple sub-tasks included in the AI task. By calculating the first time length of each communication step, the total communication time length of the AI task is obtained. The accurate prediction of the communication time length is realized, and the accuracy of the performance evaluation of the AI task is improved.

[0123] In the embodiments of the present application, the simulation system can also evaluate and output other performances of the AI task running on the cluster. The possible implementation modes are described below.

[0124] In some optional embodiments, the simulation system can output the average conflict coefficient of each communication step, which is the quotient of the actual execution time of each step and the ideal execution time, and can intuitively reflect the performance of the AI task in the communication time. The process of determining and outputting the average conflict coefficient of each communication step by the simulation system is described below.

[0125] In summary, the simulation system determines the transmission time of each communication pair according to the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair. The second time length of each communication step is determined, and the second time length is the transmission time of the communication pair with the longest time length in the corresponding communication step. The second time length of each communication step can also be called the conflict-free time length of each step. The average conflict coefficient of each communication step is output, and the average conflict coefficient is the quotient of the first time length and the second time length.

[0126] The transmission time of each communication pair is the ideal transmission time of each communication pair, which can be understood as the time during which each communication pair exclusively transmits data on its communication link.

[0127] For example, in the embodiment shown in FIG. 8, the transmission time of the communication pair <subtask 1 - subtask 2> is the time during which the communication link "computing device A - switch 1 - switch 2 - computing device B" is completely used for transmitting data 1. Specifically, the time during which the sublink "computing device A - switch 1", the sublink "switch 1 - switch 2", and the sublink "switch 2 - computing device B" are used for transmitting data 1 is the longest time. The time during which each sublink is used for transmitting data 1 is the quotient of the data volume of data 1 and the transmission bandwidth of the sublink, plus the transmission delay of the sublink. Based on a similar method, the simulation system also calculates the transmission time of the communication pair <subtask 1 - subtask 3>. Then, the longest time is selected from the transmission times of the two communication pairs as the second time length. Finally, the average conflict coefficient is obtained based on the first time length and the second time length.

[0128] Optionally, the average conflict coefficient satisfies the following formula:

[0129] wherein data step is the communication volume transmitted by a communication pair in a single communication step, is the maximum number of communication pairs in a single communication link in a communication step.

[0130] In the embodiment of the present application, the simulation system can also output the average conflict coefficient of each communication step. Since the average conflict coefficient reflects the multiple of the actual time consumption of a communication step compared with the ideal time consumption, the performance of the AI task in the communication time can be intuitively reflected. The greater the average conflict coefficient, the longer the time consumption of the communication step, which can be used as a potential optimization point of the AI task.

[0131] In some optional embodiments, the simulation system can also output the first time length of each communication step.

[0132] In the embodiment of the present application, in addition to outputting the total communication time length of the AI task, the simulation system can also output the first time length of each communication step, which enriches the dimensions of performance evaluation of the AI task. In addition, according to the first time length of each communication step, the communication step with an excessively long time length can be adjusted, so as to optimize the performance of the AI task running on the cluster.

[0133] In the foregoing embodiments, the time lengths described are calculated from the perspective of the communication step or the AI task. In actual applications, the simulation system can also determine and output other time lengths.

[0134] In some optional embodiments, the parallel strategy of the AI model includes a plurality of communication methods corresponding to a plurality of communication steps. The simulation system can further determine and output, based on the first time length of each communication step, a third time length corresponding to each of the plurality of communication methods, the third time length being a sum of the first time lengths of the communication steps using the same communication method.

[0135] Specifically, the communication step includes at least one communication pair, and the communication method of the communication step is the communication method between the at least one communication pair (i.e., the subtask of sending data and the subtask of receiving data) included in the communication step. Then, one communication step can correspond to one or more communication methods.

[0136] In a general way, the communication method between different subtasks is the parallel mode of the different subtasks. For example, in the embodiment shown in FIG. 6, the communication method between subtask 1-1 and subtask 1-2, and the communication method between subtask 2-1 and subtask 2-2 are both data parallelism. The communication method between subtask 1-1 and subtask 2-1, the communication method between subtask 1-1 and subtask 2-2, the communication method between subtask 1-2 and subtask 2-1, and the communication method between subtask 1-2 and subtask 2-2 are all pipeline parallelism.

[0137] For example, assuming that in the embodiment shown in FIG. 8, the communication methods of the communication pair <subtask 1-subtask 2> and the communication pair <subtask 1-subtask 3> included in the communication step are the same, both being pipeline parallelism. Then, the first time length of this communication step is the time length of this communication method of pipeline parallelism.

[0138] For example, assuming that in the embodiment shown in FIG. 8, the communication methods of the communication pair <subtask 1-subtask 2> and the communication pair <subtask 1-subtask 3> included in the communication step are different, being pipeline parallelism and tensor parallelism respectively. Then, the first time length of this communication step is the time length of this communication method of pipeline parallelism, and is also the time length of this communication method of tensor parallelism.

[0139] In the embodiments of the present application, the simulation system can also output the third time length corresponding to each communication method, evaluate the performance of the AI task from more angles, and enrich the performance evaluation results. In addition, the third time length corresponding to each communication method can also be used as a reference index for optimizing the AI task, for example, optimizing the communication method with long time consumption, so as to optimize the performance of the AI task running on the cluster.

[0140] In some optional embodiments, the simulation system can also output the communication link of each communication pair and / or the communication volume transmitted by each communication pair.

[0141] Exemplarily, as shown in FIG. 9, the simulation system can output the communication link of each communication pair and / or the communication volume transmitted by each communication pair in the form of code. Alternatively, the simulation system can output the foregoing information in other forms, such as images, tables, etc., which are not limited herein.

[0142] The simulation system further enriches the dimensions of performance evaluation of the AI task and can intuitively reflect the communication process between the plurality of subtasks in the AI task.

[0143] In some alternative embodiments, the communication topology further indicates a port set corresponding to the communication link of each communication pair, and the port set includes a port of a computing device that sends data, a port of a computing device that receives data, and a port of at least one switch that transmits data. The computing device that sends data is the computing device that runs the subtask of sending data. The computing device that receives data is the computing device that runs the subtask of receiving data.

[0144] Exemplarily, in the embodiment shown in FIG. 8, the computing device A is the computing device that sends data, the computing device B is the computing device that receives data 1, and the computing device C is the computing device that receives data 2.

[0145] The simulation system determines the communication volume of each port in the port set corresponding to each communication link according to the communication volume of each communication pair. Then, the simulation system determines the total communication volume of each port of each computing device and the total communication volume of each port of each switch in the at least one switch according to the communication volume of each port in the port set corresponding to each communication link after running the AI task. The following further describes the process in combination with the schematic diagram.

[0146] Please refer to FIG. 10, which is a schematic diagram of a communication topology provided in an embodiment of the present application.

[0147] In the embodiment shown in FIG. 10, it is assumed that the communication volume of data 1 is 1 GB and the communication volume of data 2 is 2 GB. In a communication step, if the computing device A sends data 1 through port 1 and sends data 2 through port 2. The switch receives data 1 and data 2 through port 3, sends data 1 to the computing device B through port 4, and sends data 2 to the computing device C through port 5. The computing device B receives data 1 through port 6, and the computing device C receives data 2 through port 7.

[0148] Then, the communication volume of port 1 is 1 GB, the communication volume of port 2 is 2 GB, the communication volume of port 3 is 3 GB, the communication volume of port 4 is 1 GB, the communication volume of port 5 is 2 GB, the communication volume of port 6 is 1 GB, and the communication volume of port 7 is 2 GB.

[0149] The simulation system sums up the traffic of the same port of the same device in each communication step, and thus obtains the total traffic of the port after the AI task is completed.

[0150] In the present application, the simulation system can also determine the traffic of the ports of the devices included in each communication link, and thus determine the total traffic of the ports of the devices after the AI task is run, so as to reflect the data transmission between the multiple sub-tasks when the AI task is run.

[0151] In some optional embodiments, the simulation system can also output the traffic of any one or more ports of the devices in each communication step.

[0152] In some optional embodiments, the simulation system can also output the total traffic of the first port of the target computing device and / or the total traffic of the second port of the target switch. This further enriches the dimensions of the performance evaluation of the AI task by the simulation system and enriches the performance evaluation result.

[0153] The target computing device is included in the multiple computing devices, and the first port is part or all of the ports of the target computing device, that is, the simulation system can output the total traffic of any one or more ports of the target computing device after the AI task is completed.

[0154] The target switch is included in the at least one switch, and the second port is part or all of the ports of the target switch, that is, the simulation system can output the total traffic of any one or more ports of the target switch after the AI task is completed.

[0155] In some optional embodiments, the simulation system can also sum up the traffic of the ports of each device in each communication step, obtain the total traffic of the devices in each communication step, and output the total traffic of one or more devices in each communication step.

[0156] In some optional embodiments, the simulation system can also sum up the total traffic of the same device in each communication step, and obtain the total traffic of each device after the AI task is completed. The simulation system can also output the total traffic of one or more devices after the AI task is completed.

[0157] Next, please refer to FIG. 11, which is a flowchart of the simulation method provided by the embodiments of the present application.

[0158] As shown in FIG. 11, the simulation system obtains input information, which includes parameters of the AI model, a parallel strategy of the AI model, a resource topology, a ranktable, and a collective communication algorithm. The simulation system calculates communication amounts of different parallel modes based on the parameters of the AI model and the parallel strategy of the AI model. The simulation system calculates communication domains of different parallel modes based on the parallel strategy of the AI model. The simulation system determines communication pairs included in each communication step based on the collective communication algorithm. The simulation system constructs communication topologies of multiple communication pairs, i.e., constructs communication topologies of multiple subtasks, based on the ranktable and the resource topology. The simulation system increases the communication amount on a communication link corresponding to each communication step according to the communication pairs and the communication topologies of each communication topology. The simulation system can also calculate a first time length and a second time length of each communication step. The simulation system processes the input information in multiple aspects to obtain output results. The output results can reflect the performance of the AI task in the time length, the communication amount, and the average conflict coefficient dimension.

[0159] The time length includes the first time length of each communication step, the total communication time length of the AI task, the third time length corresponding to each communication method, and the time length introduced in the foregoing embodiments. The communication amount includes the communication amount of the communication step, the communication amount of the port, the communication amount of the device, and the communication amount introduced in the foregoing embodiments. The average conflict coefficient includes the average conflict coefficient of each communication step.

[0160] Further, the simulation method provided in the embodiments of the present application has the flow shown in FIG. 12.

[0161] In the embodiments of the present application, the simulation system has multiple forms of output results. For example, the output results can be as shown in Table 1.

[0162] Table 1

[0163] As shown in the first row of Table 1, the communication pair in the first communication step of the AI task adopts the DP communication mode and passes through a switch with an IP address of 26.71.102.210. The 16th port of the switch is a port for receiving data, and the communication amount via the port is 1.1 GB. The average conflict coefficient of the first communication step is 2, and the transmission time is 0.051 seconds. The transmission time here is the first time length of the communication step in the foregoing embodiments.

[0164] For example, the output results can also be as shown in Table 2.

[0165] Table 2

[0166] The output result shown in Table 2 includes the switch-related information in a single iteration of the training process of the AI model. As shown in the first row of Table 2, the switch with the IP address of 26.71.102.210 receives 1.1 GB of data at the 16th port in a single iteration.

[0167] For example, the output result can also be as shown in Table 3:

[0168] Table 3

[0169] The output result shown in Table 2 includes the switch-related information in a single iteration of the training process of the AI model. As shown in the first row of Table 2, the switch with the IP address of 26.71.102.210 receives 1.1 GB of data at the 16th port in a single iteration.

[0170] For example, the output result of the simulation can also be in the form of a flowchart as shown in FIG. 13. Referring to FIG. 13, FIG. 13 is a schematic diagram of an output result provided by an embodiment of the present application.

[0171] In the embodiment shown in FIG. 13, the “spine-leaf” switch architecture is taken as an example. The spines 5, 6, and 12 represent the switches in the spine layer. The switches with the IP addresses of 26.71.102.224 and 26.71.102.227 are included in the communication topology as switches in the leaf layer, which can also be referred to as tor switches. The rectangular boxes in the lowermost layer represent the computing devices running the sub-tasks.

[0172] The simulation system simulates the performance of the AI task running on the cluster, and can calculate the actual communication volume of each sub-link, which can be used for comparison with the theoretical communication volume.

[0173] It should be further noted that the number of AI tasks simulated by the simulation system is not limited to one or more, and is not limited herein.

[0174] Next, the related devices provided by the embodiments of the present application are described.

[0175] Referring to FIG. 14, FIG. 14 is a structural schematic diagram of a simulation system provided by an embodiment of the present application. As shown in FIG. 14, the simulation system 1400 includes a simulation module 1401, a calculation module 1402, and a visualization module 1403.

[0176] The simulation module 1401 is configured to simulate the process of the AI task running on the cluster based on the input information.

[0177] The computing module 1402 is configured to calculate performance of the simulation AI task running on the cluster, including the time length, the communication volume, the average conflict coefficient, and the like introduced in the foregoing embodiments. The performance can be output as a result of the simulation.

[0178] The visualizing module 1403 is configured to display the output result of the simulation in a visualized manner.

[0179] The simulation system 1400 is configured to implement the simulation method provided in the embodiments of the present application, and details are shown in the foregoing, which will not be repeated here.

[0180] Please refer to FIG. 15, which is a structural schematic diagram of a simulation system provided in the embodiments of the present application. In the embodiments of the present application, the simulation system 1500 is configured to simulate performance of an AI task running on a cluster, the AI task including training or inference of an AI model, the cluster including a plurality of computing devices connected through at least one switch. The AI task includes a plurality of subtasks, and communication between the plurality of subtasks includes a plurality of communication steps.

[0181] As shown in FIG. 15, the simulation system 1500 includes an obtaining unit 1501 and a processing unit 1502.

[0182] In some optional embodiments, the obtaining unit 1501 is configured to receive input information, the input information including parameters of the AI model, a parallel strategy of the AI model, a ranktable, a resource topology, and a collective communication algorithm used by the AI model, the ranktable including a mapping relationship between identifications of the plurality of subtasks and address information of the plurality of computing devices, and the resource topology indicating a connection relationship between the plurality of computing devices and the at least one switch, and between the at least one switch and each other.

[0183] The processing unit 1502 is configured to calculate a first time length of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the ranktable, the resource topology, and the collective communication algorithm, the first time length being an execution time of the corresponding communication step. The processing unit 1502 is further configured to output a total communication time length of the AI task, the total communication time length being a sum of the first time lengths of each communication step.

[0184] In some optional embodiments, the processing unit 1502 is specifically configured to: decompose the AI task into the plurality of subtasks according to the parameters of the AI model and the parallel strategy of the AI model; determine a communication topology of the plurality of subtasks according to the ranktable and the resource topology, the communication topology indicating a communication connection relationship between the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; and calculate the first time length of each communication step according to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology.

[0185] In some optional embodiments, the resource topology further indicates a transmission bandwidth and a transmission latency between any two devices having a connection relationship in the plurality of computing devices and the at least one switch. The communication topology further indicates a transmission bandwidth and a transmission latency between any two sub-tasks having a link relationship in the plurality of sub-tasks when the plurality of sub-tasks run on the plurality of computing devices.

[0186] The processing unit 1502 is specifically configured to: determine, according to the collective communication algorithm, at least one communication pair included in each communication step, where each communication pair includes a sub-task sending data and a sub-task receiving data. Determine, according to the communication topology, a communication link of each communication pair, a transmission bandwidth of each communication link, and a transmission latency of each communication link. Determine, according to the parameters of the AI model, the parallel strategy of the AI model, and the collective communication algorithm, a total communication volume of the at least one communication pair, the total communication volume of the at least one communication pair being a sum of communication volumes of each communication pair. Determine, according to the total communication volume of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission latency of each communication link, a first duration of each communication step.

[0187] In some optional embodiments, the processing unit 1502 is further configured to: determine, according to the communication volume of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission latency of the communication link of each communication pair, respectively, a transmission time of each communication pair. Determine a second duration of each communication step, the second duration being the transmission time of the communication pair with the longest duration in the corresponding communication step. Output an average conflict coefficient of each communication step, the average conflict coefficient being a quotient of the first duration and the second duration.

[0188] In some optional embodiments, the processing unit 1502 is further configured to output the first duration of each communication step.

[0189] In some optional embodiments, the parallel strategy of the AI model includes a plurality of communication methods corresponding to the plurality of communication steps.

[0190] The processing unit 1502 is further configured to: determine, based on the first duration of each communication step, a third duration corresponding to each communication method in the plurality of communication methods, the third duration being a sum of the first durations of the communication steps using the same communication method. Output the third duration corresponding to each communication method.

[0191] In some optional embodiments, the processing unit 1502 is further configured to output the communication link of each communication pair and / or the communication volume transmitted by each communication pair.

[0192] In some optional embodiments, the communication topology further indicates a port set corresponding to each communication link of each communication pair, the port set including a port of the computing device sending data, a port of the computing device receiving data, and a port of the at least one switch transmitting data.

[0193] The processing unit 1502 is further configured to determine, according to the communication volume of each communication pair, a communication volume of each port in the port set corresponding to each communication link. The processing unit 1502 is further configured to determine, according to the communication volume of each port in the port set corresponding to each communication link, a total communication volume of each port of each computing device in the plurality of computing devices and a total communication volume of each port of each switch in the at least one switch after running the AI task.

[0194] In some optional embodiments, the processing unit 1502 is further configured to output the total communication volume of the first port of the target computing device and / or the total communication volume of the second port of the target switch, the target computing device being included in the plurality of computing devices, and the target switch being included in the at least one switch.

[0195] The obtaining unit 1501 and the processing unit 1502 can be implemented by software or by hardware. For example, the implementation of the processing unit 1502 is described below.

[0196] As an example of a software functional unit, the processing unit 1502 can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device, processor, server), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the processing unit 1502 can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Generally, one region can include multiple AZs.

[0197] Likewise, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Among them, usually one VPC is set in one region, and communication between two VPCs in the same region and between VPCs in different regions needs to set a communication gateway in each VPC to realize the interconnection between VPCs through the communication gateway.

[0198] As an example of a hardware functional unit, the processing unit 1502 can include at least one computing device, such as a server, etc. Alternatively, the processing unit 1502 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. Among them, the above-mentioned PLD can be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0199] The multiple computing devices included in the processing unit 1502 can be distributed in the same region or in different regions. The multiple computing devices included in the processing unit 1502 can be distributed in the same AZ or in different AZs. Likewise, the multiple computing devices included in the processing unit 1502 can be distributed in the same VPC or in multiple VPCs. Among them, the multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs, etc.

[0200] It should be noted that the acquisition unit 1501 and the processing unit 1502 respectively implement different steps in the simulation method to realize the overall function of the simulation system 1500. The simulation system 1500 is used to perform the operations performed by the cloud platform in the embodiment shown in FIG. 2 or the operations performed by the simulation system in the embodiments shown in FIGS. 5 to 13 to realize the simulation method provided in the present application. Details are shown in the foregoing, and will not be repeated here.

[0201] Referring to FIG. 16, FIG. 16 is a structural schematic diagram of a computing device provided by an embodiment of the present application. The computing device 1600 includes a processor 1601, a communication interface 1602, a bus 1603 and a memory 1604. Among them, the processor 1601, the communication interface 1602 and the memory 1604 communicate through the bus 1603, and in actual application, communication can also be realized through wireless transmission and other means, which is not limited here.

[0202] The computing device 1600 can be a server or a terminal device, and it should be understood that the number of processors and memories in the computing device 1600 is not limited by the present application.

[0203] The processor 1601 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP) and the like.

[0204] The communication interface 1602 uses a transceiver module such as but not limited to a network interface card, a transceiver, and the like to realize the communication between the computing device 1600 and other devices or communication networks.

[0205] The bus 1603 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus and the like. The bus can be divided into address bus, data bus, control bus and the like. For the convenience of representation, only one line is shown in FIG. 16, but it does not mean that there is only one bus or one type of bus. The bus 1603 can include a channel for transmitting information between various components (for example, the memory 1604, the processor 1601, the communication interface 1602) of the computing device 1600.

[0206] The memory 1604 can include a volatile memory such as a random access memory (RAM). The memory 1604 can also include a non-volatile memory such as a read-only memory (ROM), a flash memory, a mechanical hard disk drive (HDD) or a solid state drive (SSD).

[0207] The executable program code stored in the memory 1604 is executed by the processor 1601 to realize the functions of the aforementioned acquisition unit 1501 and processing unit 1502 respectively, thereby realizing the simulation method. That is, the instructions for executing the simulation method are stored in the memory 1604.

[0208] The embodiments of the present application also provide a computing device cluster, which includes at least one computing device. The computing device can be a server, for example, a central server, an edge server, or a local server in a local data center. In some optional embodiments, the computing device can also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

[0209] Please refer to FIG. 17 and FIG. 18, both of which are structural schematic diagrams of the computing device cluster provided by the embodiments of the present application.

[0210] As shown in FIG. 17, the computing device cluster includes at least one computing device 1600. The memory 1604 in one or more computing devices 1600 in the computing device cluster can store the same instructions for executing the simulation method provided by the embodiments of the present application.

[0211] In some possible embodiments, the memory 1604 of one or more computing devices 1600 in the computing device cluster can also respectively store partial instructions for executing the simulation method. In other words, the combination of one or more computing devices 1604 can collectively execute the instructions for executing the simulation method.

[0212] It should be noted that the memories 1604 in different computing devices 1600 in the computing device cluster can store different instructions, respectively used to execute part of the operations of the cloud platform or the simulation system. That is, the instructions stored in the memories 1604 in different computing devices 1600 can realize the functions of one or more units of the acquisition unit 1501 and the processing unit 1502, so as to realize the simulation method provided by the embodiments of the present application.

[0213] In some possible implementation, one or more of the computing devices in the cluster of computing devices can be connected through a network. The network can be a wide area network or a local area network, etc. Figure 18 illustrates one possible implementation. As shown in Figure 18, two computing devices 1600A and 1600B are connected through a network. Specifically, the computing devices are connected to the network through a communication interface in each of the computing devices. In this type of possible implementation, the memory 1604 in the computing device 1600A stores instructions for performing the functions of the obtaining unit 1501. Meanwhile, the memory 1604 in the computing device 1600B stores instructions for performing the functions of the processing unit 1502.

[0214] The connection between the cluster of computing devices shown in Figure 18 can be such that the functions of the obtaining unit 1501 are performed by the computing device 1600A and the functions of the processing unit 1502 are performed by the computing device 1600B, considering that the simulation method provided in the present application separates the processing operation from the operations other than the processing operation.

[0215] It should be understood that the functions of the computing device 1600A shown in Figure 18 can also be performed by multiple computing devices 1600. Similarly, the functions of the computing device 1600B can also be performed by multiple computing devices 1600.

[0216] The embodiments of the present application also provide another cluster of computing devices. The connection between the computing devices in the cluster of computing devices can be similar to the connection between the computing devices in the cluster of computing devices described with reference to Figures 17 and 18, which will not be described herein again.

[0217] The embodiments of the present application also provide a computer program product containing instructions. The computer program product can be software or a program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device is caused to perform the simulation method provided in the embodiments of the present application.

[0218] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium can be any available medium that can be used to store data that can be accessed by a computing device or a data storage device such as a data center containing one or more available media. The available media can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk), etc. The computer readable storage medium contains instructions that instruct the computing device to perform the simulation method provided in the embodiments of the present application.

[0219] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0220] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A simulation method, characterized by, The method is performed by a simulation system for simulating performance of an AI task running on a cluster, the AI task including training or inference of an AI model, the cluster including a plurality of computing devices connected by at least one switch; the AI task including a plurality of subtasks, communication among the plurality of subtasks including a plurality of communication steps; The method includes: receiving input information including parameters of the AI model, a parallel strategy of the AI model, a rank table, a resource topology, and a collective communication algorithm used by the AI model, the rank table including a mapping relationship between identifications of the plurality of subtasks and address information of the plurality of computing devices, the resource topology indicating connection relationships among the plurality of computing devices and the at least one switch, among the at least one switch and each other; calculating a first duration of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the rank table, the resource topology, and the collective communication algorithm, the first duration being an execution time of the corresponding communication step; outputting a total communication duration of the AI task, the total communication duration being a sum of the first durations of the plurality of communication steps.

2. The method of claim 1, wherein, The calculation of the first duration of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the rank table, the resource topology, and the collective communication algorithm includes: decomposing the AI task into the plurality of subtasks according to the parameters of the AI model and the parallel strategy of the AI model; determining a communication topology of the plurality of subtasks according to the rank table and the resource topology, the communication topology indicating a communication connection relationship among the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; calculating the first duration of each communication step according to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology.

3. The method of claim 2, wherein, The resource topology further indicates a transmission bandwidth and a transmission latency between any two devices having a connection relationship in the plurality of computing devices and the at least one switch; The communication topology further indicates a transmission bandwidth and a transmission latency between any two subtasks having a connection relationship in the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; The calculation of the first duration of each communication step according to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology includes: determining at least one communication pair included in each communication step according to the collective communication algorithm, wherein each communication pair includes a subtask sending data and a subtask receiving data; determining a communication link of each communication pair, a transmission bandwidth of each communication link, and a transmission latency of each communication link according to the communication topology; determining, according to the parameters of the AI model, the parallel strategy of the AI model, and the collective communication algorithm, a total traffic of the at least one communication pair, the total traffic of the at least one communication pair being a sum of the traffic of each communication pair; determining, according to the total traffic of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission latency of each communication link, a first duration of each communication step.

4. The method of claim 3, wherein, The method further comprises: determining, according to the traffic of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission latency of the communication link of each communication pair, a transmission time of each communication pair; determining a second duration of each communication step, the second duration being the transmission time of the communication pair with the longest duration in the corresponding communication step; outputting an average conflict coefficient of each communication step, the average conflict coefficient being a quotient of the first duration and the second duration.

5. The method according to claim 3 or 4, characterized in that, The method further comprises: outputting the first duration of each communication step.

6. The method according to any one of claims 2 to 5, characterized in that, The parallel strategy of the AI model comprises a plurality of communication methods corresponding to the plurality of communication steps; the method further comprises: determining, based on the first duration of each communication step, a third duration corresponding to each communication method in the plurality of communication methods, the third duration being a sum of the first duration of the communication steps using the same communication method; outputting the third duration corresponding to each communication method.

7. The method according to any one of claims 4 to 6, characterized in that, The method further comprises: outputting the communication link of each communication pair and / or the traffic transmitted by each communication pair.

8. The method according to any one of claims 4 to 7, characterized in that, The communication topology further indicates a port set corresponding to the communication link of each communication pair, the port set comprising a port of a computing device sending data, a port of a computing device receiving data, and a port of at least one switch transmitting data; The method further comprises: determining, according to the traffic of each communication pair, the traffic of each port in the port set corresponding to each communication link; determining, according to the traffic of each port in the port set corresponding to each communication link, a total traffic of each port of each computing device in the plurality of computing devices and a total traffic of each port of each switch in the at least one switch after running the AI task.

9. The method of claim 8, wherein, The method further comprises: outputting the total traffic of the first port of the target computing device and / or the total traffic of the second port of the target switch, the target computing device being included in the plurality of computing devices, and the target switch being included in the at least one switch.

10. An emulation system, comprising: The simulation system is used to simulate the performance of an AI task running on a cluster, the AI task comprising training or inference of an AI model, the cluster comprising a plurality of computing devices connected through at least one switch; the AI task comprising a plurality of subtasks, communication between the plurality of subtasks comprising a plurality of communication steps; the simulation system comprising: The acquisition unit is configured to receive input information, the input information including parameters of the AI model, a parallel strategy of the AI model, a ranktable, a resource topology, and a collective communication algorithm used by the AI model, the ranktable including a mapping relationship between identifications of the plurality of subtasks and address information of the plurality of computing devices, and the resource topology indicating a connection relationship between the plurality of computing devices and the at least one switch, and between the at least one switch and each other; The processing unit is configured to calculate a first duration of each communication step in the plurality of communication steps according to the parameters of the AI model, the parallel strategy of the AI model, the ranktable, the resource topology, and the collective communication algorithm, the first duration being an execution time of the corresponding communication step. The processing unit is further configured to output a total communication duration of the AI task, the total communication duration being a sum of the first durations of the plurality of communication steps.

11. The simulation system of claim 10, wherein, The processing unit is specifically configured to: According to the parameters of the AI model and the parallel strategy of the AI model, the AI task is decomposed into the plurality of subtasks; According to the ranktable and the resource topology, a communication topology of the plurality of subtasks is determined, the communication topology indicating a communication connection relationship between the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; According to the parameters of the AI model, the parallel strategy of the AI model, the collective communication algorithm, and the communication topology, the first duration of each communication step is calculated.

12. The simulation system of claim 11, wherein, The resource topology further indicates a transmission bandwidth and a transmission delay between any two devices having a connection relationship in the plurality of computing devices and the at least one switch; The communication topology further indicates a transmission bandwidth and a transmission delay between any two subtasks having a connection relationship in the plurality of subtasks when the plurality of subtasks run on the plurality of computing devices; The processing unit is specifically configured to: According to the collective communication algorithm, at least one communication pair included in each communication step is determined, wherein each communication pair includes a subtask sending data and a subtask receiving data; According to the communication topology, a communication link of each communication pair, a transmission bandwidth of each communication link, and a transmission delay of each communication link are determined; According to the parameters of the AI model, the parallel strategy of the AI model, and the collective communication algorithm, a total communication amount of the at least one communication pair is determined, the total communication amount of the at least one communication pair being a sum of communication amounts of each communication pair; According to the total communication amount of the at least one communication pair, the communication link of each communication pair, the transmission bandwidth of each communication link, and the transmission delay of each communication link, the first duration of each communication step is determined.

13. The simulation system of claim 12, wherein, The processing unit is further configured to: According to the communication amount of each communication pair, the communication link of each communication pair, the transmission bandwidth of the communication link of each communication pair, and the transmission delay of the communication link of each communication pair, a transmission time of each communication pair is determined respectively. determine a second duration of each of the communication steps, the second duration being a transmission time of a communication pair with a longest duration in a corresponding communication step; output an average conflict coefficient of each of the communication steps, the average conflict coefficient being a quotient of the first duration and the second duration.

14. The simulation system of claim 12 or 13, characterized in that, The processing unit is further configured to output the first duration of each of the communication steps.

15. The simulation system of any one of claims 11 to 14, wherein, The parallel strategy of the AI model comprises a plurality of communication methods corresponding to the plurality of communication steps; the processing unit is further configured to: determine, based on the first duration of each of the communication steps, a third duration corresponding to each of the plurality of communication methods, the third duration being a sum of the first duration of the communication steps using a same communication method; output the third duration corresponding to each of the plurality of communication methods.

16. The simulation system of any one of claims 13 to 15, wherein, The processing unit is further configured to output a communication link of each of the communication pairs and / or a communication volume transmitted by each of the communication pairs.

17. The simulation system of any one of claims 13 to 16, wherein, The communication topology further indicates a port set corresponding to a communication link of each of the communication pairs, the port set comprising a port of a computing device sending data, a port of a computing device receiving data, and a port of at least one switch transmitting data; the processing unit is further configured to: determine, according to the communication volume of each of the communication pairs, a communication volume of each port in the port set corresponding to each of the communication links; determine, according to the communication volume of each port in the port set corresponding to each of the communication links, a total communication volume of each port of each of the plurality of computing devices and a total communication volume of each port of each of the at least one switch after running the AI task.

18. The simulation system of claim 17, wherein, The processing unit is further configured to: output the total communication volume of the first port of the target computing device and / or the total communication volume of the second port of the target switch, the target computing device being included in the plurality of computing devices, and the target switch being included in the at least one switch.

19. A cluster of computing devices, characterized in that, The at least one computing device comprises a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method of any one of claims 1 to 9.

20. A computer program product comprising instructions, characterized in that, The instructions, when executed by the computing device cluster, cause the computing device cluster to perform the method of any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises computer program instructions, which, when executed by the computing device cluster, cause the computing device cluster to perform the method of any one of claims 1 to 9.

Images