A computing system and a method of controlling communication relationships among computing groups

Abstract

The application discloses a computing system and a communication relationship control method among computing groups, the computing system comprising N computing groups, a plurality of computing cards in each computing group being electrically connected, and the plurality of computing groups being connected through an optical switching device; a management node distributes X subtasks in an AI task to X computing cards of an i-th computing group to an N-th computing group with less available resource quantity according to available resource quantity of each computing group, distributes Y subtasks having a communication relationship with the X subtasks to the first computing group to the i-2-th computing group or the i-1-th computing group according to the communication relationship among M subtasks, and distributes remaining M-(X+Y) subtasks to the i-1-th computing group. Through the operation of the management node, the computing group with less available resource quantity only needs to perform inter-group communication with one or as few computing groups as possible, data conflict caused by few optical ports is solved, and communication performance is improved.

Description

Technical Field

[0001] This application relates to the field of computing, and more particularly to a computing system and a method for controlling communication relationships between computing groups. Background Technology

[0002] The optoelectronic hybrid network comprises multiple computing groups for executing AI tasks. Each computing group includes multiple computing cards, which communicate with each other via electrical switches. The computing groups communicate with each other via optical switches. Typically, an AI task is divided into multiple subtasks, which are then assigned to multiple computing cards within the computing groups for execution. During the execution of an AI task, communication between these subtasks requires cross-computing group communication. However, since optical switches can only communicate point-to-point via optical ports, the connection paths between computing groups are limited. When there is a large amount of communication between computing groups, communication conflicts can easily occur, especially for computing groups with a small number of optical ports, where the conflicts are more severe. Therefore, reducing communication between computing groups is an urgent problem to be solved. Summary of the Invention

[0003] This application provides a method for controlling the communication relationship between a computing system and computing groups. By having a management node allocate multiple communication objects corresponding to subtasks executed by computing groups with fewer available resources to the same computing group based on the determined communication relationship between tasks and the available resources provided by multiple computing groups, the data transmission conflict caused by insufficient optical ports can be avoided, thus affecting the system's communication performance.

[0004] In a first aspect, embodiments of this application provide a computing system comprising N computing groups, each computing group comprising multiple computing cards, the multiple computing cards within each computing group being connected via an electrical switching device, and the multiple computing groups being connected via an optical switching device. A management node in the computing system pre-obtains the communication relationships between M subtasks executing an AI task. Then, it sorts the N computing groups that need to execute the task according to their available resources, wherein the available resources of the (j-1)th computing group are greater than or equal to the available resources of the jth computing group. X subtasks from the M subtasks are assigned to X computing cards in the i-th to N-th computing groups with fewer available resources. Y subtasks that have communication relationships with the X subtasks are assigned to the 1st to (i-2nd)th or (i-1st)th computing groups with larger available resources. The remaining M-(X+Y) subtasks are assigned to the (i-1th)th computing group.

[0005] In summary, the computing system provided in this application embodiment, through a management node, based on the communication relationships between multiple subtasks of an acquired AI task and the available resources provided by multiple computing groups participating in the task execution, prioritizes allocating computing cards to some subtasks of the AI ​​task to the computing group at the bottom of the list according to the order of available resources (e.g., the number of available optical ports). Then, it selects the computing group at the top of the list to allocate computing cards to subtasks that have communication relationships with the already allocated subtasks. Next, it selects the computing group with the second-to-last available resources to allocate computing cards to the remaining subtasks, and so on, until all subtasks are allocated. This can effectively reduce the communication volume between computing groups, thereby reducing conflicts, especially for computing groups with limited available resources, which can more effectively reduce conflicts and improve communication efficiency.

[0006] For example, the specific process by which the management node assigns M subtasks of an AI task to M computing cards in N computing groups is as follows: X1 subtasks are assigned to the Nth computing group among the N computing groups, and Y1 subtasks that have communication relationships with the X1 subtasks are pre-determined to be assigned to computing cards in the first computing group among the N computing groups, where the first computing group is the computing group with the largest amount of available resources.

[0007] If the number of computing cards available in the first computing group is greater than or equal to Y1, then Y1 subtasks are assigned to Y1 computing cards in the first computing group. However, if the number of computing cards available in the first computing group is less than Y1, then Y1 subtasks are assigned to the first computing group and the second computing group out of N computing groups, each with its own Y1 computing cards. Next, the management node assigns X2 subtasks from the remaining subtasks to X2 computing cards in the (N-1)th computing group, and assigns Y2 computing cards to Y2 subtasks in the first and / or second computing groups that have communication relationships with X2 computing cards, until the remaining subtasks are assigned to computing cards in the (i-1)th computing groups.

[0008] The above process further illustrates that the computing group with the fewest available resources is most prone to data transmission bottlenecks and communication conflicts. The management node first assigns subtasks to the computing group with the fewest available resources, and then assigns multiple subtasks that have communication relationships with the subtasks to the computing group with the most available resources. Only when the computing group with the most available resources can provide a limited number of computing cards will multiple subtasks be assigned to the computing group with the most available resources and the computing group with the second most available resources. Then, the above steps are repeated to reselect the computing group with the fewest available resources among the multiple computing groups that have not been assigned subtasks. This can minimize the communication volume between the computing group with fewer available resources and multiple computing groups, and ensure the communication performance of the system.

[0009] For example, when the number of computing cards in the first computing group is less than Y1, the specific process by which the management node allocates Y1 subtasks to Y1 computing cards in the first and second computing groups is as follows: The management node allocates the y1 subtasks that have a communication relationship with the first subtask of the X1 subtasks to the y1 computing cards in the first computing group; firstly, the first computing group is selected to allocate computing cards to the y2 subtasks that have a communication relationship with the second subtask of the X1 subtasks; then, it is determined whether the number of remaining computing cards in the first computing group can cover the number of y2 subtasks. When the number of remaining computing cards in the first computing group is greater than or equal to y2, the y2 subtasks are allocated to the y2 computing cards in the first computing group; or, when the number of remaining computing cards in the first computing group is less than y2, the y2 subtasks are allocated to the y2 computing cards in the second computing group, so that multiple subtasks that each subtask in the X1 subtasks needs to communicate are allocated to the same computing group as much as possible.

[0010] When multiple subtasks need to be assigned to more than one computing group, the above process assigns multiple subtasks that have communication relationships among the X1 subtasks to the same computing group based on the available resources included in the computing group. This ensures that the same computing card only needs to communicate with one computing group, thereby reducing data conflicts and improving communication performance.

[0011] For example, the amount of available resources for each computing group is the number of optical ports corresponding to each computing group.

[0012] The number of optical ports is typically related to the number of computing cards in each computing group. With a larger number of computing cards, each card may exchange data with other computing cards through optical ports, thus requiring a greater number of optical ports. The number of optical ports may also be related to switch configuration, etc., but this application does not impose specific limitations. In a computing system, the optical ports corresponding to a computing group can support the high bandwidth requirements of AI tasks. Optical communication links are established through the optical ports in two computing groups, thereby meeting the inter-group communication needs between computing groups.

[0013] For example, when the management node is the scheduler in the computing system, the process by which the management node obtains the communication relationship between the M subtasks of an AI task is as follows: The scheduler receives the AI ​​task issued by the AI ​​platform, as well as the number of computing cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm, and generates the communication relationship between the M subtasks of the AI ​​task based on the number of computing cards, the parallel strategy, and the communication algorithm.

[0014] In the above process, the parallelism strategy includes one or more of the following: data parallelism (DP), tensor parallelism (TP), and pipeline parallelism (PP). The communication algorithm includes one of the following: broadcast, gather, scatter, all-gather, reduce, all-reduce, and reduce-scatter. In addition to the types mentioned above, the parallelism strategy and communication algorithm may include more types, which are not specifically limited in this application. The communication relationship between the M subtasks of the AI ​​task can be determined based on the number of computing cards executing the subtasks, the parallelism strategy, and the communication algorithm.

[0015] Communication relationships can be represented by multiple communication pairs. Each communication pair can be represented by Rank_ID. Each Rank_ID is used to indicate a subtask. For example, in the case of M subtasks, Rank_0 to Rank_(M-1) are determined, and multiple communication pairs such as (Rank_0, Rank_1), (Rank_0, Rank_M), and (Rank_1, Rank_2) are determined, thereby determining the communication relationships between the M subtasks of the AI ​​task, as well as the multiple subtasks with communication relationships corresponding to each subtask.

[0016] For example, when the management node is a computing card in the computing system, the process by which the management node obtains the communication relationship between M subtasks that execute an AI task is as follows: the computing card obtains the communication relationship between the M subtasks from the scheduler, and the specific process by which the scheduler obtains the communication relationship between the M subtasks will not be described here.

[0017] Secondly, embodiments of this application provide a method for controlling communication relationships between computing groups. This method is applied to a management node in a computing system, which includes N computing groups, each computing group including multiple computing cards. The multiple computing cards within each computing group are electrically connected, and the multiple computing groups are connected to each other through an optical switching device. The method includes: acquiring the communication relationships between M subtasks executing an AI task; allocating X subtasks from the M subtasks to X computing cards in the i-th to N-th computing groups of the N computing groups, wherein the N computing groups are sorted according to resource quantity, and the available resource quantity of the (j-1)-th computing group is greater than or equal to the available resource quantity of the j-th computing group; allocating Y subtasks from the M subtasks that have communication relationships with the X subtasks to the 1-i-2 or 1-i-1 computing groups of the N computing groups, and allocating the remaining M-(X+Y) subtasks to the (i-1)-th computing group.

[0018] For example, the specific process of assigning Y subtasks that communicate with X subtasks out of M subtasks to the first to the (i-2)th computing group or the first to the (i-1)th computing group in N computing groups, and assigning the remaining M-(X+Y) subtasks to the (i-1)th computing group, is as follows: Assign X1 subtasks to the Nth computing group; select the first computing group in N computing groups to assign computing cards to the Y1 subtasks that communicate with X1 subtasks; when the number of computing cards in the first computing group is greater than or equal to the number of subtasks in the first computing group, the assignment is made according to the following steps: When the number of subtasks equals Y1, the Y1 subtasks are assigned to the Y1 computing cards in the first computing group. When the number of computing cards in the first computing group is less than Y1, the Y1 subtasks are assigned to the Y1 computing cards in the first computing group and the second computing group. Continue to select the (N-1)th computing group to assign X2 computing cards to the remaining X2 subtasks, and continue to assign computing cards to the Y2 subtasks that have communication relationships with the X2 computing cards in the first computing group or the second computing group, until all the remaining subtasks are assigned to the computing cards in the (i-1)th computing group.

[0019] For example, when the number of computing cards in the first computing group is less than Y1, allocating Y1 subtasks to Y1 computing cards in the first and second computing groups includes: allocating y1 subtasks that have a communication relationship with the xth subtask in the X1 subtasks to y1 computing cards in the first computing group; selecting the first computing group to allocate computing cards to y2 subtasks that have a communication relationship with the (x+1)th subtask in the X1 subtasks; when the number of remaining computing cards in the first computing group is greater than or equal to y2, allocating y2 subtasks to y2 computing cards in the first computing group; when the number of remaining computing cards in the first computing group is less than y2, allocating y2 subtasks to y2 computing cards in the second computing group.

[0020] For example, the amount of available resources for each computing group is the number of optical ports corresponding to each computing group.

[0021] For example, the computing system also includes a scheduler. When the management node is deployed on the scheduler, the specific process of obtaining the communication relationship between the M subtasks that execute an AI task is as follows: The scheduler receives the AI ​​task and the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm; and generates the communication relationship between the M subtasks based on the number of cards, the parallel strategy, and the communication algorithm.

[0022] For example, when the management node is deployed on the computing card, the specific process of obtaining the communication relationship between the M subtasks that execute an AI task is as follows: The computing card obtains the communication relationship between the M subtasks that execute an AI task from the scheduler of the computing system.

[0023] Thirdly, embodiments of this application provide a scheduler for deploying management nodes to execute the method described in the second aspect.

[0024] Fourthly, embodiments of this application provide a chip for deploying a management node to perform the method described in the second aspect.

[0025] Fifthly, embodiments of this application provide a computer program product containing instructions that, when executed by a management node, cause the management node to implement the method described in the second aspect.

[0026] In a sixth aspect, embodiments of this application provide a computer-readable storage medium including computer program instructions that, when executed by a computing device, cause a management node to implement the method described in the second aspect.

[0027] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0029] Figure 1A This is a schematic diagram of a port connection mode that meets the requirements of Mesh communication, provided in an embodiment of this application.

[0030] Figure 1B This is a schematic diagram of a port connection mode that meets the requirements of Ring communication, provided in an embodiment of this application.

[0031] Figure 1C This is a schematic diagram of another port connection mode provided in an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the structure of a task system provided in an embodiment of this application;

[0033] Figure 3 This is a flowchart illustrating the operation of a computing system when a management node is deployed on a scheduler, as provided in an embodiment of this application.

[0034] Figure 4 This is a flowchart illustrating the operation of a computing system when a management node is deployed on a computing card, as provided in an embodiment of this application.

[0035] Figure 5 This is a schematic diagram of subtask allocation under a hybrid parallel strategy provided in an embodiment of this application;

[0036] Figure 6 This is a schematic diagram of subtask allocation under another hybrid parallel strategy provided in the embodiments of this application;

[0037] Figure 7 This is a schematic diagram of subtask allocation under an HD algorithm provided in an embodiment of this application;

[0038] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0039] Figure 9 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0041] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0042] Currently, the scale of computing systems performing AI tasks is growing rapidly, with the number of individual hardware cards (NICs) increasing dramatically, reaching tens of thousands or even hundreds of thousands of cards. This leads to increased system deployment costs, and the electrical switches in traditional Fat-tree networks cannot meet the data transmission rate requirements. Furthermore, as the number of NICs increases, the number of switch layers in the network topology increases, resulting in more complex cabling and a higher system failure rate. To address these issues, a hybrid optical-electrical network is proposed to enable communication between computing cards in the computing system. This hybrid network comprises multiple computing groups, with multiple computing cards within each group communicating via electrical switches, and communication between computing groups via optical switches. Since optical switching equipment has significant advantages in terms of capacity, cost, and power consumption, this hybrid optical-electrical network can effectively improve network capacity, reduce network construction costs, and thus lower system deployment costs.

[0043] Unlike electrical switching equipment, which has full connectivity, optical switching equipment uses a point-to-point port-level forwarding communication mode. It can only form a fixed network topology through point-to-point ports and cannot form a fully connected network between any ports. In order to meet different communication needs, optical switching equipment needs to be configured with different port connection modes to achieve switching between different network topologies.

[0044] Specifically, such as Figure 1A As shown, Figure 1A This is a schematic diagram of a port connection mode that meets the requirements of Mesh communication, provided in an embodiment of this application. The system includes an optical switch and four computing groups (first computing group, second computing group, third computing group, and fourth computing group). The optical switch configures four optical ports for each of the four computing groups. Mesh communication requirements refer to the distributed mesh topology formed by interconnecting computing groups. Therefore, each computing group needs to establish cross-group communication connections with the other three computing groups, requiring it to connect its own four optical ports to the optical ports of the three different computing groups.

[0045] For example, connecting the first optical port corresponding to the first computing group to the fifth optical port corresponding to the second computing group establishes an optical communication link between the first and second computing groups; connecting the second optical port corresponding to the first computing group to the ninth optical port corresponding to the third computing group establishes an optical communication link between the first and third computing groups; connecting the third optical port corresponding to the first computing group to the thirteenth optical port corresponding to the fourth computing group, and connecting the fourth optical port corresponding to the first computing group to the fourteenth optical port corresponding to the fourth computing group, establishes an optical communication link between the first and fourth computing groups. The process of establishing communication links between the second and third computing groups and the fourth computing group, as well as between the third and fourth computing groups, is similar to the above process and will not be described in detail here.

[0046] like Figure 1B As shown, Figure 1B This is a schematic diagram of a port connection mode that meets the requirements of Ring communication, provided in an embodiment of this application. The system includes an optical switch and four computing groups (first computing group, second computing group, third computing group, and fourth computing group). The optical switch configures four optical ports for each of the four computing groups. The Ring communication requirement refers to the implementation of a ring network topology by connecting the computing groups end-to-end. Therefore, each computing group needs to establish cross-group communication connections with two adjacent computing groups, requiring the allocation of its four optical ports to two different computing groups.

[0047] For example, connecting the first optical port corresponding to the first computing group to the fifteenth optical port corresponding to the fourth computing group, and connecting the second optical port corresponding to the first computing group to the sixteenth optical port corresponding to the fourth computing group, establishes an optical communication link between the first and fourth computing groups; connecting the first optical port corresponding to the first computing group to the fifth optical port corresponding to the second computing group, and connecting the fourth optical port corresponding to the first computing group to the sixth optical port corresponding to the fourth computing group, establishes an optical communication link between the first and second computing groups; connecting the seventh optical port corresponding to the second computing group to the ninth optical port corresponding to the third computing group, and connecting the eighth optical port corresponding to the second computing group to the tenth optical port corresponding to the third computing group, establishes an optical communication link between the second and third computing groups; connecting the eleventh optical port corresponding to the third computing group to the thirteenth optical port corresponding to the fourth computing group, and connecting the twelfth optical port corresponding to the third computing group to the fourteenth optical port corresponding to the fourth computing group, establishes an optical communication link between the third and fourth computing groups.

[0048] In the two examples above, each computing group has the same amount of available resources. When a subtask executed by a computing card in each computing group needs to communicate with multiple other computing groups, data transmission can be performed through a single optical port. However, in actual AI task execution, multiple computing cards executing subtasks are typically deployed in different computing groups. Since the number of computing cards in different computing groups is usually different, the number of optical ports in each computing group, i.e., the amount of available resources, is also usually different. Specifically, as shown below... Figure 1C As shown, Figure 1C This is a schematic diagram of another port connection mode provided in this application embodiment. The first computing group includes four optical ports (1-4), the second computing group includes three optical ports (5-7), the third computing group includes two optical ports (8, 9), and the fourth computing group includes only one optical port (10). When the computing cards in each computing group still need to communicate with multiple computing cards in the other three computing groups, the computing cards in the first computing group can send three data packets with different destinations through three of the four optical ports. However, the computing cards in the fourth computing group can only send three data packets with different destinations through one optical port. If the data volume of the three data packets sent by the two computing cards mentioned above is the same, the three data packets sent by the computing cards in the fourth computing group may compete for ports, leading to conflicts at the optical ports, resulting in slower data packet transmission rates, network congestion, and other issues that affect the overall communication performance of the system.

[0049] To address the communication conflicts that arise when multiple computing groups communicate with each other via optical switches, this application provides a computing system comprising N computing groups. Each computing group includes multiple computing cards, and the multiple computing cards within each computing group are connected via electrical switching equipment within that group. The multiple computing groups are connected to each other via optical switching equipment. The management node acquires the communication relationships between M subtasks executing an AI task and sorts the resource availability of each computing group participating in the execution of the subtasks, wherein the available resource availability of the (j-1)th computing group is greater than or equal to the available resource availability of the j-th computing group. X subtasks from the M subtasks are assigned to X computing cards in the i-th to N-th computing groups among the N computing groups. Y subtasks from the M subtasks that have communication relationships with the X subtasks are assigned to either the 1st to the (i-2nd)th or the (i-1st)th computing group among the N computing groups, and the remaining M-(X+Y) subtasks are assigned to the (i-1th)th computing group. The management node prioritizes allocating computing cards to some subtasks within the AI ​​task by selecting the computing group at the bottom of the list based on available resources (e.g., the number of available optical ports). Then, it selects the computing group at the top of the list to allocate computing cards to subtasks that communicate with the already assigned subtasks. Next, it selects the computing group with the second-to-last available resources to allocate computing cards to the remaining subtasks, and so on, until all subtasks are allocated. This effectively reduces communication between computing groups, thereby reducing conflicts, especially for computing groups with limited available resources, thus improving communication efficiency. The computing system provided in this application specifically enhances the system's communication performance. Figure 2 As shown.

[0050] Figure 2 This is a schematic diagram of a task system provided in an embodiment of this application. The system includes a client 100, an AI platform 110, and a computing system 200. The communication connection between the client 100, the AI ​​platform 110, and the computing system 200 can be a wired connection or a wireless connection. Wired connections include Ethernet, dedicated lines, cables, etc., while wireless connections include wireless local area networks (Wi-Fi), cellular networks, etc. It may also include the Internet, local area networks (LANs), etc., which simultaneously support wired and wireless connections. Of course, other connection types may also be included, which are not specifically limited in this application.

[0051] Client 100 is used for human-computer interaction and can be deployed on terminal devices, computing devices, or edge computing devices. Terminal devices include personal computers, smartphones, wearable devices, handheld processing devices, tablets, mobile laptops, augmented reality (AR) devices, virtual reality (VR) devices, smart conferencing devices, etc., without specific limitations. Computing devices can be bare metal servers (BMS), virtual machines, or containers. BMS refers to a general-purpose physical server, such as an ARM server or an x86 server; a virtual machine refers to a complete computer system simulated by software, possessing full hardware system functionality and running in a completely isolated environment. Any task that can be performed on a physical computer can also be performed in a virtual machine. When creating a virtual machine on a computing device, a portion of the physical machine's hard drive and memory capacity needs to be used as the virtual machine's hard drive and memory capacity. Each virtual machine has its own independent basic input / output system (BIOS), hard drive, and operating system, allowing it to be operated like a physical machine. A container is a portable software unit that combines an application and all its dependencies into a single software package. This package is not limited by the underlying host operating system, eliminating the need to build complex environments and simplifying the application development and deployment process. Edge computing devices are hardware devices located close to the data source or data user.

[0052] AI platform 110 receives AI task execution requests sent by client 100, generates corresponding AI tasks based on the requests, and distributes the AI ​​tasks to computing system 200. The AI ​​tasks include information such as parallel strategies and communication algorithms. The parallel strategies required for the AI ​​tasks include data parallelism (DP), tensor parallelism (TP), and pipeline parallelism (PP). Data parallelism refers to dividing the dataset and distributing it across multiple computing cards for training; tensor parallelism refers to distributing model parameters across multiple computing cards for training; and pipeline parallelism refers to dividing the model by network layers and distributing it across multiple computing cards for training. Of course, more types of parallel strategies may also be included, but this application does not specifically limit them. It should be noted that hybrid parallel strategies can also be used during AI task execution, such as a hybrid parallel strategy that uses both data parallelism and tensor parallelism, or other hybrid parallel strategies, but this application does not specifically limit them.

[0053] The ensemble communication strategy refers to the process of distributing the training task to multiple accelerator cards during the training of distributed AI models. The accelerator cards need to exchange data simultaneously so that each accelerator card can obtain the training results from other accelerator cards. This process is called ensemble communication, and the algorithm used in ensemble communication is called the communication algorithm. Communication algorithms can include broadcast, gather, scatter, all-gather, reduce, all-reduce, reduce-scatter, etc., and may also include other communication algorithms. This application does not specifically limit the specific communication algorithms.

[0054] AI Platform 110 is a system or service integrating various AI technologies and tools to help users (including but not limited to developers, enterprises, researchers, etc.) quickly build, train, deploy, and apply AI models. AI Platform 110 can provide users with a training environment for machine learning and deep learning models, supporting various algorithms and frameworks such as TensorFlow and PyTorch. It allows users to deploy trained models to production environments and provides API interfaces, enabling users to easily call AI models to achieve functional integration. For example, image recognition, natural language processing, and speech recognition functions can be directly called through the APIs provided by AI Platform 110. It can also provide users with automated machine learning (AutoML) tools to recommend optimal model parameter configurations. It may also include more functions; this application does not limit the specific functional types of the AI ​​platform. In specific implementations, AI Platform 110 can be deployed on one or more computing devices; this application does not make specific limitations. In other possible implementations, AI Platform 110 can also be replaced with other types of application platforms; this application does not make specific limitations in this regard.

[0055] The computing system 200 includes a scheduler 210, an optical switching device 220, and N computing groups 230 (the first computing group to the Nth computing group).

[0056] Scheduler 210 receives AI tasks from AI platform 110, along with the number of cards required to execute the AI ​​tasks, the parallel strategy, and the communication algorithm. It also generates communication relationships between M subtasks of the AI ​​task based on the number of cards required, the parallel strategy, and the communication algorithm, where M is an integer greater than 3. The scheduler further allocates the subtasks to multiple computing cards in multiple computing groups within the computing system.

[0057] In one possible implementation, the scheduler is equipped with a management node. Upon receiving the AI ​​task and the number of computing cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm, the scheduler determines the communication relationship between the M subtasks of the AI ​​task and allocates subtasks to the computing cards in multiple computing groups according to the principle of subtask allocation provided by the management node in this application. The specific process will be described in detail below and will not be repeated here.

[0058] The scheduler 210 is responsible for monitoring, managing, and maintaining the hardware resources of the computing system 200, including computing resources, storage resources, and network resources. Specific functions include, but are not limited to, hardware configuration and network configuration of computing cards in the computing group, fault detection and handling, and monitoring resource usage. The scheduler 210 ensures that all hardware resources are maintained in optimal condition to support the efficient operation of the computing system. Common schedulers 210 include Kubernetes and Apache Mesos, which are not specifically limited in this application.

[0059] Optical switching device 220 is a communication device that uses optical signals for data exchange. Specifically, optical switching device 220 can be an optical cross-connect (OXC) device. An OXC device has multiple micromirror arrays built-in. By adjusting the rotation angle of the micromirror arrays, the input signal can be deflected to different output ports, thereby realizing port mapping adjustment of the optical signal and thus enabling switching between different network topologies. In specific implementations, the OXC device can be a microelectromechanical system (MEMS) OXC, a liquid crystal OXC, a wavelength selective switch (WSS) OXC, a 3D MEMS OXC, an optical switch chip, etc., and this application does not specifically limit it. In one possible implementation, Figure 2 Taking a computing system including an optical switching device as an example, this application does not specifically limit the type or number of optical switching devices.

[0060] Optical switching device 220 is used to connect N computing groups in a computing system to realize inter-group data transmission between computing groups. Specifically, optical switching device 220 routes optical signals from one port to another by adjusting the mapping relationship between ports to realize cross-group communication between multiple computing groups.

[0061] The computing group 230 includes multiple computing cards 231 and electrical switching equipment 232. The computing cards in the same computing group communicate through the electrical switching equipment included in the computing group. The computing cards in multiple computing groups communicate with each other through optical switching equipment and electrical switching equipment. The number of computing groups is not specifically limited in this embodiment.

[0062] The computing card 231 is used to execute subtasks of AI tasks and can be a chip or a chip system. The chip may include a processor implemented using a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system-on-chip (SoC), or any combination thereof; this application does not impose specific limitations. It may also include processors or accelerator cards that have good performance when performing machine learning tasks and can improve the speed and efficiency of neural network training, such as graphics processing units (GPUs), neural network processing units (NPUs), data processing units (DPUs), tensor processing units (TPUs), etc.; this application does not impose specific limitations. In addition, the computing card 231 deploys chip plug-ins for a distributed training framework.

[0063] In one possible implementation, a management node is deployed in the computing card. Upon receiving the communication relationship between multiple subtasks of the AI ​​task determined by the scheduler, as well as the originally assigned subtasks, the management node redetermines the subtasks to be executed according to the principle of subtask allocation provided in this application. The specific process will be described in detail below and will not be repeated here.

[0064] Electrical switching device 232 is used to convert electrical signals into optical signals for input to optical switching devices. It is also used to convert received optical signals transmitted by optical switching devices into electrical signals for data transmission between multiple computing cards within a computing group, data processing, and traffic management operations. This application does not specifically limit the specific type or quantity of electrical switching devices. Electrical switching device 232 can be a switch, router, modem, repeater, etc. This application does not specifically limit the type or quantity of electrical switching devices.

[0065] In one specific implementation, such as Figure 2 As shown, taking the physical topology of optical switching device 220 connecting N computing groups as an example, the optical switching device only adjusts the optical switching links between computing groups and does not care about the topology within the computing group. The computing group can be a Layer 1 electrical switching, a Layer 2 electrical switching Fat-tree network, or any other topology that can achieve electrical interconnection between domains. When computing card a231 in the first computing group 230 communicates with computing card b231, the data packets sent by computing card a231 can be switched within the group via electrical switching device 232 in the first computing group 230, and then reach computing card b231; when computing card a231 communicates with computing card c231 in the (N-1)th computing group 230, the data packets sent by computing card a231 can first be switched within the group via electrical switching device 232 in the first computing group 230, and then reach optical switching device 220, and then be switched across groups via optical switching device 220, reaching the (N-1)th computing group 230, and then be switched within the group via electrical switching device 232 in the (N-1)th computing group 230, and finally reach computing card c231.

[0066] In one possible implementation, the above Figure 2 The computing system shown may also include more or fewer devices, and this application does not specifically limit this.

[0067] In one possible implementation, the AI ​​platform 110 and the computing system 200 can be deployed in a data center owned by the enterprise. The electrical switching equipment and optical switching equipment in the computing system 200 are deployed on corresponding switching equipment purchased by the enterprise. The AI ​​platform 110, the scheduler 210, and the computing cards in the computing system 200 are deployed in servers or server clusters purchased by the enterprise. The client 100 is a remote desktop connection tool or server management console software. Employees of the enterprise can remotely connect to the server or server cluster through the client 100 and use the scheduler or computing cards to perform sub-task allocation operations so that multiple computing cards can perform multiple sub-tasks of the AI ​​task.

[0068] In another possible implementation, the AI ​​platform 110 and computing system 200 can be deployed in a cloud environment. Specifically, a user can initiate a purchase request for AI cloud services through client 100, which includes the services corresponding to the computing system provided in this application. Specifically, after client 100 sends the purchase request to the cloud platform, the cloud platform can grant client 100 access to the aforementioned AI platform 110 and computing system 200, allowing the user to use multiple computing cards to execute multiple subtasks corresponding to the AI ​​task. During the execution of the AI ​​task, the scheduler or computing cards can be used to allocate rules for the subtasks.

[0069] It should be understood that the above application scenarios are for illustrative purposes only. The client 100, AI platform 110, and computing system 200 can be flexibly deployed according to actual business needs. They will not be listed here.

[0070] The possible system architecture of this application has been described above. The operation of the computing system will be described in detail below.

[0071] like Figure 3 As shown, Figure 3 This is a flowchart illustrating the operation of a computing system when a management node is deployed on a scheduler, as provided in an embodiment of this application.

[0072] S310: Client 100 sends an AI task request to AI platform 110.

[0073] S320: AI platform 110 generates AI tasks based on AI task requests, determines the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm, and sends the AI ​​task, the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm to scheduler 210.

[0074] AI task requests can be AI model training requests or inference requests, and can include dataset information, model configuration information, etc. Dataset information includes the location of the dataset used to train the model, such as a file path or database link. Model configuration information includes the AI ​​model's structural data, hyperparameters, and configuration information. Structural data includes data describing the model's structure, such as the number of layers and activation functions. Hyperparameters include information such as the model's learning rate, batch size, and number of training epochs. Configuration information includes data such as optimizer type, loss function type, regularization method, and flow pattern. The above examples are for illustrative purposes only, and this application does not specifically limit the content included in AI task requests.

[0075] Optionally, the AI ​​platform determines that all necessary data required for the AI ​​task is available based on the AI ​​task request, then generates the corresponding AI task, as well as the number of computing cards, parallel strategies, and communication algorithms required to execute the AI ​​task, and sends all of the above to the scheduler 210. It should be understood that the AI ​​platform may also send more quantity and type of information, such as the specifications of the computing cards, etc., which are not specifically limited in this application.

[0076] S330: The scheduler 210 generates the communication relationship between the M subtasks of the AI ​​task based on the received AI task, the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm.

[0077] In one possible implementation, the communication relationship between the M subtasks of the AI ​​task can be represented by multiple communication pairs. Each communication pair is represented by two Rank_IDs, each Rank_ID indicating one subtask. One Rank_ID indicates the source task, and the other Rank_ID indicates the target task. The communication pair indicates that there is a communication relationship between the corresponding two subtasks through the included two Rank_IDs. The Rank_ID can be a consecutive integer greater than or equal to zero. For example, the Rank_IDs corresponding to the M subtasks of the AI ​​task are Rank_0 to Rank_M-1, respectively. Alternatively, the Rank_ID can be represented in binary form. For example, the Rank_IDs corresponding to the 8 subtasks of the AI ​​task are 000, 001, 010, 011, 100, 101, 110, and 111, respectively. This application does not specifically limit the representation of the Rank_ID.

[0078] In another possible implementation, the communication relationship between the M subtasks of the AI ​​task can also be identified in other ways, which are not specifically limited in this application.

[0079] The scheduler 210 determines the communication relationships between the M subtasks of the AI ​​task based on the received AI task, the number of computing cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm. For example, if the scheduler determines that the AI ​​task requires 12 computing cards and the parallel strategy is a hybrid parallel strategy using both data parallelism and pipelined parallelism, with data parallelism DP=3, pipelined parallelism PP=4, and the 12 subtasks corresponding to 12 Rank_IDs (Rank_0~Rank_11), it can be determined that there are three rows in the Rank grid, each row including four computing cards. The four computing cards on each horizontal axis have PP communication, and the three computing cards on each vertical axis have DP communication. Each subtask has a communication relationship with the four subtasks. Specifically, multiple communication pairs include (Rank_0, Rank_1), (Rank_0, Rank_4), (Rank_8, Rank_0), (Rank_3, Rank_0), (Rank_1, Rank_2), and (Rank_1, Rank_5), etc., which will not be described in detail here.

[0080] The established communication relationships between subtasks can be used for the allocation of subsequent subtasks. This identifies multiple subtasks that have communication relationships with each subtask, thereby ensuring that multiple subtasks that have communication relationships with each subtask are allocated to the same computing group as much as possible.

[0081] S340: Scheduler 210 assigns X subtasks out of M subtasks to X computing cards in the i-th to N-th computing groups of N computing groups.

[0082] The N computing groups are sorted according to their resource quantity, and the available resource quantity of the (j-1)th computing group is greater than or equal to the available resource quantity of the jth computing group.

[0083] Before allocating subtasks to multiple computing cards, the scheduler sorts the N computing groups according to their available resource quantities. This ensures that the available resource quantity of the (j-1)th computing group is greater than or equal to that of the jth computing group. Here, the available resource quantity is the number of optical ports corresponding to each computing group, which is related to the number of computing cards included in each computing group. This application does not specifically limit this quantity.

[0084] In the above process, the scheduler can determine the computing group with the fewest optical ports based on the available resources of each computing group, and thus pre-allocate subtasks to the computing group and optimize the configuration of the communication objects of the computing group. This allows for centralized allocation of communication objects, avoids data transmission conflicts caused by a small number of optical ports, and improves communication performance.

[0085] The specific process by which scheduler 210 assigns X subtasks out of M subtasks to X computing cards in the i-th to N-th computing groups of N computing groups is as follows: The scheduler assigns X1 subtasks to X1 computing cards in the N-th computing group with the least available resources, and then executes step S350.

[0086] S350: Scheduler 210 determines Y subtasks that have communication relationships with X subtasks based on the communication relationships between the M subtasks of the AI ​​task.

[0087] Based on the communication relationships between the M subtasks of the AI ​​task determined in step S330, scheduler 210 determines Y subtasks that have communication relationships with the X subtasks. For example, after allocating X1 subtasks to X1 computing cards in the Nth computing group with the least available resources among the N computing groups, scheduler 210 determines Y1 subtasks that have communication relationships with the X1 subtasks and executes step S360.

[0088] S360: Scheduler 210 assigns Y subtasks to either the first to the (i-2)th computing group or the first to the (i-1)th computing group among the N computing groups, and assigns the remaining M-(X+Y) subtasks to the (i-1)th computing group.

[0089] After identifying the Y1 subtasks that have communication relationships with the X1 subtasks, the scheduler pre-determines to assign the Y1 subtasks to the first computing group among the N computing groups. However, the scheduler also needs to determine whether it is normal to assign all Y1 subtasks to the first computing group based on the number of computing cards in the first computing group and Y1.

[0090] In one possible implementation, when the number of computing cards in the first computing group is greater than or equal to Y1, Y1 subtasks are assigned to Y1 computing cards in the first computing group, wherein the first computing group is the computing group with the largest amount of available resources among the N computing groups in the system.

[0091] In another possible implementation, when the number of computing cards in the first computing group is less than Y1, Y1 subtasks are assigned to Y1 computing cards in the first computing group and the second computing group. Specifically, Y1 / 2 subtasks can be assigned to the first computing group and Y1 / 2 subtasks can be assigned to the second computing group, wherein the second computing group is the computing group with the second largest available resources among the N computing groups in the system.

[0092] When Y1 subtasks need to be assigned to Y1 computing cards in the first computing group and the second computing group, try to assign multiple subtasks corresponding to each of the X1 subtasks to the same computing group. For example, assign multiple y1 subtasks of Y1 subtasks that have a communication relationship with the x1th subtask of the X1 subtasks to the first computing group, and assign multiple y2 subtasks of Y1 subtasks that have a communication relationship with the x2th subtask of the X1 subtasks to the second computing group.

[0093] In the above process, the scheduler will assign multiple subtasks that have communication relationships with the assigned subtasks to the computing cards in the computing group with the largest amount of available resources. This ensures that multiple subtasks are assigned to the same computing group, and that the subtasks executed by the computing group with the fewest available resources communicate with as many computing groups as possible. This avoids data conflicts caused by a large amount of data to be transmitted at optical ports when the number of optical ports in the computing group is fixed or small. This solves the communication bottleneck in the computing system and improves communication performance.

[0094] Then, repeat steps S340, S350, and S360. For example, continue to allocate the X2 subtasks from the remaining subtasks to the X2 computing cards in the N-1th computing group with the second least available resources. Based on the communication relationship between the M subtasks of the AI ​​task, determine the Y2 subtasks that have a communication relationship with the X2 subtasks, and continue to allocate computing cards to the Y2 subtasks that have a communication relationship with the X2 computing cards in the first computing group and / or the second computing group, until only one computing group remains without any allocated subtasks, or without all allocated subtasks. Then, allocate the remaining subtasks to the computing cards in the remaining (i-1th)th computing group.

[0095] S370: Scheduler 210 sends the correspondence between M subtasks and M computing cards in N computing groups, the parallel strategy, and the communication algorithm to the computing cards, so that the computing cards can determine the communication domain according to the parallel strategy, determine the communication step according to the communication algorithm, and execute the corresponding subtask.

[0096] In one possible implementation, scheduler 210 sends a mapping file to the computing cards. This mapping file typically uses JavaScript Object Notation (JSON) format and another markup language for expressing data serialization (YAML). The mapping file includes the correspondence between M subtasks and the M computing cards in N computing groups, the parallelization strategy, and the communication algorithm. Alternatively, scheduler 210 may also send the correspondence between the M subtasks and the M computing cards in N computing groups, the parallelization strategy, and the communication algorithm to the computing cards via environment variables; this application does not specifically limit this approach.

[0097] The computing card, through a distributed training framework, divides multiple local communication domains based on the correspondence between the acquired M subtasks and the M computing cards in N computing groups, as well as the parallelism strategy. Each computing card can belong to a different communication domain; for example, in the case of a hybrid parallelism strategy combining DP and PP, the computing card can participate in both data parallelism and pipelined parallelism simultaneously. Furthermore, based on the correspondence between the M subtasks and the M computing cards in the N computing groups, and the communication algorithm, it determines which tasks each of the M subtasks communicates with in each communication step. After determining the communication domains and communication steps, the computing card, according to the correspondence between the M subtasks and the M computing cards in the N computing groups, determines at different times whether to execute its corresponding subtask to achieve the AI ​​task. Specifically, the compute card is used to call underlying operators and the communication library. The underlying operators can be provided by a heterogeneous computing architecture for neural networks (CANN). CANN generates instructions for each compute card to execute based on the communication domain, communication step, and other information determined by the distributed training framework. The communication library can be the Huawei collective communication library (HCCL). HCCL provides communication algorithm primitives such as AllReduce and AllGather, supporting communication between compute cards within and between compute cards in the same computing group. After the distributed training framework determines the communication domain and communication step to which the compute card belongs, it executes the corresponding subtask through CANN, while simultaneously using HCCL to achieve communication between different subtasks.

[0098] In one possible implementation, when the AI ​​task is large-scale distributed training, the computing card determines the communication domain and different communication steps through the distributed training framework, deploys the training data to multiple computing cards participating in the training, and each computing card performs subtasks through CANN, performs forward computation to generate loss values, performs backward computation to generate local gradients, and sends the local gradients to other computing cards through HCCL, and receives gradients sent by other computing cards. Then, CANN updates the model parameters through the aggregated gradients, and repeats the above steps until training is completed.

[0099] S380: After completing the AI ​​task, the computing card 231 sends the results of the AI ​​task to the client.

[0100] In another possible implementation, in addition to deploying the management node on the scheduler, the management node can also be deployed on the computing cards in the computing group, so that the computing cards can perform the subtask allocation related operations of the AI ​​task, which will be described in detail.

[0101] like Figure 4 As shown, Figure 4 This is a flowchart illustrating the operation of a computing system when a management node is deployed on a computing card, as provided in an embodiment of this application.

[0102] S410: Client 100 sends an AI task request to AI platform 110.

[0103] S420: AI platform 110 generates AI tasks based on AI task requests, including the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm, and sends the AI ​​task, the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm to scheduler 210.

[0104] AI task requests can be AI model training requests or inference requests, and can include dataset information, model configuration information, etc. Dataset information includes the location of the dataset used to train the model, such as a file path or database link. Model configuration information includes the AI ​​model's structural data, hyperparameters, and configuration information. Structural data includes data describing the model's structure, such as the number of layers and activation functions. Hyperparameters include information such as the model's learning rate, batch size, and number of training epochs. Configuration information includes data such as optimizer type, loss function type, regularization method, and flow pattern. The above examples are for illustrative purposes only, and this application does not specifically limit the content included in AI task requests.

[0105] Optionally, the AI ​​platform determines that all necessary data required for the AI ​​task is available based on the AI ​​task request, then generates the corresponding AI task, as well as the number of computing cards, parallel strategies, and communication algorithms required to execute the AI ​​task, and sends all of the above to the scheduler 210. It should be understood that the AI ​​platform may also send more quantity and type of information, such as the specifications of the computing cards, etc., which are not specifically limited in this application.

[0106] S430: The scheduler 210 generates the communication relationship between the M subtasks of the AI ​​task based on the received AI task, the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm. It then sends the communication relationship between the M subtasks of the AI ​​task to the computing cards and assigns the M subtasks of the AI ​​task to the M computing cards in the N computing groups.

[0107] The communication relationship between the M subtasks of an AI task can be represented by multiple communication pairs, or identified by other means, which is not specifically limited in this application.

[0108] The scheduler 210 determines the communication relationships between the M subtasks of the AI ​​task based on the received AI task, the number of computing cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm. For example, if the scheduler determines that the AI ​​task requires 64 computing cards, and that the communication algorithm is a halving-doubling (HD) algorithm, and that the 64 subtasks each have 64 Rank_IDs (Rank_0 to Rank_63), the scheduler determines the communication relationships between the 64 subtasks. Each subtask has communication relationships with six other subtasks. Specifically, the Rank_IDs of the communication objects for each subtask change in a power of 2 manner at different communication steps. For example, Rank_0 has communication relationships with Rank_1, Rank_2, Rank_4, Rank_8, Rank_16, and Rank_32 at different communication steps, and Rank_63 has communication relationships with Rank_62, Rank_61, Rank_59, Rank_55, Rank_47, and Rank_31 at different communication steps. These relationships are not detailed here.

[0109] Additionally, the scheduler, based on random algorithms, allocates the M subtasks of the AI ​​task to M computing cards in N computing groups. For example, if the AI ​​task includes 64 subtasks, with the first computing group providing 49 computing cards, the second providing 6, the third providing 4, the fourth providing 3, and the fifth providing 2, then subtasks 1 through 49 are allocated to the 49 computing cards in the first computing group; subtasks 50 through 55 are allocated to the 6 computing cards in the second computing group; subtasks 56 through 59 are allocated to the 4 computing cards in the third computing group; subtasks 60 through 62 are allocated to the 3 computing cards in the fourth computing group; and subtasks 63 through 64 are allocated to the 2 computing cards in the fifth computing group.

[0110] S440: Before executing the subtasks assigned by the scheduler, computing card 231 reassigns X subtasks out of the M subtasks to X computing cards in the i-th to N-th computing groups of the N computing groups.

[0111] The N computing groups are sorted according to their resource quantity, and the available resource quantity of the (j-1)th computing group is greater than or equal to the available resource quantity of the jth computing group.

[0112] Before reallocating subtasks, the computing card can sort the N computing groups according to their available resource quantities based on the available resource quantities of each of the N computing groups sent by the scheduler. This will determine that the available resource quantity of the (j-1)th computing group is greater than or equal to the available resource quantity of the jth computing group. Here, the available resource quantity is the number of optical ports corresponding to each computing group, which is related to the number of computing cards included in each computing group. This application does not make specific limitations on this.

[0113] The specific process by which computing card 231 assigns X subtasks out of M subtasks to X computing cards in the i-th to N-th computing groups of N computing groups is as follows: The computing card assigns X1 subtasks to X1 computing cards in the N-th computing group with the least available resources, and then executes step S350.

[0114] S450: Computing card 231 determines Y subtasks that have communication relationships with X subtasks based on the communication relationships between M subtasks of the AI ​​task.

[0115] Based on the communication relationships between the M subtasks of the AI ​​task determined in step S330, computing card 231 determines the Y subtasks that have communication relationships with the X subtasks. For example, after assigning the X1 subtasks to the X1 computing cards in the Nth computing group with the least available resources among the N computing groups, the Y1 subtasks that have communication relationships with the X1 subtasks are determined, and step S360 is executed.

[0116] S460: Computing card 231 assigns Y subtasks to either the first to the (i-2)th computing group or the first to the (i-1)th computing group among N computing groups, and assigns the remaining M-(X+Y) subtasks to the (i-1)th computing group.

[0117] After identifying the Y1 subtasks that have communication relationships with the X1 subtasks, the computing card pre-determines to assign the Y1 subtasks to the first computing group among the N computing groups. However, the computing card also needs to determine whether it is normal to assign all Y1 subtasks to the first computing group based on the number of computing cards in the first computing group and Y1.

[0118] In one possible implementation, when the number of computing cards in the first computing group is greater than or equal to Y1, Y1 subtasks are assigned to Y1 computing cards in the first computing group, wherein the first computing group is the computing group with the largest amount of available resources among the N computing groups in the system.

[0119] In another possible implementation, when the number of computing cards in the first computing group is less than Y1, Y1 subtasks are allocated to Y1 computing cards in the first computing group and the second computing group. Specifically, Y1 / 2 subtasks can be allocated to the first computing group and Y1 / 2 subtasks can be allocated to the second computing group. The second computing group is the computing group with the second largest available resources among the N computing groups in the system. In one possible implementation, the computing cards can also allocate Y1 subtasks to the first computing group and the second computing group according to different allocation ratios. This application does not specifically limit this.

[0120] Then, repeat steps S340, S350, and S360. For example, continue to assign the X2 subtasks from the remaining subtasks to the X2 computing cards in the N-1th computing group with the second least available resources, and continue to assign computing cards to the Y2 subtasks that have communication relationships with the X2 computing cards in the first computing group and / or the second computing group, until only one computing group remains without any subtasks assigned. Then, assign the remaining subtasks to the computing cards in the remaining i-1th computing group.

[0121] S470: Computing card 231 determines the communication domain and communication step based on the correspondence between the modified M subtasks and the M computing cards in the N computing groups, the parallel strategy, and the communication algorithm, and executes the corresponding subtask.

[0122] In the modified correspondence between the M subtasks and the M computing cards in the N computing groups, the subtasks executed by each computing card may be the same as or different from the subtasks assigned by the scheduler. This application does not make specific limitations on this.

[0123] Computation card 231 determines the communication domain and different communication steps based on the correspondence between the modified M subtasks and the M computing cards in the N computing groups, as well as the parallel strategy. Then, the computing card executes the corresponding subtasks via CANN and simultaneously achieves communication between different subtasks via HCCL to complete the AI ​​task.

[0124] S480: After completing the AI ​​task, the computing card 231 sends the results of the AI ​​task to the client.

[0125] In summary, in a computing system provided by this application embodiment, when multiple computing cards required for AI task execution are deployed across multiple computing groups, the management node, based on the communication relationships between multiple subtasks of the AI ​​task and the available resources provided by the multiple computing groups, prioritizes assigning subtasks to computing cards in the computing group with the least available resources, ranked last, according to the order of available resources. Then, it assigns multiple subtasks that communicate with the assigned subtask to multiple computing cards in the computing group with the largest available resources, ranked first. Next, it selects the computing group with the second-to-last available resources to assign computing cards to the remaining subtasks, continuing until all subtasks are assigned. This ensures that subtasks executed by computing groups with fewer available resources communicate with one or fewer computing groups, effectively reducing the communication volume between computing groups and thus reducing communication conflicts caused by the limited number of optical ports, thereby improving communication performance. For computing groups with fewer optical ports, this is even more effective in reducing communication conflicts and improving communication efficiency. Furthermore, the above management node operation only reallocates the subtasks executed by each computing card and does not require modification of the underlying algorithm logic, making the operation convenient.

[0126] In one possible implementation, Figure 5 This is a schematic diagram of subtask allocation under a hybrid parallel strategy provided in an embodiment of this application, as shown below. Figure 5 As shown. With a management node deployed in the scheduler, the scheduler receives AI tasks from the AI ​​platform, specifying the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm. Based on the required number of cards (12) for executing the AI ​​task, the scheduler selects 12 computing cards from multiple computing groups. For example, it selects eight computing cards from electrical switching group A, two from electrical switching group B, and two from electrical switching group C. Assigning Rank_0 to Rank_11 to the 12 subtasks within the AI ​​task, the scheduler generates the communication relationships between the 12 subtasks based on the received AI task, the required number of cards (12), and a hybrid parallel strategy using both data parallelism and pipelined parallelism.

[0127] Specifically, in the pipeline parallel strategy, different layers of the model are assigned to different devices, each device processes a different stage, and data is transferred between stages in micro-batches. Pipeline stages need to transfer activation values ​​or gradients across DP groups. The data parallel strategy, on the other hand, replicates the same model to multiple devices, each device processes a different subset of data, and then synchronizes gradients. Gradient synchronization needs to be performed between all nodes within the DP group. With PP=4 and DP=3, there is PP communication between the four nodes on each horizontal axis and DP communication between the three nodes on each vertical axis, as detailed below. Figure 5The grid shown is 3 rows and 4 columns. Each row corresponds to a DP group and each column corresponds to a PP stage. It is determined that each subtask has a communication relationship with the four surrounding subtasks. For example, the first subtask has a communication relationship with the second, fifth, ninth and fourth subtasks, the second subtask has a communication relationship with the first, third, sixth and tenth subtasks, and so on. It will not be explained in detail here.

[0128] The scheduler sorts multiple computing groups based on the available resources corresponding to computing groups A, B, and C. For example, if the number of optical ports in computing group A is greater than the number of optical ports in computing group B, and the number of optical ports in computing group B is equal to the number of optical ports in computing group C, the sorting is determined to be A>B=C.

[0129] The scheduler assigns two subtasks out of the 12 subtasks (e.g., subtask 1 and subtask 2) to two computing cards in computing group B. Then, based on the communication relationships between the 12 subtasks, it identifies several subtasks that communicate with subtask 1 and subtask 2: subtask 2, subtask 3, subtask 4, subtask 5, subtask 6, subtask 9, and subtask 10, a total of six subtasks. If the number of computing cards in computing group A is greater than 6, the above six subtasks are assigned to the six computing cards in computing group A. Next, from the remaining computing groups, computing group C with the fewest optical ports is selected, and subtask 11 and subtask 12 are assigned to two computing cards in computing group C. Finally, the remaining two subtasks, for example, subtask 7 and subtask 8, are assigned to the remaining two computing cards in computing group A. Figure 5 In (B) of this application, the scheduler determines the subtasks to be executed by the computing cards in each computing group according to the subtask allocation rules provided in this application, and represents them with the corresponding Rank_ID.

[0130] The above example is only one possible implementation. In another possible implementation, the scheduler may assign the seventh and eighth subtasks to two computing cards in computing group C, and the remaining eleventh and twelfth tasks to the remaining two computing cards in computing group A, in addition to assigning the six subtasks to the six computing cards in computing group A.

[0131] In determining how to allocate each subtask to multiple computing cards in multiple computing groups, the scheduler sends the allocation information to the distributed training framework. This allows the distributed training framework to re-divide the communication domain and communication steps based on the newly allocated Rank_ID and the mapping relationship between the computing cards, as well as the corresponding parallel strategy. The underlying heterogeneous computing architecture (CANN) ultimately generates the corresponding communication relationship based on the corresponding set of communication domains and algorithm characteristics, so that multiple computing cards can execute the corresponding subtasks.

[0132] Figure 5 In (A), the subtasks to be executed by the computing cards in each computing group are determined by the scheduler through random algorithms and other allocation rules, and are also represented by their corresponding Rank_IDs. Figure 5 (A) and Figure 5 In (B), it can be determined that a management node is deployed in the scheduler to allocate subtasks. Subtasks can be allocated to computing group B, which has the fewest optical ports, first, and then multiple subtasks with communication relationships can be allocated to computing cards in computing group A, which has the most optical ports. Subtasks can then be allocated to computing group C, which also has the fewest optical ports. This can ensure that the subtasks that computing group B and computing group C need to communicate with are concentrated in computing group A, so as to avoid data conflicts caused by insufficient optical ports when computing group B and computing group C need to communicate with multiple computing groups, thereby improving communication performance.

[0133] In one possible implementation, Figure 6 This is a schematic diagram of subtask allocation under another hybrid parallel strategy provided in this application embodiment, as shown below. Figure 6 As shown, with a management node deployed in the scheduler, the scheduler receives AI tasks from the AI ​​platform, along with the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm. Based on the required number of cards (12) for executing the AI ​​task, the scheduler selects 12 computing cards from multiple computing groups to allocate to the subtasks of the AI ​​task. For example, it selects five computing cards from electrical switching group A, five from electrical switching group B, and two from electrical switching group C. Assigning Rank_0 to Rank_11 to the 12 subtasks of the AI ​​task, the scheduler generates the communication relationships between the 12 subtasks based on the received AI task, the required number of cards (12), and the hybrid parallel strategy using both data parallelism and pipelined parallelism.

[0134] Specifically, with PP=4 and DP=3, it is determined that each subtask has a communication relationship with the four surrounding subtasks. For example, the first subtask has a communication relationship with the second, fifth, ninth, and fourth subtasks, and the second subtask has a communication relationship with the first, third, sixth, and tenth subtasks, and so on. These relationships will not be explained in detail here.

[0135] The scheduler sorts multiple computing groups based on the available resources corresponding to computing groups A, B, and C. For example, if the number of optical ports in computing group A is equal to the number of optical ports in computing group B, and the number of optical ports in computing group B is greater than the number of optical ports in computing group C, the sorting is determined to be A = B > C.

[0136] The scheduler assigns two subtasks out of the 12 subtasks (e.g., subtask 1 and subtask 2) to two computing cards in computing group C. Then, based on the communication relationships between the 12 subtasks, it identifies several subtasks that communicate with subtask 1 and subtask 2: subtask 2, subtask 3, subtask 4, subtask 5, subtask 6, subtask 9, and subtask 10, a total of six subtasks. If the number of computing cards in computing group A is greater than 6 (5), it is impossible to assign all six subtasks to the six computing cards in computing group A. Therefore, another computing group B with a larger number of optical ports is selected, and the six subtasks are assigned to the six computing cards in both computing group A and computing group B. For example, subtasks 4, 5, and 9, which communicate with subtask 1, are assigned to three computing cards in computing group A, and subtasks 3, 6, and 10, which communicate with subtask 2, are assigned to three computing cards in computing group B.

[0137] Furthermore, since the twelfth subtask communicates with the fourth, eighth, ninth, and eleventh subtasks, and the eighth subtask communicates with the fourth, fifth, seventh, and twelfth subtasks, and since the fourth, fifth, and ninth subtasks have already been assigned to the three computing cards in switch group A, the eighth and twelfth subtasks can be assigned to the remaining two computing cards in switch group A. Finally, the remaining two seventh and twelfth subtasks are assigned to the remaining two computing cards in switch group B. Similarly, Figure 6 In this context, the subtasks executed by the computing cards in each computing group are represented by their corresponding Rank_ID.

[0138] In one possible implementation, such as Figure 7 As shown, Figure 7This is a schematic diagram of subtask allocation under the HD algorithm provided in this application embodiment. When a management node is deployed in the computing card, the scheduler generates the communication relationships between the 64 subtasks of the AI ​​task based on the AI ​​task issued by the AI ​​platform, the number of cards required to execute the AI ​​task (64), and the HD algorithm. Specifically, the HD algorithm determines the six subtasks with communication relationships for each subtask according to the rule of powers of 2. Specifically, the six subtasks with communication relationships for the first subtask can be: the second subtask separated from the first subtask by one subtask; the third subtask separated from the first subtask by two subtasks; the fifth subtask separated from the first subtask by four subtasks; the ninth subtask separated from the first subtask by eight subtasks; the seventeenth subtask separated from the first subtask by sixteen subtasks; and the thirty-third subtask separated from the first subtask by thirty-two subtasks. The communication objects of other subtasks can also be determined using the above method and will not be listed individually. The scheduler sends the determined communication relationships between the 64 subtasks to the computing cards and assigns the 64 subtasks to the computing cards in the five computing groups according to the sequential allocation rules. For example, subtasks 1 to 49 are assigned to the computing network cards in computing group A, subtasks 50 to 55 are assigned to the computing network cards in computing group B, subtasks 56 to 59 are assigned to the computing network cards in computing group C, subtasks 60 to 62 are assigned to the computing network cards in computing group D, and subtasks 63 and 64 are assigned to the computing network cards in computing group E.

[0139] Before executing the subtasks assigned by the scheduler, the computing card sorts the multiple computing groups according to the available resources of each computing group, determining computing groups A>B>C>D>E. The first and second subtasks are then reassigned to computing group E, which has the scarcest available resources. Based on the communication relationships between the 64 subtasks, subtasks communicating with the first and second subtasks are assigned to computing group A, which has the most available resources. The seventh, eighth, and eleventh subtasks are assigned to computing group D, which has the scarcest available resources among the remaining computing groups. Multiple subtasks that have communication relationships with each other are assigned to computing group A, which has the most available resources. Then, the fourteenth, twentieth, twenty-first, and twenty-second subtasks are assigned to computing group C, which has the scarcest available resources among the remaining computing groups. Multiple subtasks that have communication relationships with the fourteenth, twentieth, twenty-first, and twenty-second subtasks are assigned to computing group A, which has the most available resources. After assigning the thirty-first to thirty-fifth subtasks and the forty-first subtask to computing group B, the remaining subtasks are assigned to the remaining computing cards in computing group A.

[0140] The computing card performs corresponding operations based on the reassigned subtasks, ensuring that computing group E does not need to communicate with the other four computing groups as shown in the initial subtask allocation diagram. It only needs to communicate with computing group A. Computing group E can avoid data conflicts caused by the limited number of optical ports that cannot communicate with multiple computing groups simultaneously. Similarly, computing groups B, C, and D can also avoid data conflicts caused by the limited number of optical ports and the need for inter-group communication with multiple computing groups, thereby improving the system's communication performance.

[0141] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. The computing device can be used as... Figure 2 The scheduler in the computing system shown, such as Figure 8 As shown, the computing device 800 includes: a bus 801, a processor 802, a memory 803, an accelerator card 804, and a communication interface 805. The processor 802, the memory 803, and the communication interface 805 communicate with each other via the bus 801. The computing device 800 may be a server. It should be understood that this application does not limit the number of processors, memory, and accelerator cards in the computing device 800.

[0142] Memory 803 is used to store instructions, and processor 802 is used to execute the instructions stored in memory 803. Memory 803 stores program code, and processor 802 can call the program code stored in memory 803 to execute the operations performed by the management node during subtask allocation.

[0143] The 801 bus can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The Unified Bus is also known as the Lingqu bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus is represented by only one line, but this does not mean that there is only one bus or one type of bus. Bus 801 may include a path for transmitting information between various components of computing device 800 (e.g., processor 802, memory 803, communication interface 805). The unified bus may also be called the Lingqu bus.

[0144] The processor 802 may include any one or more of the following computing devices: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP) or digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.

[0145] Memory 803 may include volatile memory, such as random access memory (RAM). Memory 803 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 803 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0146] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 803, or two or more types of storage media can be configured to realize the function of memory 803. This application does not limit this.

[0147] The communication interface 805 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 800 and other devices or communication networks.

[0148] Furthermore, the accelerator card 804 can be implemented through computing devices such as DPU, NPU, GPU, XPU, SoC, or computing units such as AI chips. The accelerator card 804 is used to implement sub-tasks of AI tasks. The accelerator card 804 includes a processing unit and a power supply circuit. The power supply circuit is used to supply power to the processing unit, enabling the processing unit to implement various sub-tasks in the AI ​​task.

[0149] It should be understood that the accelerator card 804 can be understood as the processor used to perform subtasks of AI tasks as mentioned above, while the processor 802 refers to the processing core in the computing device 800, such as the CPU. The two are different concepts.

[0150] This application also provides a computing device cluster, such as Figure 9 As shown, Figure 9 This is an example diagram of a computing device cluster provided in an embodiment of this application. The cluster may include at least one Figure 8 The computing device of the embodiment is a cluster of computing devices used to deploy the computing cards described above.

[0151] This application embodiment also provides a chip, which includes a processing unit and a power supply circuit. The power supply circuit is used to supply power to the processing unit, so that the processing unit performs subtask allocation operations performed by the management node in the computing system.

[0152] This application also provides a computer-readable storage medium storing instructions that, when executed on a management node, Figure 3 , Figure 4 The process of subtask allocation operations executed by the management node in the computing system shown is thus realized.

[0153] This application also provides a computer program product that, when running on a smart network card, Figure 3 , Figure 4 The process of subtask allocation operations executed by the management node in the computing system shown is thus realized.

[0154] When computer program instructions are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0155] Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or a data storage node such as a server or data center that contains at least one set of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media).

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A computing system, characterized in that, The computing system includes N computing groups, each computing group includes multiple computing cards, the multiple computing cards in each computing group are electrically connected, and the multiple computing groups are connected to each other through optical switching equipment. The management node obtains the communication relationships between the M subtasks that execute an AI task; The management node assigns X subtasks from the M subtasks to X computing cards in the i-th to N-th computing groups of the N computing groups, wherein the N computing groups are sorted according to resource quantity, and the available resource quantity of the (j-1)-th computing group is greater than or equal to the available resource quantity of the j-th computing group. The management node assigns Y subtasks that have communication relationships with the X subtasks out of the M subtasks to the first to the (i-2)th computing group or the first to the (i-1)th computing group among the N computing groups, and assigns the remaining M-(X+Y) subtasks to the (i-1)th computing group.

2. The system according to claim 1, characterized in that, The management node assigns X subtasks from the M subtasks to X computing cards in the i-th to N-th computing groups of the N computing groups, and assigns Y subtasks from the M subtasks that have communication relationships with the X subtasks to the 1-i-2 or 1-i-1 computing groups of the N computing groups, and assigns the remaining M-(X+Y) subtasks to the i-1 computing group, including: Assign X1 subtasks to the Nth computing group among the N computing groups; Select the first computing group among the N computing groups to allocate computing cards to the Y1 subtasks that have communication relationships with the X1 subtasks; When the number of computing cards in the first computing group is greater than or equal to Y1, the Y1 subtasks are assigned to the Y1 computing cards in the first computing group. When the number of computing cards in the first computing group is less than Y1, the Y1 subtasks are assigned to the Y1 computing cards in the first computing group and the second computing group. Continue to select the (N-1)th computing group to allocate X2 computing cards to the X2 subtasks in the remaining subtasks, and continue to allocate computing cards to the Y2 subtasks that have communication relationships with the X2 computing cards in the first or second computing group, until all the remaining subtasks are allocated to computing cards in the (i-1)th computing group.

3. The system according to claim 2, characterized in that, When the number of computing cards in the first computing group is less than Y1, the allocation of the Y1 subtasks to the Y1 computing cards in the first computing group and the second computing group includes: The management node assigns the y1 subtasks that have a communication relationship with the xth subtask in the X1 subtasks to the y1 computing cards in the first computing group; The first computing group is selected to allocate computing cards to the y2 subtasks that have a communication relationship with the (x+1)th subtask in the X1 subtasks; When the number of remaining computing cards in the first computing group is greater than or equal to y2, the y2 subtasks are assigned to the y2 computing cards in the first computing group. When the number of remaining computing cards in the first computing group is less than y2, the y2 subtasks are assigned to the y2 computing cards in the second computing group.

4. The system according to any one of claims 1-3, characterized in that, The available resources for each compute group are the number of optical ports corresponding to that compute group.

5. The system according to any one of claims 1-4, characterized in that, The management node is the scheduler of the computing system, and the management node obtains the communication relationships between the M subtasks executing an AI task, including: The scheduler receives the AI ​​task, the number of cards required to execute the AI ​​task, the parallel strategy, and the communication algorithm. The scheduler generates the communication relationships between the M subtasks based on the number of cards, the parallel strategy, and the communication algorithm.

6. The system according to any one of claims 1-4, characterized in that, The management node is deployed on each computing card; The management node obtains the communication relationships between the M subtasks that execute an AI task, including: Obtain the communication relationships between M subtasks from the scheduler of the computing system.

7. A method for controlling communication relationships between computing groups, characterized in that, The method is applied to a management node in a computing system, which includes N computing groups, each computing group including multiple computing cards, the multiple computing cards within each computing group being electrically connected, and the multiple computing groups being connected through an optical switching device. The method includes: Obtain the communication relationships between M subtasks that perform an AI task; X subtasks from the M subtasks are assigned to X computing cards in the i-th to N-th computing groups of the N computing groups, wherein the N computing groups are sorted according to the amount of resources, and the available resources of the (j-1)-th computing group are greater than or equal to the available resources of the j-th computing group. The Y subtasks that have communication relationships with the X subtasks among the M subtasks are assigned to the first to the (i-2)th computing group or the first to the (i-1)th computing group among the N computing groups, and the remaining M-(X+Y) subtasks are assigned to the (i-1)th computing group.

8. The method according to claim 1, characterized in that, The step of assigning Y subtasks from the M subtasks that have communication relationships with the X subtasks to the first to the (i-2)th computing groups or the first to the (i-1)th computing groups in the N computing groups, and assigning the remaining M-(X+Y) subtasks to the (i-1)th computing group, includes: Assign X1 subtasks to the Nth computing group among the N computing groups; Select the first computing group among the N computing groups to allocate computing cards to the Y1 subtasks that have communication relationships with the X1 subtasks; When the number of computing cards in the first computing group is greater than or equal to Y1, the Y1 subtasks are assigned to the Y1 computing cards in the first computing group. When the number of computing cards in the first computing group is less than Y1, the Y1 subtasks are assigned to the Y1 computing cards in the first computing group and the second computing group. Continue to select the (N-1)th computing group to allocate X2 computing cards to the X2 subtasks in the remaining subtasks, and continue to allocate computing cards to the Y2 subtasks that have communication relationships with the X2 computing cards in the first or second computing group, until all the remaining subtasks are allocated to computing cards in the (i-1)th computing group.

9. The method according to claim 8, characterized in that, When the number of computing cards in the first computing group is less than Y1, the allocation of the Y1 subtasks to the Y1 computing cards in the first computing group and the second computing group includes: The y1 subtasks that have a communication relationship with the xth subtask in the X1 subtasks are assigned to the y1 computing cards in the first computing group; The first computing group is selected to allocate computing cards to the y2 subtasks that have a communication relationship with the (x+1)th subtask in the X1 subtasks; When the number of remaining computing cards in the first computing group is greater than or equal to y2, the y2 subtasks are assigned to the y2 computing cards in the first computing group. When the number of remaining computing cards in the first computing group is less than y2, the y2 subtasks are assigned to the y2 computing cards in the second computing group.

10. The method according to any one of claims 7-9, characterized in that, The available resources for each compute group are the number of optical ports corresponding to that compute group.

11. The method according to any one of claims 7-10, characterized in that, The computing system also includes a scheduler. When the management node is deployed on the scheduler, obtaining the communication relationships between the M subtasks executing an AI task includes: Receive the AI ​​task and the number of cards, parallel strategy, and communication algorithm required to execute the AI ​​task; The communication relationships between the M subtasks are generated based on the number of cards, the parallel strategy, and the communication algorithm.

12. The method according to any one of claims 7-10, characterized in that, When the management node is deployed on the computing card, obtaining the communication relationships between the M subtasks executing an AI task includes: Obtain the communication relationships between the M subtasks that execute an AI task from the scheduler of the computing system.

13. A computer program product containing instructions, characterized in that, When the instruction is executed by the management node, the management node performs the method as described in any one of claims 7-12.

14. A computer-readable storage medium, characterized in that, Includes computer program instructions that, when executed by a management node, cause the management node to implement the method as described in any one of claims 7-12.

Images