Resource configuration method, management node and related devices

Abstract

A resource configuration method, a management node and related devices relate to the technical field of artificial intelligence. The management node obtains traffic information, which is used to indicate communication traffic generated when multiple DCs perform AI tasks in parallel. Each data center includes a computing node, and the computing nodes between multiple DCs are interconnected through an optical switching network. The management node determines the communication bandwidth requirement between the multiple DCs according to the traffic information, and adjusts the bandwidth configuration between the multiple DCs according to the communication bandwidth requirement. In this way, the multiple DCs have sufficient bandwidth resources to transmit the cross-DC communication traffic between the multiple computing nodes, which can improve the communication efficiency between the multiple computing nodes, thereby improving the overall efficiency of the computing nodes in the multiple DCs performing AI tasks, and avoiding the idle state of part of the bandwidth resources for a long time due to the static allocation of fixed-size bandwidth resources to the DCs, thereby improving the overall bandwidth utilization of the optical switching network.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a resource allocation method, management node and related equipment. Background Technology

[0002] With the development of artificial intelligence (AI) technology, the parameter scale of AI models is also gradually increasing; for example, the parameter scale of a single AI model can reach hundreds of billions. Correspondingly, a single data center (DC) may be insufficient to meet the cluster scale requirements for AI model training or inference. In practical applications, computing nodes in multiple DCs can typically be used to jointly execute the same AI task, which can specifically be a task of training an AI model or a task of using an AI model for inference.

[0003] Different data centers (DCs) can be interconnected via optical switching networks. An optical switching network is a network that uses optical switching technology for data transmission; for example, it could be an optical transport network (OTN). Different DCs can also establish point-to-point connections via optical switching networks. For instance, ... Figure 1 As shown, multiple DCs can achieve full mesh connectivity based on an OTN network. Furthermore, the electrical switching nodes in the spine layer of DC1 support a total communication bandwidth of 32*400Gbps (gigabits per second). DC1 will then divide this bandwidth into two parts to connect to DC2 and DC3 respectively; that is, DC1 will interconnect with DC2 and DC3 based on a bandwidth of 16*400Gbps each.

[0004] However, when using multiple data centers to jointly execute AI tasks, there is often a problem of low execution efficiency for the AI ​​tasks. Summary of the Invention

[0005] This application provides a resource allocation method to improve the execution efficiency of AI tasks. Furthermore, this application also provides a corresponding management node, a computer-readable storage medium, and a computer program product.

[0006] Firstly, this application provides a resource scheduling method that can be executed by a corresponding management node. Specifically, the management node acquires traffic information, which indicates the communication traffic generated when multiple data centers (DCs) execute AI (artificial intelligence) tasks in parallel. Each data center includes computing nodes (such as GPUs, NPUs, etc.), and the computing nodes between multiple DCs are interconnected through an optical switching network. For example, computing nodes within different DCs can be interconnected through multiple optical switches. Then, based on the traffic information, the management node determines the communication bandwidth requirements between the multiple DCs and adjusts the bandwidth configuration between the multiple DCs accordingly. For example, the management node can adjust the bandwidth configuration by adjusting the interconnection status between ports of different optical switches in the optical switching network.

[0007] Thus, when utilizing computing nodes within multiple data centers (DCs) to execute AI tasks, the management node determines the bandwidth requirements between different DCs based on the communication traffic between the computing nodes. It then adjusts the bandwidth configuration across multiple DCs to meet these requirements. This ensures sufficient bandwidth resources across the DCs to transmit cross-DC communication traffic between computing nodes, thereby improving communication efficiency and ultimately enhancing the overall efficiency of AI task execution within the DCs. Furthermore, the dynamic adjustment of bandwidth configurations across DCs avoids the problem of statically allocating fixed bandwidth resources to DCs, preventing some bandwidth resources from remaining idle for extended periods. This improves the overall bandwidth utilization of the optical switching network.

[0008] In one possible implementation, the traffic information includes a traffic matrix and a mapping relationship. The rows and columns of the traffic matrix correspond to computing nodes within multiple data centers (DCs), and the element values ​​in the traffic matrix indicate the communication traffic volume between different computing nodes within a DC. The mapping relationship is a mapping between the identifiers of the computing nodes within the multiple DCs and the identifiers of the multiple DCs. Thus, the management node determines the communication traffic volume between different computing nodes based on the traffic matrix and distinguishes the communication traffic between computing nodes within different DCs based on the mapping relationship, enabling the management node to further determine the communication bandwidth requirements between different DCs.

[0009] In one possible implementation, taking multiple data centers (DCs), including a first DC and a second DC, as an example, when the management node determines the communication bandwidth requirements between the multiple DCs based on traffic information, it can specifically determine multiple communication traffic flows between computing nodes in the first DC and computing nodes in the second DC based on a traffic matrix and mapping relationships, and aggregate these multiple communication traffic flows to obtain the communication bandwidth requirements between the first DC and the second DC. In this way, the management node can determine the overall communication bandwidth requirements between the first DC and the second DC by aggregating the communication traffic between multiple computing nodes in these two DCs. Similarly, the management node can also determine the communication bandwidth requirements between the remaining DCs in the same manner.

[0010] In one possible implementation, computing nodes within multiple data centers (DCs) execute AI tasks based on configuration files. The configuration files indicate subtasks within the AI ​​tasks executed by the computing nodes within the DCs, and include a mapping between the identifiers of the computing nodes within the multiple DCs and the identifiers of the multiple DCs.

[0011] In one possible implementation, the multiple data centers (DCs) include a first DC, a second DC, and a third DC. When the management node adjusts the bandwidth configuration among the multiple DCs based on the communication bandwidth requirements, specifically, it may allocate a portion of the bandwidth resources between the second and third DCs to the first DC. This portion of bandwidth resources is used to transmit communication traffic between computing nodes within the first DC and computing nodes within the second DC. In this way, the management node 100 dynamically adjusts bandwidth resources within the scope of the multiple DCs participating in the execution of the AI ​​task.

[0012] In one possible implementation, the multiple data centers (DCs) include a first DC and a second DC. The second DC is also interconnected with a fourth DC that is not involved in performing the AI ​​task via an optical switching network. When the management node adjusts the bandwidth configuration among the multiple DCs based on the communication bandwidth requirements between them, specifically, it may allocate a portion of the bandwidth resources between the second DC and the fourth DC to the first DC based on the communication bandwidth requirements between the first DC and the second DC. This portion of the bandwidth resources is used to transmit communication traffic between computing nodes within the first DC and computing nodes within the second DC. In this way, the management node reallocates a portion of the bandwidth resources of the DCs not involved in performing the AI ​​task to the multiple DCs involved in performing the AI ​​task, ensuring that the bandwidth resources among the multiple DCs involved in performing the AI ​​task can meet their communication bandwidth requirements.

[0013] In one possible implementation, the optical switching network may specifically include an OTN (Optical Transport Network). Accordingly, the optical switching nodes in the optical switching network may specifically be OTN devices.

[0014] Secondly, this application provides a management node, which includes: an acquisition module for acquiring traffic information, the traffic information being used to indicate the communication traffic generated when multiple DCs (data centers) execute AI (artificial intelligence) tasks in parallel, wherein each data center includes a computing node, and the computing nodes between the multiple DCs are interconnected through an optical switching network; a determination module for determining the communication bandwidth requirements between the multiple DCs based on the traffic information; and an adjustment module for adjusting the bandwidth configuration between the multiple DCs based on the communication bandwidth requirements between the multiple DCs.

[0015] In one possible implementation, the traffic information includes a traffic matrix and a mapping relationship. The rows and columns in the traffic matrix correspond to computing nodes within multiple DCs, and the element values ​​in the traffic matrix are used to indicate the amount of communication traffic between different computing nodes within a DC. The mapping relationship is a mapping relationship between the identifiers of computing nodes within multiple DCs and the identifiers of multiple DCs.

[0016] In one possible implementation, the multiple DCs include a first DC and a second DC; the determining module is configured to: determine multiple communication flows between computing nodes in the first DC and computing nodes in the second DC based on the traffic matrix and mapping relationship; and aggregate the multiple communication flows to obtain the communication bandwidth requirements between the first DC and the second DC.

[0017] In one possible implementation, computing nodes within multiple data centers (DCs) execute AI tasks based on configuration files. The configuration files indicate subtasks within the AI ​​tasks executed by the computing nodes within the DCs, and include a mapping between the identifiers of the computing nodes within the multiple DCs and the identifiers of the multiple DCs.

[0018] In one possible implementation, the multiple DCs include a first DC, a second DC, and a third DC; the adjustment module is configured to: allocate a portion of the bandwidth resources between the second DC and the third DC to the first DC according to the communication bandwidth requirements between the multiple DCs, wherein the portion of the bandwidth resources is used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

[0019] In one possible implementation, the multiple DCs include a first DC and a second DC, the second DC being interconnected with a fourth DC that is not involved in performing the AI ​​task via an optical switching network; the adjustment module is used to: allocate a portion of the bandwidth resources between the second DC and the fourth DC to the first DC according to the communication bandwidth requirements between the first DC and the second DC, the portion of the bandwidth resources being used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

[0020] In one possible implementation, the optical switching network includes an OTN (Optical Transport Network).

[0021] The management node provided in the second aspect corresponds to the resource scheduling method provided in the first aspect. Therefore, the technical effects of any implementation method in the second aspect can be found in the relevant descriptions of the technical effects of the corresponding implementation methods in the first aspect, and will not be repeated here.

[0022] Thirdly, this application provides a management node, which includes a processor and a memory. The processor and the memory communicate with each other. The processor executes instructions stored in the memory to cause the management node to perform a resource scheduling method as described in the first aspect or any implementation thereof. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The computing device may also include a bus. The processor is connected to the memory via the bus. The memory may include readable storage and random access memory.

[0023] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computing device, cause the computing device to perform the operation steps of the resource scheduling method described in the first aspect or any implementation thereof.

[0024] Fifthly, this application provides a computer program product containing instructions that, when run on a computing device, causes the computing device to perform the operational steps of the resource scheduling method described in the first aspect or any implementation thereof.

[0025] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0026] Figure 1 A schematic diagram illustrating full connectivity between multiple data centers based on an OTN network;

[0027] Figure 2 A schematic diagram of the structure of an exemplary data processing system provided in this application;

[0028] Figure 3 This is a schematic diagram of the structure of another exemplary data processing system provided in this application;

[0029] Figure 4 A flowchart illustrating a resource scheduling method provided in this application;

[0030] Figure 5 This is a diagram illustrating the addition of mapping relationships in the rank table file;

[0031] Figure 6A schematic diagram illustrating the adjustment of the connection status between the source end and different destination ends of an optical switching node;

[0032] Figure 7 A schematic diagram of the structure of a management node provided in this application;

[0033] Figure 8 This is a schematic diagram of the hardware structure of a management node provided in this application. Detailed Implementation

[0034] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a method of distinction used in describing objects with the same attributes in the embodiments of this application.

[0035] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0036] See Figure 2 This is a schematic diagram of the structure of an exemplary data processing system 10. Figure 2 As shown, the data processing system 10 includes a management node 100 and multiple data centers (DCs), each DC including multiple computing nodes. For ease of understanding, Figure 2 The following is an example of a data processing system 10 comprising four data centers (DC1 to DC4) and four computing nodes in each DC.

[0037] Different data centers (DCs) can be interconnected via optical switching networks. An optical switching network refers to a network that transmits data using optical switching technology, such as an OTN network. An optical switching network can include multiple optical switching nodes, such as... Figure 2 The optical switching nodes 301 to 304 are shown. Furthermore, different data centers (DCs) can communicate with each other via these optical switching nodes. For example, data sent by a computing node in DC1 can be transmitted to a computing node in DC2 via optical switching nodes 301 and 302. An optical switching node is a switching node that uses optical signals for data exchange, such as OTN equipment or optical cross-connect (OXC) equipment, where the optical signals can be transmitted via optical fiber.

[0038] And, as Figure 2As shown, the multiple DCs in the data processing system 10 can adopt a full-mesh architecture, meaning that any DC can be directly connected to other DCs through an optical switching node. Alternatively, the multiple DCs in the data processing system 10 may not adopt a full-mesh architecture; for example, DC4 can be directly connected to DC1 and DC3 only through an optical switching node, and not directly connected to DC2, etc.

[0039] For example, a computing node in the data processing system 10 refers to a node with data computing capabilities. A computing node can be implemented using processor cores, such as a die containing multiple processor cores. Alternatively, a computing node can be implemented using a processor or a computing device including a processor, such as a server. Alternatively, a computing node can also be implemented using processes within a processor; and when a processor runs only one process, the processor or the process within that processor can be considered a computing node. The processor can be any type of processor or any combination thereof, such as a central processing unit (CPU), an accelerator, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system-on-chip (SoC), a software-defined infrastructure (SDI) chip, an artificial intelligence (AI) chip, or a data processing unit (DPU). Accelerators can be, for example, graphics processing units (GPUs), neural network processing units (NPUs), or tensor processing units (TPUs).

[0040] Within a data center (DC), multiple computing nodes can communicate via electrical switching nodes. For example, ... Figure 2As shown, computing nodes 201 to 204 within DC1 can communicate via electrical switching nodes 1 to 3. An electrical switching node is a switching node that uses electrical signals for data exchange. These electrical signals can be transmitted via twisted-pair or coaxial cables, such as electrical switches, network interface cards (NICs), and routers. Furthermore, multiple electrical switching nodes within each DC can be deployed in multiple layers. For example, as... Figure 2 As shown, electrical switching nodes 1 to 3 within DC1 can be deployed in two layers. Electrical switching nodes 1 and 2 can be deployed as leaf nodes in the first layer (L1), and electrical switching node 3 can be deployed as a spine node in the second layer (L2). The spine node can forward data between different leaf nodes.

[0041] Figure 2 The data processing system 10 shown can run AI tasks, which refer to tasks involving AI models, such as training AI models or performing inference using AI models. In practical applications, when the AI ​​model has a large number of parameters, the data processing system 10 can utilize multiple computing nodes within a data center to jointly execute the AI ​​task. For example, the AI ​​model can be a large language model (LLM), a bidirectional encoder representations from transformers (BERT) model, or a generative pre-trained transformer 3 (GPT-3) model, or other types of models such as GPT-4, etc., without limitation.

[0042] When performing AI tasks using computing nodes within multiple data centers (DCs), these nodes can communicate with each other via optical switching networks. If the communication bandwidth between different DCs is statically configured, it can easily lead to both insufficient bandwidth resources between some DCs and wasted bandwidth resources between others.

[0043] For example, suppose data processing system 10 uses computing nodes in DC1, DC2, and DC3 to execute the same AI task, and the communication bandwidth between DC1 and DC3 is 4Tbps (terabits per second). Since the computing nodes in different DCs execute different parts of the AI ​​task, the communication traffic between the computing nodes in different DCs may differ, resulting in different bandwidth requirements for the AI ​​task. Suppose the AI ​​task requires 5Tbps of communication bandwidth between DC1 and DC2, and 3Tbps between DC2 and DC3 (computing nodes in DC1 and DC3 do not need to communicate). Then, the 4Tbps communication bandwidth between DC1 and DC2 is insufficient to meet the 5Tbps required for executing the AI ​​task. This will lead to longer data communication times between computing nodes in DC1 and DC2, thus reducing the overall efficiency of data processing system 10 in executing the AI ​​task. Meanwhile, the 4Tbps communication bandwidth between DC2 and DC3 is greater than the 3Tbps communication bandwidth required to perform AI tasks. This will result in 1Tbps of communication bandwidth resources being idle between DC2 and DC3 for a long time, thus wasting bandwidth resources and reducing the bandwidth utilization of the optical switching network.

[0044] Based on this, in the data processing system 10 provided in this application, the management node 100 dynamically adjusts the bandwidth configuration between different DCs so that different DCs can have sufficient bandwidth resources to transmit cross-DC communication traffic between different computing nodes and improve the bandwidth utilization of the optical switching network.

[0045] In practice, the management node 100 can acquire traffic information, which indicates the communication traffic generated when multiple data centers (DCs) participating in the AI ​​task execute the task in parallel. This traffic information reflects the volume of cross-DC communication traffic between computing nodes within different DCs. Based on this traffic information, the management node 100 can determine the communication bandwidth requirements between multiple DCs and adjust the bandwidth configuration accordingly. For example, this can be achieved by dynamically adjusting the ports of optical switching nodes in the optical switching network.

[0046] In this way, multiple data centers (DCs) have sufficient bandwidth resources to transmit cross-DC communication traffic between multiple computing nodes, thereby improving the communication efficiency between computing nodes within different DCs and thus improving the overall efficiency of the computing nodes in these multiple DCs in executing AI tasks. Furthermore, the bandwidth configuration among multiple DCs can be dynamically adjusted, which avoids the situation where a fixed amount of bandwidth is statically allocated to each DC, resulting in some bandwidth resources remaining idle for extended periods, thereby improving the overall bandwidth utilization of the optical switching network.

[0047] Continuing with the example of using computing nodes in DC1, DC2, and DC3 to execute the same AI task, assume the communication bandwidth between DC1 and DC3 is 4Tbps. During the execution of this AI task, management node 100 can obtain the traffic information generated by the computing nodes in DC1 to DC3, and determine the communication bandwidth requirement between DC1 and DC2 as 5Tbps and between DC2 and DC3 as 3Tbps based on this traffic information. Then, management node 100 can adjust the ports of optical switching nodes 302 and 301 according to the different communication bandwidth requirements between DCs, thereby reallocating 1Tbps of bandwidth resources between DC2 and DC3 to DC1. Thus, the bandwidth resource between DC1 and DC2 changes to 5Tbps (i.e., 4Tbps + 1Tbps), and the bandwidth resource between DC2 and DC3 changes to 3Tbps (i.e., 4Tbps - 1Tbps). In this way, DC1 and DC2 have sufficient bandwidth resources to transmit cross-DC communication traffic between multiple computing nodes within these two DCs, while there is not much bandwidth resource between DC2 and DC3 that remains idle for extended periods. Thus, by dynamically adjusting the bandwidth configuration among DC1, DC2, and DC3, not only can the overall efficiency of these multiple DCs in performing AI tasks be improved, but the bandwidth utilization of the optical switching network can also be increased.

[0048] For example, the management node 100 can be implemented by software or hardware. When implemented by software, the management node 100 can be an application running on hardware, such as a process running on a computing node / server. When implemented by hardware, the management node 100 can be a processor, or a computing device including a processor, or it can be implemented by multiple computing devices, etc., without limitation.

[0049] Furthermore, the management node 100 can connect to various optical switching nodes in the optical switching network. For example, it can connect optical switching nodes 301 to 304 through a management plane network or a service plane network. Figure 2As shown, this allows for adjustments to the port configurations of each optical switching node based on the connection, thereby enabling adjustments to the bandwidth configuration between different DCs.

[0050] It is worth noting that the above Figure 2 The data processing system 10 shown is only an example and is not intended to be limiting.

[0051] In another example, and in other possible implementations, the data processing system 10 may further include a greater number of DCs, computing nodes, electrical switching nodes, and optical switching nodes. Furthermore, the connection method between different DCs can be... Figure 2 In addition to the fully connected configuration shown, non-fully connected configurations are also possible. Furthermore, computing nodes within each DC can be connected through more or fewer levels of electrical switching nodes, or different computing nodes within the DC can be connected via a bus, etc.

[0052] In yet another example, the data processing system 10 may also include other types of nodes. For example, in Figure 3 The data processing system 10 shown may also include a scheduling node 400, which can be used to schedule tasks (such as AI tasks) to computing nodes within the data processing system 10. In practical applications, the scheduling node 400 may be, for example, a scheduler in a Kubernetes (K8S) platform. Figure 3 In the data processing system 10 shown, the management node may specifically include a master management node 101, multiple slave management nodes (such as slave management node 111, slave management node 112, etc.), and a network cloud engine-transport (NCE-T) 121. Among them, such as... Figure 3 As shown, each slave management node can be responsible for collecting the communication traffic between computing nodes within a DC and other computing nodes when executing AI tasks. The master management node 101 can determine the communication bandwidth requirements between multiple DCs participating in the execution of the AI ​​task based on the traffic information collected by each slave management node, and instruct NCE-T 121 to adjust the bandwidth configuration between different DCs according to the communication bandwidth requirements.

[0053] For ease of understanding, embodiments of the resource scheduling method provided in this application will be described below with reference to the accompanying drawings.

[0054] See Figure 4 , Figure 4 This is a flowchart illustrating a resource scheduling method provided in an embodiment of this application. This method can be applied to... Figure 2 or Figure 3The data processing system 10 shown can be applied to other suitable data processing systems. Each data center (DC) includes one or more computing nodes.

[0055] in, Figure 4 The resource scheduling method shown may specifically include the following steps.

[0056] S401: Management node 100 obtains traffic information, which is used to indicate the communication traffic generated when multiple DCs execute AI tasks in parallel.

[0057] In this embodiment, the data processing system 10 can utilize computing nodes within multiple data centers to execute AI tasks in parallel. For example, the AI ​​task can be scheduled by the scheduling node 400 within the data processing system 10 to multiple computing nodes within the data centers. Figure 4 As shown in the diagram, before scheduling computing nodes to execute AI tasks, scheduling node 400 can pre-acquire global topology information. This global topology information can be used to indicate the network topology between computing nodes, electrical switching nodes, and optical switching nodes in the data processing system 10. For example, global topology information can be generated by accessing routing information in each computing node, electrical switching node, and optical switching node. Furthermore, scheduling node 400 can also acquire the resource status of each data center (DC) and the communication latency between different DCs. The resource status of a DC can include the status of available resources such as computing resources (e.g., available computing nodes) and communication resources (e.g., remaining available communication bandwidth between different computing nodes). Then, based on the global topology information, the resource status of each DC, and the communication latency between different DCs, scheduling node 400 can schedule AI tasks to computing nodes within the corresponding multiple DCs and instruct the computing nodes within those multiple DCs to execute the AI ​​tasks.

[0058] During the execution of AI tasks, computing nodes within different data centers (DCs) can communicate with each other, resulting in cross-DC communication traffic. This traffic is transmitted based on the communication bandwidth between DCs. Simultaneously, different computing nodes within a single DC may also communicate with each other, generating inter-node communication traffic within the same DC.

[0059] In practical applications, within a data center (DC), network links are typically designed with a 1:1 convergence ratio between the total bandwidth of all uplink interfaces and the total bandwidth of all downlink interfaces. Therefore, the communication traffic generated by multiple computing nodes executing AI tasks within the DC usually does not experience congestion during transmission through the network links, ensuring efficient data communication between different computing nodes. However, the cost of network bandwidth connecting different DCs via optical switching networks is typically high, thus limiting the total bandwidth resources available across multiple DCs. Figure 1 The total bandwidth resources between DC1, DC2, and DC3 are 32*400Gbps, which can easily affect the data communication efficiency between computing nodes in different DCs.

[0060] To this end, management node 100 can obtain traffic information used to instruct different computing nodes to perform AI tasks, so as to optimize the bandwidth configuration between different DCs based on the traffic information.

[0061] For example, the traffic information obtained by the management node 100 may specifically include a traffic matrix and mapping relationships.

[0062] A traffic matrix is ​​a matrix that reflects the amount of communication traffic between different computing nodes. Each row in the traffic matrix corresponds to a computing node participating in the execution of the AI ​​task, and each column also corresponds to a computing node participating in the execution of the AI ​​task. Thus, the value of each element in the traffic matrix indicates the computing node... row With computing nodes column The amount of communication traffic between them. Among them, the computing nodes... row This refers to the compute node indicated by the row containing the element; compute node column , refers to the computed node indicated by the column where the element is located.

[0063] The mapping relationship refers to the mapping relationship between the identifiers of computing nodes within multiple data centers (DCs) and the identifiers of the multiple DCs. Specifically, the identifier of one computing node can be mapped to the identifier of one DC. Thus, the management node 100 can determine the DC to which each computing node participating in the execution of the AI ​​task belongs based on this mapping relationship.

[0064] In one possible implementation, each data center (DC) may include a device driver. This device driver may pre-store a mapping between the identifiers of computing nodes within the DC and the identifier of the DC itself, as well as a mapping between the identifiers of other computing nodes in other DCs that execute the same communication operator as the computing nodes within the DC and the identifiers of the DCs to which those other computing nodes belong. The device driver may be a driver used to manage the computing nodes. For example, when computing nodes are implemented using accelerators such as GPUs, the device driver may specifically be a baseboard management controller (BMC) driver in one or more computing servers housing the accelerators. In practical applications, before executing AI tasks, the mapping between the identifiers of each computing node within multiple DCs and the identifiers of the DCs to which those computing nodes belong can be pre-generated through low-level design (LLD) planning and written into the device driver within the DC using appropriate service tools. Specifically, when the device driver is a BMC driver, for multiple computing servers within the DC, the service tool can write the corresponding mapping relationship of each computing server to the BMC driver of each computing server. The mapping relationship stored in each BMC driver can be the mapping relationship between the identifiers of each computing node included in its computing server and the DC identifier, as well as the mapping relationship between the identifiers of other computing nodes communicating with the computing node and the identifiers of the DC to which the other computing nodes belong.

[0065] Then, the computing nodes or computing servers (including one or more computing nodes) within the DC can generate a traffic matrix based on the mapping relationship stored in the device driver, and send the traffic matrix and the mapping relationship stored in the device driver to the management node 100. Taking the computing server sending information to the management node 100 as an example, this embodiment provides the following two implementation examples.

[0066] In the first implementation example, before executing the AI ​​task, the scheduling node 400 in the data processing system 10 can obtain the mapping relationship between the identifiers of computing nodes and the identifiers of DCs from the device drivers in each DC. For example, the scheduling node 400 can access the mapping relationship from the device driver through the DCMI (DaVinci card management interface) provided by the device plug-in. Then, the scheduling node 400 can send a configuration file to the collective communication library in each DC. This configuration file can carry the mapping relationship accessed by the scheduling node 400, and it is used to indicate the sub-tasks in the AI ​​task executed by the computing nodes in the DC. For example, the configuration file sent by the scheduling node 400 can be a rank table file, which can be used to indicate the sub-tasks executed by each computing node participating in the AI ​​task. For example, when the AI ​​task is specifically a training task for an AI model, the rank table can record information such as the attributes of each computing node (e.g., identifier), the training samples used by each computing node, and the parameters of the model it is responsible for training. Figure 5 As shown, scheduling node 400 can... Figure 5 Adding mapping relationships to the rank table file shown above will yield the following result: Figure 5 The rank table file shown below contains the mapping relationships.

[0067] In this way, during the execution of AI tasks by the computing nodes in the computing server, the computing server can obtain the mapping relationship between the identifiers of the computing nodes and the DC identifiers from the configuration file. Furthermore, the computing server can determine the communication traffic between the computing nodes in the computing server and the communication traffic between that computing node and other computing nodes based on the communication operators executed by the computing nodes in the computing server, and generate a corresponding traffic matrix based on the communication traffic between different computing nodes. Therefore, the computing server can send the generated traffic matrix and the mapping relationship obtained from the configuration file to the management node 100. When the management node 100 includes a master management node and slave management nodes, the computing server can send the traffic matrix and mapping relationship to the slave management node, which then provides it to the master management node.

[0068] In the second implementation example, the computing server can directly access the mapping relationship between the identifiers of computing nodes and the DC identifiers from the device driver. Furthermore, the computing server can determine the communication traffic between computing nodes within the server and the communication traffic between that computing node and other computing nodes based on the communication operators executed by the computing nodes in the server, and generate a corresponding traffic matrix accordingly. The computing server can then send the generated traffic matrix and the obtained mapping relationship to the management node 100.

[0069] In this way, by aggregating the traffic matrices and mapping relationships sent by each computing server, the management node 100 can obtain the above traffic information and determine the communication traffic between computing nodes in multiple DCs.

[0070] It is understood that the various implementation methods for the management node 100 to obtain traffic information described above are merely illustrative examples and are not intended to limit the scope. In other embodiments, the management node 100 may also obtain traffic information in other ways, such as through reporting by various computing nodes, and the traffic information obtained by the management node 100 may also be in other forms, not limited to traffic matrices, etc.

[0071] S402: Management node 100 determines the communication bandwidth requirements between multiple DCs based on traffic information.

[0072] After determining the communication traffic between computing nodes within multiple DCs, the management node 100 can aggregate the communication traffic between computing nodes belonging to different DCs to obtain the communication traffic between different DCs, thereby determining the communication bandwidth requirement between different DCs.

[0073] In one possible implementation, when the traffic information specifically includes a traffic matrix and mapping relationships, the management node 100 can determine the communication traffic between computing nodes within multiple data centers (DCs) based on the traffic matrix, and distinguish between the communication traffic between computing nodes within different DCs and the communication traffic between different computing nodes within the same DC based on the DC to which each computing node belongs, as indicated by the mapping relationships. Thus, by aggregating the identified communication traffic between computing nodes in different DCs, the management node 100 can obtain the communication bandwidth requirements between different DCs.

[0074] Taking the determination of communication bandwidth requirements between DC1 and DC2 as an example, assume that there is cross-DC communication traffic between computing nodes 201 and 205, with a traffic volume of 1.25 Tbps; between computing nodes 202 and 206, with a traffic volume of 1.25 Tbps; between computing nodes 203 and 207, with a traffic volume of 1.25 Tbps; and between computing nodes 204 and 208, with a traffic volume of 1.25 Tbps. Simultaneously, computing node 201 will also communicate with computing nodes 202 through 204, thus generating communication traffic within DC1; similarly, computing node 205 will also communicate with computing nodes 206 through 208, thus generating communication traffic within DC2. Therefore, management node 100 can identify the cross-DC communication traffic between computing nodes 201 to 204 in DC1 and computing nodes 205 to 208 in DC2 from the traffic matrix according to the mapping relationship. Then, management node 100 can aggregate these communication traffic and determine that the communication traffic size between DC1 and DC2 is 5Tbps (i.e., 4*1.25Tbps). Thus, management node 100 can determine that the communication bandwidth requirement between DC1 and DC2 is 5Tbps.

[0075] S403: Management node 100 adjusts the bandwidth configuration between multiple DCs according to the communication bandwidth requirements between multiple DCs.

[0076] After determining the communication bandwidth requirements between different data centers (DCs) during the execution of AI tasks, the management node 100 can adjust the bandwidth configuration among multiple DCs so that they can continue to execute AI tasks based on the adjusted bandwidth configuration. At this point, the multiple DCs have sufficient bandwidth resources to transmit communication traffic between them.

[0077] In practice, the management node 100 can adjust the bandwidth configuration between multiple DCs by adjusting the number of ports connected between different optical switching nodes in the optical switching network.

[0078] like Figure 6As shown, optical switching nodes can use lenses to connect input and output fiber optic ports, such as micro-electro-mechanical system (MEMS) lenses. Furthermore, the optical switching node can control the connection state between the input fiber optic port and different output fiber optic ports by adjusting the rotation angle of the lens, thereby controlling the connection or disconnection between the source end and different destination ends. For example, before the optical switching node adjusts the rotation angle of the lens, the source end can maintain a connection with destination end 1; that is, the optical switching node can transmit the optical signal sent by the source end to destination end 1 through output port 1. After the optical switching node adjusts the rotation angle of the lens, the optical signal sent by the source end will be transmitted to destination end 2 through output port 2. At this time, the connection between the source end and destination end 1 is disconnected, while the connection between the source end and the destination end remains maintained, thereby achieving bandwidth adjustment between the source end and different destination ends.

[0079] Based on this, management node 100 can adjust the number of ports on the optical switching nodes connecting each DC in the optical switching network according to the communication bandwidth requirements between multiple DCs, thereby adjusting the bandwidth resources between multiple DCs. For example, assuming that optical switching node 301 connecting DC1 and optical switching node 302 connecting DC2 are connected through 16 ports, and the total bandwidth resource between DC1 and DC2 is 4Tbps, then management node 100 can increase the number of ports connecting optical switching node 302 and optical switching node 301 from 16 to 20, so that the total bandwidth resource between DC1 and DC2 increases from 4Tbps to 5Tbps, thus meeting the communication bandwidth requirement (5Tbps) between DC1 and DC2.

[0080] In this embodiment, the following two implementation methods for adjusting the bandwidth configuration among multiple DCs are provided.

[0081] In the first implementation, the management node 100 dynamically adjusts bandwidth resources across multiple DCs participating in the execution of the AI ​​task.

[0082] In a specific implementation, assuming that AI tasks are performed using DC1, DC2, and DC3, and the amount of bandwidth resources between DC1 and DC2 is less than the communication bandwidth requirement between DC1 and DC2, while the amount of bandwidth resources between DC2 and DC3 is greater than the communication bandwidth requirement between DC2 and DC3, then the management node 100 can allocate a portion of the bandwidth resources between DC2 and DC3 to DC1, so that this portion of bandwidth resources can be used to transmit communication traffic between computing nodes in DC1 and computing nodes in DC2.

[0083] For example, assuming the bandwidth resources between DC1 and DC2, and between DC2 and DC3, are both 4Tbps, and optical switching node 302 is connected to optical switching nodes 301 and 303 via 16 ports respectively, then when the communication bandwidth requirement between DC1 and DC2 is 5Tbps, and the communication bandwidth requirement between DC2 and DC3 is 3Tbps, management node 100 can, based on the different communication bandwidth requirements between DCs, disconnect the four ports connecting optical switching nodes 302 and 303, and connect them to optical switching node 301 via those four ports. This increases the number of ports connecting optical switching nodes 302 and 301 from 16 to 20, thereby increasing the bandwidth resource between DC1 and DC2 from 4Tbps to 5Tbps. Correspondingly, the number of ports connecting optical switching nodes 302 and 303 decreases from 16 to 12, thereby reducing the bandwidth resource between DC2 and DC3 from 4Tbps to 3Tbps.

[0084] In the second implementation, the management node 100 reallocates a portion of the bandwidth resources of the DCs that are not involved in executing the AI ​​task to the multiple DCs that are involved in executing the AI ​​task, so that the bandwidth resources among the multiple DCs involved in executing the AI ​​task can meet the communication bandwidth requirements among the multiple DCs.

[0085] In a specific implementation, assuming that AI tasks are executed using computing nodes within DC1 and DC2, and DC4 is not involved in executing the AI ​​task, then the management node 100 can allocate a portion of the bandwidth resources between DC2 and DC4 to DC1 based on the communication bandwidth requirements between DC1 and DC2, so that the communication traffic between computing nodes in DC1 and computing nodes in DC2 can be transmitted using this portion of bandwidth resources.

[0086] For example, assuming the bandwidth resource between DC1 and DC2 is 4Tbps, optical switching node 302 is connected to optical switching node 301 through 16 ports. Furthermore, optical switching node 302 can also be connected to optical switching node 304 through 12 ports; correspondingly, the bandwidth resource between DC2 and DC4 is 3Tbps. Then, when the communication bandwidth requirement between DC1 and DC2 is 6Tbps, management node 100 can, based on the bandwidth requirement between DC1 and DC2, disconnect 8 ports connecting optical switching node 302 and optical switching node 304, and connect to optical switching node 301 through these 8 ports. This increases the number of ports connecting optical switching node 302 and optical switching node 301 from 16 to 24, thereby increasing the bandwidth resource between DC1 and DC2 from 4Tbps to 6Tbps. Correspondingly, the number of ports connecting optical switching node 302 and optical switching node 304 decreases from 12 to 4, thereby reducing the bandwidth resource between DC2 and DC4 from 3Tbps to 1Tbps.

[0087] It is understood that the above-described implementation of adjusting the bandwidth configuration among multiple DCs is merely an illustrative example and is not intended to limit the scope. For example, in other embodiments, when the total bandwidth resources among multiple DCs participating in the AI ​​task are less than the sum of the communication bandwidth requirements among the multiple DCs, the management node 100 can not only reallocate some of the bandwidth resources among the DCs to other DCs, but also reallocate some of the bandwidth resources of DCs not participating in the AI ​​task to the multiple DCs participating in the AI ​​task. Furthermore, in practical application scenarios, when the management node 100 includes a master management node 101 and an NCE-T 121, the master management node 101 can determine the communication bandwidth requirements among different DCs and provide these requirements to the NCE-T 121. The NCE-T 121 can then adjust the connection status of the ports of the corresponding optical switching nodes in the optical switching network according to these bandwidth requirements, thereby achieving the adjustment of the bandwidth configuration among multiple DCs.

[0088] In this way, the management node 100 adjusts the bandwidth configuration among multiple DCs based on the actual bandwidth requirements generated during the execution of AI tasks by multiple DCs. This ensures that multiple DCs have sufficient bandwidth resources to transmit cross-DC communication traffic between multiple computing nodes, thereby improving the communication efficiency between multiple computing nodes and thus improving the overall efficiency of the computing nodes within the multiple DCs in executing AI tasks. Furthermore, after dynamically adjusting the bandwidth configuration among multiple DCs, the reallocated bandwidth resources can be used to transmit communication traffic between different DCs, thus preventing these bandwidth resources from remaining idle for extended periods and improving the overall bandwidth utilization of the optical switching network.

[0089] It should be noted that the AI ​​tasks executed by the computing nodes within the aforementioned multiple data centers (DCs) can be a single AI task or multiple AI tasks. For example, when the AI ​​task is specifically a training task for an AI model, the data processing system 10 can utilize the computing nodes within DC1, DC2, and DC3 to train two AI models simultaneously. In this case, each computing node can run two processes, with different processes used to train different AI models. The management node 100 can obtain the communication bandwidth requirements generated between DC1 and DC3 during the parallel training of two AI models and adjust the bandwidth configuration among the multiple DCs accordingly to ensure that the overall efficiency of the data processing system 10 in training multiple AI models reaches a high level. Simultaneously, each computing node can execute multiple AI tasks in parallel, thereby improving the utilization rate of computing resources in the data processing system 10.

[0090] It is worth noting that other reasonable combinations of steps that can be conceived by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0091] The above combination Figures 1 to 6 The resource scheduling method provided in the embodiments of this application will be introduced. Next, the structure of the management node provided in the embodiments of this application will be described with reference to the accompanying drawings.

[0092] See Figure 7 This illustrates a schematic diagram of a management node structure. For example... Figure 7 As shown, management node 700 includes:

[0093] The acquisition module 701 is used to acquire traffic information, which is used to indicate the communication traffic generated when multiple DCs (data centers) execute AI (artificial intelligence) tasks in parallel. Each data center includes a computing node, and the computing nodes between multiple DCs are interconnected through an optical switching network.

[0094] The determination module 702 is used to determine the communication bandwidth requirements between multiple DCs based on traffic information;

[0095] The adjustment module 703 is used to adjust the bandwidth configuration between multiple DCs according to the communication bandwidth requirements between multiple DCs.

[0096] In one possible implementation, the traffic information includes a traffic matrix and a mapping relationship. The rows and columns in the traffic matrix correspond to computing nodes within multiple DCs, and the element values ​​in the traffic matrix are used to indicate the amount of communication traffic between different computing nodes within a DC. The mapping relationship is a mapping relationship between the identifiers of computing nodes within multiple DCs and the identifiers of multiple DCs.

[0097] In one possible implementation, the plurality of DCs includes a first DC and a second DC; the determining module 702 is configured to:

[0098] Based on the traffic matrix and mapping relationship, determine multiple communication traffic between computing nodes in the first DC and computing nodes in the second DC;

[0099] Multiple communication traffic streams are aggregated to obtain the communication bandwidth requirements between the first DC and the second DC.

[0100] In one possible implementation, computing nodes within multiple data centers (DCs) execute AI tasks based on configuration files. The configuration files indicate subtasks within the AI ​​tasks executed by the computing nodes within the DCs, and include a mapping between the identifiers of the computing nodes within the multiple DCs and the identifiers of the multiple DCs.

[0101] In one possible implementation, the plurality of DCs includes a first DC, a second DC, and a third DC; the adjustment module 703 is used for:

[0102] Based on the communication bandwidth requirements between multiple DCs, a portion of the bandwidth resources between the second DC and the third DC are allocated to the first DC, and the remaining bandwidth resources are used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

[0103] In one possible implementation, the multiple DCs include a first DC and a second DC, the second DC also being interconnected via an optical switching network with a fourth DC that is not involved in performing the AI ​​task; the adjustment module 703 is used for:

[0104] Based on the communication bandwidth requirements between the first DC and the second DC, a portion of the bandwidth resources between the second DC and the fourth DC are allocated to the first DC, and the remaining bandwidth resources are used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

[0105] In one possible implementation, the optical switching network includes an OTN (Optical Transport Network).

[0106] because Figure 7 The management node 700 shown corresponds to the above. Figure 4 The management node 100 in the illustrated embodiment, therefore Figure 7For details on the implementation of the management node 700 and its technical effects, please refer to the above. Figure 4 The relevant descriptions in the illustrated embodiments are not repeated here.

[0107] Figure 8 This is a schematic diagram of the structure of a management node provided in this application. Figure 8 As shown, the management node 800 includes a processor 801, a memory 802, a communication interface 803, and a bus 804. The processor 801, memory 802, and communication interface 803 communicate via the bus 804. The bus 804 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus. The communication interface 803 is used for external communication, such as sending data to or receiving data from other computing nodes.

[0108] It should be understood that in the embodiments of this application, processor 801 may be a CPU, or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete device assemblies, etc. General-purpose processors may be microprocessors or any conventional processors, etc.

[0109] The memory 802 may include read-only memory and random access memory, and provides instructions and data to the processor 801. The memory 802 may also include non-volatile random access memory. For example, the memory 802 may also store device type information.

[0110] The memory 802 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0111] The memory 802 stores executable code, and the processor 801 executes the executable code to perform the aforementioned actions. Figure 4 The method executed by management node 100 in the illustrated embodiment.

[0112] It should be understood that the management node 800 according to the embodiments of this application may correspond to the management node 100 in the embodiments of this application, and may correspond to the execution of the implementation according to the embodiments of this application. Figure 4 The methods executed by management node 100 in the illustrated method, and the aforementioned and other operations and / or functions implemented by management node 800, are respectively for the purpose of implementing... Figure 4 The process of the corresponding methods in [the document] will not be elaborated here for the sake of brevity.

[0113] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the resource scheduling method described above.

[0114] This application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application are generated.

[0115] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0116] The computer program product can be a software installation package. When any of the aforementioned resource scheduling methods is required, the computer program product can be downloaded and executed on a computing device.

[0117] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0118] The terminology used in the above embodiments is for the purpose of describing specific embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, “one or more” refers to one, two, or more; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship. In the embodiments of this application, “simultaneously” means within the same time period, including situations where they are at the same moment.

[0119] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0120] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A resource scheduling method, characterized in that, The method includes: Acquire traffic information, which is used to indicate the communication traffic generated when multiple data center DCs execute artificial intelligence (AI) tasks in parallel, wherein each data center includes a computing node, and the computing nodes between the multiple DCs are interconnected through an optical switching network; Based on the traffic information, determine the communication bandwidth requirements between the multiple DCs; Adjust the bandwidth configuration among the multiple DCs according to the communication bandwidth requirements among the multiple DCs.

2. The method according to claim 1, characterized in that, The traffic information includes a traffic matrix and a mapping relationship. The rows and columns in the traffic matrix correspond to the computing nodes within the multiple DCs, and the element values ​​in the traffic matrix are used to indicate the communication traffic volume between different computing nodes within the DCs. The mapping relationship is a mapping relationship between the identifiers of the computing nodes within the multiple DCs and the identifiers of the multiple DCs.

3. The method according to claim 2, characterized in that, The plurality of DCs includes a first DC and a second DC; Determining the communication bandwidth requirements among the multiple DCs based on the traffic information includes: Based on the traffic matrix and the mapping relationship, multiple communication traffic flows between computing nodes in the first DC and computing nodes in the second DC are determined. The multiple communication traffic flows are aggregated to obtain the communication bandwidth requirements between the first DC and the second DC.

4. The method according to claim 2 or 3, characterized in that, The computing nodes within the plurality of DCs execute the AI ​​task based on a configuration file. The configuration file is used to indicate the sub-tasks in the AI ​​task executed by the computing nodes within the DCs. The configuration file includes a mapping relationship between the identifiers of the computing nodes within the plurality of DCs and the identifiers of the plurality of DCs.

5. The method according to any one of claims 1 to 4, characterized in that, The plurality of DCs includes a first DC, a second DC, and a third DC; The step of adjusting the bandwidth configuration among the multiple DCs according to the communication bandwidth requirements among the multiple DCs includes: Based on the communication bandwidth requirements between the plurality of DCs, a portion of the bandwidth resources between the second DC and the third DC are allocated to the first DC. The portion of the bandwidth resources is used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

6. The method according to any one of claims 1 to 4, characterized in that, The plurality of DCs includes a first DC and a second DC, the second DC being interconnected with a fourth DC that is not involved in performing the AI ​​task via the optical switching network; The step of adjusting the bandwidth configuration among the multiple DCs according to the communication bandwidth requirements among the multiple DCs includes: Based on the communication bandwidth requirements between the first DC and the second DC, a portion of the bandwidth resources between the second DC and the fourth DC are allocated to the first DC. These portion of the bandwidth resources are used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

7. The method according to any one of claims 1 to 6, characterized in that, The optical switching network includes an optical transport network (OTN).

8. A management node, characterized in that, The management node includes: The acquisition module is used to acquire traffic information, which is used to indicate the communication traffic generated when multiple data center DCs execute artificial intelligence (AI) tasks in parallel. Each data center includes a computing node, and the computing nodes between the multiple DCs are interconnected through an optical switching network. The determining module is used to determine the communication bandwidth requirements between the multiple DCs based on the traffic information. The adjustment module is used to adjust the bandwidth configuration among the multiple DCs according to the communication bandwidth requirements among the multiple DCs.

9. The management node according to claim 8, characterized in that, The traffic information includes a traffic matrix and a mapping relationship. The rows and columns in the traffic matrix correspond to the computing nodes within the multiple DCs, and the element values ​​in the traffic matrix are used to indicate the communication traffic volume between different computing nodes within the DCs. The mapping relationship is a mapping relationship between the identifiers of the computing nodes within the multiple DCs and the identifiers of the multiple DCs.

10. The management node according to claim 9, characterized in that, The plurality of DCs includes a first DC and a second DC; The determining module is used for: Based on the traffic matrix and the mapping relationship, multiple communication traffic flows between computing nodes in the first DC and computing nodes in the second DC are determined. The multiple communication traffic flows are aggregated to obtain the communication bandwidth requirements between the first DC and the second DC.

11. The management node according to claim 9 or 10, characterized in that, The computing nodes within the plurality of DCs execute the AI ​​task based on a configuration file. The configuration file is used to indicate the sub-tasks in the AI ​​task executed by the computing nodes within the DCs. The configuration file includes a mapping relationship between the identifiers of the computing nodes within the plurality of DCs and the identifiers of the plurality of DCs.

12. The management node according to any one of claims 8 to 11, characterized in that, The plurality of DCs includes a first DC, a second DC, and a third DC; The adjustment module is used for: Based on the communication bandwidth requirements between the plurality of DCs, a portion of the bandwidth resources between the second DC and the third DC are allocated to the first DC. The portion of the bandwidth resources is used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

13. The management node according to any one of claims 8 to 11, characterized in that, The plurality of DCs includes a first DC and a second DC, the second DC being interconnected with a fourth DC that is not involved in performing the AI ​​task via the optical switching network; The adjustment module is used for: Based on the communication bandwidth requirements between the first DC and the second DC, a portion of the bandwidth resources between the second DC and the fourth DC are allocated to the first DC. These portion of the bandwidth resources are used to transmit communication traffic between computing nodes in the first DC and computing nodes in the second DC.

14. The management node according to any one of claims 8 to 13, characterized in that, The optical switching network includes an optical transport network (OTN).

15. A management node, characterized in that, The management node includes a processor and a memory; The processor is configured to execute instructions stored in the memory to cause the management node to perform the method as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computing device, cause the computing device to perform the method as described in any one of claims 1 to 7.

17. A computer program product containing instructions, characterized in that, When it is run on at least one computing device, it causes the at least one computing device to perform the method as described in any one of claims 1 to 7.

Images