A server cluster, port allocation method and related device

Abstract

Embodiments of the present application provide a server cluster, a port allocation method and related equipment, which are used to improve the calculation efficiency of a model. The server cluster provided by the embodiments of the present application runs a model, and the server cluster includes a plurality of servers and a management module. The plurality of servers include a first server and a second server. The first server includes a plurality of accelerators and a first port. The second server includes a plurality of accelerators and a second port. The management module is configured to: obtain model information of the model and a network topology of the plurality of servers, wherein the model information indicates a data communication relationship between the accelerators included in each server; determine, according to the model information and the network topology, whether there is any one server in the server cluster that has data communication with the accelerators of the first server through the first port of the first server; and when it is determined that the first server will have data communication with the second server through the first port and the second port, create a corresponding relationship between the first port and the second port.

Description

Technical Field

[0001] This application relates to the field of optical communication, and in particular to a server cluster, a port allocation method, and related equipment. Background Technology

[0002] Computing nodes based on optical networks are networked together, connecting multiple computing nodes with computing power. Each computing node includes one or more accelerators. Distributed model computation can be achieved by using accelerator clusters composed of multiple computing nodes.

[0003] Accelerator clusters can be divided into different parallel domains, with a communication domain used to coordinate data exchange between these domains. Multiple accelerators within the same communication domain need to communicate with each other. In optical networks, the distribution of accelerators within each computing node is unpredictable, making it impossible to predict whether any two computing nodes share a communication domain (i.e., whether any two computing nodes include accelerators belonging to the same communication domain) or whether there is a communication requirement between them. To meet the communication needs of all possible accelerator distributions on different computing nodes in an optical network, it can be assumed that any two computing nodes may include accelerators in the same communication domain, meaning that communication between any two computing nodes is necessary. Therefore, multiple computing nodes are typically connected in a full-mesh configuration, meaning that any computing node is connected to all computing nodes in the network except itself. Thus, in a full-mesh connection, ports for optical communication are reserved between any two computing nodes in the network.

[0004] However, in reality, some computing nodes do not share a common communication domain accelerator, meaning no traffic passes between these two nodes. The reserved ports between these two computing nodes are idle, with no signal flowing through them, resulting in wasted bandwidth and reduced communication efficiency.

[0005] The reservation of these unused ports reduces the bandwidth allocated to accelerators within computing nodes, leading to increased transmission latency. This results in communication time exceeding the corresponding computing node's computation time, thus prolonging the exposure time of communication masking by general computing, wasting computing power, and reducing the training / computation efficiency of the model. Summary of the Invention

[0006] This application provides a server cluster, port allocation method, and related equipment to reduce the communication time of distributed model computing in optical networks and improve the computational efficiency of the models.

[0007] In a first aspect, embodiments of this application provide a server cluster. The server cluster runs a model and includes multiple servers and a management module. The multiple servers include a first server and a second server. The first server includes multiple accelerators and a first port, and the second server includes multiple accelerators and a second port. The management module is used to acquire model information of the model and the network topology of the multiple servers. The model information is used to indicate the data communication relationships between the accelerators contained in each server. The management module is also used to determine, based on the model information and network topology, whether any server in the server cluster (in this embodiment, "any server" refers to any server in the server cluster other than the first server) has data communication with the accelerators of the first server through the first port. The management module is also used to create a correspondence between the first port and the second port when it is determined that the first server will send data to the second port of the second server through the first port, or when it is determined that the first server will receive data from the second port of the second server through the first port.

[0008] In this embodiment, a correspondence between the first and second ports is created when it is determined that the server will interact with the second port of the second server through the first port. Port reservations on the server are determined based on actual communication needs, rather than blindly connecting all servers. Compared to the current fullmesh connection method, this avoids the waste of port resources caused by servers connecting to other servers that do not need to communicate. It can improve the transmission bandwidth for data interaction between servers, thereby reducing transmission latency between servers, reducing communication exposure time (i.e., the duration for which the accelerator's communication time exceeds the computation time), reducing wasted computing power, and thus improving the model's computational efficiency (e.g., training efficiency, inference efficiency).

[0009] In one alternative implementation, creating a correspondence between the first port and the second port can be understood as establishing communication between the first port and the second port. For example, if the management module includes an optical switching module that connects multiple servers, the management module can adjust the optical signal paths between multiple servers in the server cluster and change the connection relationship between the ports on each server, thereby creating communication between the first port and the second port.

[0010] In another alternative implementation, creating a correspondence between the first port and the second port can be understood as indicating the communication relationship between the first port and the second port. For example, if the management module does not include an optical switching module that connects multiple servers, the management module can send first information to the optical switching module to indicate the communication relationship between the first port and the second port; the optical switching module establishes communication between the first port and the second port according to the indication of the first information.

[0011] For example, the first information can directly indicate the correspondence between the first port and the second port; or it can indirectly indicate the correspondence between the first port and the second port through the correspondence between the first port / second port and the cross-server communication domain. This application does not limit this.

[0012] In one optional implementation, the management module is specifically used to: determine, based on model information, whether any server in the server cluster has data communication with the accelerator of the first server; and, based on the existence of any server having data communication with the accelerator of the first server, determine, based on the network topology, whether any server communicates with the accelerator of the first server through the first port.

[0013] In this embodiment, the existence of data interaction between servers is used as the basis for determining whether a port connection is needed between them. First, it is determined whether any server has data communication with the first server. Then, if such a server (a server in the server cluster other than the first server that needs data interaction with the first server) exists, a first port is reserved on the first server for communication with that server. This achieves port allocation based on communication needs, reducing port resource waste and simplifying the port allocation process by using inter-server communication needs as the guide for port allocation.

[0014] In one alternative implementation, the management module is further configured to: determine the collective communication algorithm used by the accelerators that need to communicate between the second server and the first server, based on the fact that the second server communicates with the accelerators of the first server through the first port of the first server; and determine, based on the network topology and the collective communication algorithm, whether the first server will send data to the second port of the second server through the first port, or whether the first server will receive data from the second port of the second server through the first port.

[0015] In this embodiment, based on port allocation according to the communication needs between servers, and combined with the collective communication algorithm used by the accelerators that need to communicate with each other between servers, the port allocation scheme is optimized through multivariate tuning to obtain the optimal port allocation scheme (i.e., determining the ports for data communication). The optimization of the collective communication algorithm based on the accelerators can optimize the traffic distribution among multiple servers for multiple accelerators that need to communicate with each other, thereby improving the communication efficiency between the servers.

[0016] In one optional implementation, the model information is also used to indicate the data communication volume between the accelerators contained in each server, and the network topology includes the number of ports contained in each server in the server cluster. The management module is further used to determine, based on the existence of data communication between any server and the accelerator of the first server, the number of first ports used for data communication with any server, according to the data communication volume between the accelerators and the number of ports contained in each server; there may be multiple first ports. Specifically, the management module is used to create a mapping between multiple first ports and second ports.

[0017] In this embodiment, data traffic is used to indicate the data traffic between two accelerators located on different servers. Since multiple ports on a server typically share the server's optical network bandwidth, the bandwidth of a single port on a server is limited. Allocating multiple first ports for data communication between the first and second servers, with the number of first ports determined based on actual communication needs (data traffic between accelerators on different servers), can reduce latency in data communication between accelerators across servers and improve communication efficiency.

[0018] In this embodiment, the server cluster includes a communication domain, which includes multiple accelerators within the server cluster. Accelerators within the same communication domain need to communicate with each other (this can be direct or indirect). If multiple accelerators within a communication domain are all on the same server, the communication domain is called an intra-server communication domain (also known as an intra-node communication domain); if multiple accelerators within a communication domain are on multiple servers, the communication domain is called a cross-server communication domain (also known as a cross-node communication domain).

[0019] Optionally, the multiple accelerators included on the first server may belong to multiple cross-server communication domains. Besides the accelerators on the first server, these multiple cross-server communication domains also include accelerators on other servers. Therefore, the first server needs to communicate with different servers (e.g., the second server). The management module can determine the data communication volume of the multiple cross-server communication domains to which the accelerators within the first server belong, and based on the ratio of the data communication volume among the multiple cross-server communication domains to which the accelerators within the first server belong, assign ports on the first server to the corresponding cross-server communication domains (e.g., the cross-server communication domain including the accelerators of the second server). This assigns the ports on the first server to the corresponding ports of the servers where the accelerators in the corresponding cross-server communication domains reside (e.g., the second port of the second server), thereby satisfying the communication needs of the accelerators within the first server across different cross-server communication domains.

[0020] In one optional implementation, the network topology also includes the bandwidth supported by the ports of each server in the server cluster. Specifically, the management module is used to determine the number of first ports for data communication with any one server, based on the existence of data communication between any server and the accelerator of the first server, according to the data communication volume between accelerators, the number of ports of each server, and the bandwidth supported by the ports of each server.

[0021] In this embodiment, the number of first ports is determined based on the actual communication capability of the ports (the bandwidth supported by the ports), the number of ports on the server, and the actual communication needs between accelerators (the amount of data communication between accelerators on each server). This determines the bandwidth reserved for data communication between accelerators on each server (related to the number of first ports and the bandwidth supported by the ports). This ensures that the reserved bandwidth meets the actual communication needs between accelerators (e.g., the actual communication needs of the cross-server communication domains where each accelerator on the server resides), thereby reducing the latency of data communication between accelerators across servers and improving communication efficiency.

[0022] In one optional implementation, the management module is further configured to determine the first computing time for each server in the server cluster based on model information and the computing power information of the server cluster. During this first computing time, model computation and cross-server data communication can be performed synchronously. Specifically, the management module is configured to determine the number of first ports used for data communication with any server, based on the existence of data communication between any server and the accelerator of the first server, according to the data communication volume between accelerators, the number of ports contained in each server, and the first computing time of the first server. Here, the computing power information represents the number of accelerators contained in each server within the server cluster and the operating frequency of the accelerators.

[0023] The operation of the accelerator within the server includes both computation and communication processes. Some data communication needs to be completed before the computation begins. If this data communication process is performed concurrently with the computation process of the previous cycle, the accelerator's waiting time for this data communication can be reduced, allowing the computation process of the current cycle to begin earlier. In this embodiment, the cycle length of the computation process is referred to as the first computation duration. When the communication process duration is less than or equal to the first computation duration, complete computational masking can be achieved (i.e., communication time is masked within the computation time, allowing for continuous computation).

[0024] In this embodiment, the data communication volume refers to the actual amount of data to be transmitted, and the first computation duration is the duration of each computation cycle of the accelerator. Based on the data communication volume and the first computation duration, the bandwidth requirement for achieving complete compute masking can be determined. For example, the first computation duration is the window duration for achieving compute masking of the corresponding communication domain. As long as the communication duration of the communication domain is controlled within this window duration, compute masking can be achieved. The communication duration of the communication domain depends on the number of corresponding ports; the larger the number of ports, the smaller the corresponding communication duration. The size of the first computation duration is determined by the proportion of computation volume within the computation cycle to the chip's computing power. The larger the computation volume corresponding to the communication domain, the larger the first computation duration, and the longer the compute masking window time, allowing for the allocation of fewer ports to that communication domain. Conversely, the larger the first computation duration of the communication domain, the shorter the corresponding compute masking window time, requiring the allocation of more ports. Through compute masking, the computational resources wasted by the accelerator due to waiting for communication (these computational resources are actually the accelerator's time resources) are reduced, thereby improving the computational efficiency of the server cluster.

[0025] In one alternative implementation, the server cluster further includes a first optical switching module, which is used to switch the state between the first port and the second port to a communicable state.

[0026] In this embodiment of the application, the connection state between ports on each server is changed by the first optical switching module (from non-communicable state to communicable state, or from communicable state to non-communicable state), so that the connection state between each server is adjustable, and the server cluster can adapt to various communication needs, thereby adapting to various distributed computing schemes and expanding the applicable scenarios of the server cluster.

[0027] In one optional implementation, the server cluster further includes multiple optical switching modules (the aforementioned first optical switching module is one of these multiple optical switching modules). The management module is also configured to: determine multiple port connection schemes based on the existence of any server communicating with the accelerator of the first server through the first port of the first server, wherein different port connection schemes indicate that the first port of the first server is connected to different servers in the server cluster; determine a second computation time under each port connection scheme; the second computation time is used to indicate the time required to calculate the switching path of the optical switching module under the corresponding port connection scheme; and, based on the port connection scheme corresponding to the shortest computation time, determine that the first server will send data to the second port of the second server through the first port, or determine that the first server will receive data from the second port of the second server through the first port.

[0028] In this embodiment, the ports of multiple servers in the server cluster are connected through multiple optical switching modules. Different port connection schemes correspond to different switching schemes of the multiple optical switching modules, and the computational complexity of different switching schemes of the optical switching modules (e.g., the computational complexity of determining the switching scheme of the lens array within the module based on changes in port connection relationships) is different. Therefore, the second computation time (which affects the total time for port allocation) is different for different port connection schemes. Among multiple port connection schemes, the scheme with the shorter second computation time can be determined, thereby reducing the total port allocation time and improving port allocation efficiency.

[0029] In one alternative implementation, the server cluster includes a first cross-server communication domain, which comprises accelerators within different servers. The management module is further configured to, based on model information and network topology, determine a first port and a second port for transmitting data within the first cross-server communication domain, which includes accelerators within the first server and accelerators within the second server.

[0030] In one alternative implementation, the server cluster further includes a second cross-server communication domain, which comprises accelerators within different servers. The management module is also configured to, based on model information and network topology, determine which ports are used to transmit data from the first cross-server communication domain and the second cross-server communication domain, wherein the second cross-server communication domain includes accelerators from the first server and the second server.

[0031] In this embodiment, data from different cross-server communication domains on the first server are transmitted using the same port (the first port). This port allocation scheme is called port multiplexing. Traffic from multiple cross-server communication domains can share the bandwidth of the same port, thereby reducing bandwidth waste.

[0032] Secondly, embodiments of this application provide a port allocation method. This port allocation method includes: acquiring model information of a model and the network topology of multiple servers, wherein the model runs on a server cluster, the server cluster includes multiple servers, including a first server and a second server, wherein the first server includes multiple accelerators and a first port, and the second server includes multiple accelerators and a second port, the model information being used to indicate the data communication relationship between the accelerators contained in each server; determining, based on the model information and network topology, whether any server in the server cluster has data communication with an accelerator of the first server through the first port; and when it is determined that the first server will send data to the second port of the second server through the first port, or when it is determined that the first server will receive data from the second port of the second server through the first port, creating a correspondence between the first port and the second port.

[0033] The port allocation method described in the second aspect can be implemented by the management module described in the first aspect.

[0034] In one optional implementation, the step of determining whether any server in the server cluster has data communication with the accelerator of the first server through the first port of the first server, based on model information and network topology, may specifically include: determining whether any server in the server cluster has data communication with the accelerator of the first server based on model information; and, based on the existence of any server having data communication with the accelerator of the first server, determining whether any server communicates with the accelerator of the first server through the first port according to the network topology.

[0035] In one optional implementation, after determining whether any server in the server cluster has data communication with the accelerator of the first server based on the model information, the method further includes: determining the collective communication algorithm used by the accelerators of the first server that need to communicate with the second server through the first port of the first server, based on the fact that the second server has data communication with the accelerator of the first server through the first port of the first server; and determining, based on the network topology and the collective communication algorithm, whether the first server will send data to the second port of the second server through the first port or whether the first server will receive data from the second port of the second server through the first port.

[0036] In one alternative implementation, the model information is also used to indicate the data communication volume between the accelerators contained in each server, and the network topology includes the number of ports contained in each server in the server cluster. Before creating the mapping between the first port and the second port, the method further includes: based on the existence of data communication between any server and the accelerator of the first server, determining the number of first ports used for data communication with any server, according to the data communication volume between the accelerators and the number of ports contained in each server; there may be multiple first ports. The step of creating the mapping between the first port and the second port may specifically include: creating multiple mappings between the first port and the second port.

[0037] In one alternative implementation, the network topology further includes the bandwidth supported by the ports contained in each server in the server cluster; and the step of determining the number of first ports for data communication with any one server based on the data communication volume between accelerators and the number of ports contained in each server. Specifically, this step may include: determining the number of first ports for data communication with any one server based on the data communication volume between accelerators, the number of ports contained in each server, and the bandwidth supported by the ports contained in each server.

[0038] In one optional implementation, before determining the number of first ports for data communication with any server based on the data communication volume between accelerators and the number of ports contained in each server, the method further includes: determining the first computing time of each server in the server cluster based on model information and computing power information of the server cluster, during which model computation and cross-server data communication can be performed synchronously; the step of determining the number of first ports for data communication with any server based on the data communication volume between accelerators and the number of ports contained in each server may specifically include: determining the number of first ports for data communication with any server based on the data communication volume between accelerators, the number of ports contained in each server, and the first computing time of the first server.

[0039] In one alternative implementation, the server cluster further includes a first optical switching module, which is used to switch the state between the first port and the second port to a communicable state.

[0040] In one optional implementation, the server cluster further includes multiple optical switching modules; before creating the correspondence between the first port and the second port, the implementation further includes: determining multiple port connection schemes based on the existence of any server communicating with the accelerator of the first server through the first port of the first server, wherein different port connection schemes indicate that the first port of the first server is connected to different servers in the server cluster; determining a second computation time under each port connection scheme; the second computation time is used to indicate the time required to calculate the switching path of the optical switching module under the corresponding port connection scheme; and determining, based on the port connection scheme corresponding to the shortest computation time, whether the first server will send data to the second port of the second server through the first port, or whether the first server will receive data from the second port of the second server through the first port.

[0041] Thirdly, embodiments of this application provide a port allocation method. This port allocation method is applied to an optical switching module in a server cluster. The server cluster operates on a model, including multiple servers and a management module. The multiple servers include a first server and a second server. The first server includes multiple accelerators and a first port, and the second server includes multiple accelerators and a second port. The port allocation method includes: receiving first information, which indicates the correspondence between the first port and the second port; and creating a communication connection between the first port and the second port based on the first information.

[0042] Optionally, the first information can directly indicate the correspondence between the first port and the second port. The optical switching module can then directly create a communication connection between the first port and the second port based on the first information.

[0043] Optionally, the first information can also indirectly indicate the correspondence between the first port and the second port through the correspondence between the first port / second port and the cross-server communication domain. The optical switching module can determine the correspondence between the first port and the second port based on the correspondence between the first port / second port and the cross-server communication domain, thereby creating a communication connection between the first port and the second port. This application does not limit this.

[0044] In one alternative implementation, the first information is determined based on the model information of the model and the network topology of multiple servers.

[0045] In one optional implementation, the network topology includes the number of ports contained in each server of the server cluster. Optionally, the number of ports is used to determine the port allocation ratio for different cross-server communication domains corresponding to multiple accelerators on the server, and the first information is determined based on the port allocation ratio.

[0046] In this embodiment, the port allocation ratio can be determined based on: 1. the data communication volume between the accelerators contained in each server, and 2. the number of ports contained in each server in the server cluster. If multiple accelerators on a server belong to communication domain A and communication domain B, and the data transmission volume ratio between communication domain A and communication domain B is 2:3, but the number of ports on the server is not divisible by 5 (e.g., 12), then the ratio of data transmission volume can be fine-tuned based on the number of ports to obtain the port allocation ratio (e.g., fine-tuned to 5:7).

[0047] In this embodiment, multiple ports on the server typically share the server's optical network bandwidth equally. Based on the number of ports on the server, an appropriate number of ports are allocated to each communication domain to determine the bandwidth ratio allocated to each communication domain, thereby meeting the bandwidth requirements of each communication domain.

[0048] In one alternative implementation, the network topology may include accelerator distribution information, indicating the accelerators on each server. Model information may include parallelism policies, which may indicate the communication domain to which an accelerator belongs.

[0049] In this embodiment of the application, based on accelerator distribution information and parallel strategy, it is possible to determine the communication domain where the accelerator in each server is located, and whether the data in the communication domain needs to be transmitted between servers in the optical network (i.e., whether the communication domain is a cross-server communication domain), thereby determining whether the server needs to reserve a port for the communication domain.

[0050] In one alternative implementation, the network topology includes the bandwidth supported by the ports of each server in the server cluster. The bandwidth supported by the ports is used to determine the port allocation ratio for different communication domains corresponding to multiple accelerators on the server, and the first information is determined based on the port allocation ratio.

[0051] In this embodiment of the application, by combining the optical network communication requirements of the cross-server communication domain where the accelerator in the server is located with the bandwidth supported by the port, the number of ports required for the cross-server communication domain can be determined, thereby determining the port allocation scheme.

[0052] In one alternative implementation, the first information is also used to indicate the set communication algorithm used by the cross-server communication domain corresponding to the first port.

[0053] In this embodiment, based on port allocation according to the cross-server communication domain corresponding to the server port (i.e., the cross-server communication domain to which the accelerator within the server belongs), the port allocation scheme is optimized through multi-dimensional tuning based on the bandwidth allocation ratio between communication domains and the collective communication algorithm of each communication domain to obtain the optimal port allocation scheme. Specifically, the tuning of the collective communication algorithm based on each cross-server communication domain can optimize the traffic distribution between servers within the cross-server communication domain, thereby improving the communication efficiency between servers within the cross-server communication domain.

[0054] In one alternative implementation, the set communication algorithm includes at least one of the following: fullmesh algorithm, ring algorithm, halving and doubling HD algorithm, unbalanced hierarchical ring NHR algorithm, non-uniform data block communication NB algorithm, and pipeline parallel algorithm.

[0055] In one alternative implementation, the network topology includes the optical network connection relationship between the first server and the second server, and the aggregated communication algorithm is determined based on the optical network connection relationship.

[0056] In this embodiment of the application, based on the actual connection relationship between servers, the collective communication algorithm that can be implemented between multiple servers where multiple accelerators belonging to the same cross-server communication domain are located can be determined based on the actual connection relationship, thereby adjusting the collective communication algorithm of the cross-server communication domain and improving the communication efficiency between servers in the cross-server communication domain.

[0057] Fourthly, embodiments of this application provide a port allocation device, which includes one or more units.

[0058] The one or more units are used to implement the port allocation method described in the second or third aspect.

[0059] Fifthly, embodiments of this application provide a port allocation device, which includes: a processor and a memory, the processor being coupled to the memory; the memory being used to store a program; and the processor being used to execute the program in the memory, causing the port allocation device to perform the method described in the first aspect or the second aspect.

[0060] Sixthly, this application provides a computer-readable storage medium storing a computer program that, when executed, implements the methods described in the second or third aspect above.

[0061] In a seventh aspect, a computer program product is provided, which, when executed on a computer, causes the computer to perform the method described in the second or third aspect above.

[0062] The beneficial effects of aspects two through seven are described in aspect one, and will not be repeated here. Attached Figure Description

[0063] Figure 1 A schematic diagram of a parallel strategy for the hybrid parallel training method provided in this application;

[0064] Figure 2 A schematic diagram of another parallel strategy for the hybrid parallel training method provided in this application;

[0065] Figure 3 A schematic diagram of a network architecture for the distributed parallel training method provided in this application;

[0066] Figure 4a A schematic diagram of the optical interconnect topology optimization module provided in an embodiment of this application;

[0067] Figure 4b This is a schematic diagram showing the deployment location of the optical interconnect topology optimization module provided in an embodiment of this application;

[0068] Figure 5 A flowchart illustrating the port allocation method provided in this application embodiment;

[0069] Figure 6a A schematic diagram of an input / output interface of the optical interconnect topology optimization module provided in an embodiment of this application;

[0070] Figure 6b This is a schematic diagram of another input / output interface of the optical interconnect topology optimization module provided in an embodiment of this application;

[0071] Figure 7 A schematic diagram of a parallel strategy provided for an embodiment of this application;

[0072] Figure 8 For corresponding Figure 7 A diagram illustrating the network connectivity of the parallel strategy;

[0073] Figure 9 A schematic diagram of the network connection relationship before and after adjustment of the collective communication algorithm provided in this application embodiment;

[0074] Figure 10 This is a schematic diagram illustrating another network connection relationship before and after adjustment of the collective communication algorithm provided in an embodiment of this application;

[0075] Figure 11 A schematic diagram of the OXC module before and after the connection relationship adjustment provided in the embodiments of this application;

[0076] Figure 12 This is a schematic diagram of another parallel strategy provided in an embodiment of this application;

[0077] Figure 13 For corresponding Figure 12 A schematic diagram of the input / output interfaces of the optical interconnect topology optimization module using a parallel strategy;

[0078] Figure 14a This is a schematic diagram of another parallel strategy provided in an embodiment of this application;

[0079] Figure 14b A timeline schematic diagram of an accelerator provided in an embodiment of this application;

[0080] Figure 15 This is a schematic diagram of an output interface of the optical interconnect topology optimization module provided in an embodiment of this application;

[0081] Figure 16 Another flowchart illustrating the port allocation method provided in this application embodiment;

[0082] Figure 17 A schematic diagram of the structure of a port allocation device provided in an embodiment of this application;

[0083] Figure 18 Another structural schematic diagram of the port allocation device provided in the embodiments of this application;

[0084] Figure 19 Another structural schematic diagram of the port allocation device provided in the embodiments of this application. Detailed Implementation

[0085] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0086] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses. Additionally, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be expressed as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0087] Large models typically refer to machine learning models with a large number of computational parameters and complex computational structures. During the training or use of large models, due to the large number of parameters, the model and / or training data are usually split across different accelerators for collaborative processing; this computational approach is called distributed computing.

[0088] The smallest unit for implementing computation in a distributed computing architecture is an accelerator. An accelerator is a computing unit specifically designed for parallel processing of large amounts of data, and can be called a parallel computing unit or a graphics card device. Accelerators can be CPUs, GPUs, NPUs, chips, bare dies, etc., and this application does not limit them.

[0089] Distributed computing during model training is called distributed training. Distributed training utilizes accelerator clusters to enable machine learning algorithms to better train high-performance models from large datasets. The following example illustrates the process of distributed computing.

[0090] Distributed computing includes parallelism methods such as data parallelism, model parallelism, and hybrid parallelism.

[0091] Data parallelism (DP) refers to multiple accelerators training a complete neural network model, but each accelerator only trains a portion of the training data in the dataset.

[0092] In a data-parallel approach, each accelerator stores a copy of the model parameters. Training data is partitioned and distributed across the accelerators. Each accelerator trains the model using its assigned training data; that is, each accelerator performs forward and backward computations based on the same model parameters to obtain parameter gradients. Once the parameter gradients obtained by each accelerator are synchronized, the model parameters are updated. The assigned training data is typically a batch size.

[0093] Model parallelism (MP) refers to training a portion of a model on multiple accelerators using the full training data.

[0094] In a model-parallel approach, each accelerator stores a subset of the model's parameters. Each accelerator is responsible for computing different parts of the same model, collaboratively training the model. That is, each accelerator can use the same training data to perform forward and backward computations based on different model parameters, obtaining parameter gradients, and then updating the neural network model's parameters based on these gradients.

[0095] Model parallelism includes tensor parallelism (TP), pipeline parallelism (PP), expert parallelism (EP), context parallelism (CP), and sequence parallelism (SP).

[0096] Hybrid parallelism refers to both data parallelism and model parallelism within an accelerator cluster. This addresses the challenges of large-scale training without introducing excessive communication overhead, thereby improving cluster training efficiency.

[0097] In this embodiment, the model / model training data can be partitioned according to certain dimensions (e.g., tensor dimension, pipeline dimension, training data dimension, etc.). The resulting partitioned model / partitioned model training data is run by multiple accelerators. The combination of accelerators running these partitioned models / partitioned model training data is called the parallel domain of the corresponding dimension (e.g., tensor parallel domain, pipeline parallel domain, data parallel domain, where the data parallel domain is the same as the training data parallel domain). Multiple accelerators within a parallel domain share the parameters of that partitioned dimension; for example, multiple accelerators within a tensor parallel domain share the same tensor parameters.

[0098] Specifically, from the perspective of data parallelism, suppose the training dataset is divided into N subsets (each subset is called a batch). Multiple accelerators that train the model using the same subset constitute a data parallel domain (also known as the DP' parallel domain); where N is also called the degree of data parallelism. From the perspective of model parallelism, a model is divided into M parts. Multiple accelerators that run the same part of the model constitute a model parallel domain (also known as the MP' parallel domain); where M is also called the degree of model parallelism.

[0099] For example, Figure 1 A schematic diagram of a parallel strategy for a training cluster that uses a hybrid parallel approach to train a model is shown.

[0100] like Figure 1 As shown, the training data is divided into 4 parts (4 subsets), meaning the data parallelism is 4; the accelerators GPU0-GPU15 are divided into 4 groups according to the data parallelism, each group being a data parallel domain, i.e. Figure 1 The parallel domains DP1' (DP1'), DP2' (DP2'), DP3' (DP3'), and DP4' (DP4') are defined in the data parallel domain. The training data used by an accelerator combination within a single data parallel domain is a subset of the training data, and each subset corresponds one-to-one with a data parallel domain. Accelerator combinations within the same data parallel domain collectively run a complete model.

[0101] like Figure 1 As shown, the model is divided into 4 parts (4 model parts), which means the model parallelism is 4. Figure 1 The 16 accelerators GPU0-GPU15 are divided into 4 groups in a model-parallel manner, with each group forming a model-parallel domain. Figure 1 The model parallel domains are MP1' (MP1'), MP2' (MP2'), MP3' (MP3'), and MP4' (MP4'). An accelerator in a model parallel domain runs one part of the model, and each part of the model corresponds one-to-one with a model parallel domain.

[0102] For example, GPU0, GPU4, GPU8, and GPU12 constitute the DP1' parallel domain. GPU0, GPU4, GPU8, and GPU12 jointly run a complete model. GPU0 belongs to the MP1' parallel domain, GPU4 belongs to the MP2' parallel domain, GPU8 belongs to the MP3' parallel domain, and GPU12 belongs to the MP4' parallel domain. GPU0, GPU1, GPU2, and GPU3 constitute the MP1' parallel domain, all running the same part of the model.

[0103] Similarly, GPU1, GPU5, GPU9, and GPU13 constitute the DP2' parallel domain; GPU2, GPU6, GPU10, and GPU14 constitute the DP3' parallel domain; and GPU3, GPU7, GPU11, and GPU15 constitute the DP4' parallel domain. GPU4, GPU5, GPU6, and GPU7 constitute the MP2' parallel domain; GPU8, GPU9, GPU10, and GPU11 constitute the MP3' parallel domain; and GPU12, GPU13, GPU14, and GPU15 constitute the MP4' parallel domain.

[0104] In distributed computing scenarios, communication between accelerators is inevitable. This communication is achieved through a communication domain, which is a set of accelerators that need to communicate with each other, used for data exchange between parallel domains.

[0105] Taking data parallelism as an example, if 1000 training data points are input into the model for training, and as... Figure 1 The accelerator shown is divided into four data parallel domains DP1'-DP4'. Each of the data parallel domains DP1'-DP4' is trained based on 250 training data points. To obtain the result of the model trained on 1000 training data points, it is necessary to exchange the computation results (e.g., gradient data / gradient parameters) between the four DP' parallel domains. Figure 1 As shown, DP1 communication domain, DP2 communication domain, DP3 communication domain, and DP4 communication domain are set up. The DP communication domain is a combination of accelerators that interact with the gradient data of each DP' parallel domain (the DP communication domain is a combination of accelerators that interact with the gradient data corresponding to different subsets of training data). The arrangement direction of multiple accelerators in the DP communication domain is orthogonal to the arrangement direction of multiple accelerators in the DP' parallel domain.

[0106] Within the DP1 communication domain, the parameter gradients obtained from GPU0, GPU1, GPU2, and GPU3 based on different training data are synchronized, enabling each accelerator to acquire the parameter gradients corresponding to the complete training data. The other DP communication domains work similarly, enabling the interaction of parameter gradients corresponding to different parts of the training data, thereby allowing training of the model's parameters based on the complete training data.

[0107] Taking model parallelism as an example, different parts of the model run in different MP' parallel domains. Taking tensor parallelism (TP) as an example, the model is row-partitioned based on tensors to obtain four tensor parallel domains (i.e., Figure 1In the model, parallel domains MP1'-MP4' are used, where each parallel domain corresponds to a tensor in the model (e.g., MP1' corresponds to tensor A1, MP2' to tensor A2, and so on). The parallel domains MP1'-MP4' calculate the results corresponding to each tensor in the model. To obtain the complete calculation results, the calculation results of the four MP' parallel domains need to be exchanged (e.g., hidden data, i.e., the output data of the hidden layer of the model). For example... Figure 1 As shown, MP1 communication domain, MP2 communication domain, MP3 communication domain, and MP4 communication domain are set up. The MP communication domain is a combination of accelerators that exchange hidden data between the MP' parallel domains. The arrangement direction of multiple accelerators in the MP communication domain is orthogonal to the arrangement direction of multiple accelerators in the MP' parallel domain.

[0108] In this embodiment, the communication domain is an accelerator combination of communication data (or computation results) corresponding to the interactive model parallel domain (e.g., tensor parallel domain TP', pipeline parallel domain PP', expert parallel domain EP', etc.). Different types of parallel domains correspond to different communication data, and the communication domain corresponds to the parallel domain, used to transmit the communication data of the corresponding type of parallel domain.

[0109] For example, the DP communication domain corresponds to the DP' parallel domain (data parallel domain) and is used to transmit gradient data from each DP' parallel domain; the TP communication domain corresponds to the TP' parallel domain (tensor parallel domain) and is used to transmit hidden data from each TP' parallel domain; the PP communication domain corresponds to the PP' parallel domain (pipeline parallel domain) and is used to transmit inter-layer data from each PP' parallel domain; the EP communication domain corresponds to the EP' parallel domain (expert parallel domain) and is used to transmit token data from each EP' parallel domain; the SP communication domain corresponds to the SP' parallel domain (sequence parallel domain) and is used to transmit sequence data from each SP' parallel domain.

[0110] Within the MP1 communication domain, the communication data corresponding to different model parameters on GPU0, GPU4, GPU8, and GPU12 (the communication data corresponding to the various MP' parallel domains mentioned above) are synchronized. The computation results on GPU0, GPU4, GPU8, and GPU12 are then updated based on the synchronized communication data (e.g., the hidden data of tensor A1-Tensor A4 are exchanged in the tensor-parallel MP1 communication domain). The other MP communication domains are similar, enabling the exchange of communication data between different parts of the model's parameters, thereby achieving the training of each part of the model's parameters. It is worth noting that hybrid parallelism can be a combination of two or more dimensions of parallelism. By partitioning the model or its training data based on different parallelism methods, multiple parallel domains and corresponding communication domains can be obtained.

[0111] For example Figure 2 As shown, the accelerator cluster is divided into upper and lower layers based on the training data dimension. The upper-layer accelerators (GPU0, GPU1, GPU4, GPU5) and the lower-layer accelerators (GPU2, GPU3, GPU6, GPU7) train models based on different training data. Thus, the upper and lower layers are different data parallel domains DP1' and DP2'. Figure 2 As shown, the vertical direction, orthogonal to the arrangement direction (up and down direction) of multiple accelerators in the data parallel domain, is the data communication direction of the DP communication domain. Two accelerators in the same vertical column are in the same data communication domain DP. The data communication domain includes DP1, DP2, DP3, and DP4. The DP communication domain corresponds to the DP' parallel domain (data parallel domain) and is used to transmit gradient data from each DP' parallel domain.

[0112] The accelerator cluster is divided into left and right layers along the pipeline dimension (i.e., the model layer dimension). The left-layer accelerators (GPU0, GPU1, GPU2, GPU3) and the right-layer accelerators (GPU4, GPU5, GPU6, GPU7) correspond to different layers of the model (or different steps in the model training process). The left and right layers are then different pipeline parallel domains PP1' and PP2'. The left and right directions, orthogonal to the arrangement direction of multiple accelerators within the pipeline parallel domain (the left-right layer direction), are the data communication directions of the PP communication domain. Multiple accelerators in the same row along the left and right directions are in the same pipeline communication domain PP. The pipeline communication domains include PP1, PP2, PP3, and PP4. The PP communication domain corresponds to the PP' parallel domain (pipeline parallel domain) and is used to transmit inter-layer data between the PP' parallel domains.

[0113] The accelerator cluster is divided into two layers along the tensor dimension: the front-layer accelerators (GPU0, GPU2, GPU4, GPU6) and the back-layer accelerators (GPU1, GPU3, GPU5, GPU7) correspond to different parts of the model tensor, thus forming different tensor parallel domains TP1' and TP2'. The front-back direction, orthogonal to the arrangement direction of multiple accelerators within the tensor parallel domain (the front-back direction), is the data communication direction of the tensor communication domain. Multiple accelerators in the same row along the front-back direction are in the same TP communication domain. The tensor communication domains include TP1, TP2, TP3, and TP4. The TP communication domain corresponds to the TP' parallel domain (tensor parallel domain) and is used to transmit hidden data from each TP' parallel domain.

[0114] Depend on Figure 2 It can be seen that the same accelerator can exist in multiple communication domains. For example... Figure 2The accelerator GPU0 is in the same data communication domain DP1 as GPU2, the same pipeline communication domain PP3 as GPU4, and the same tensor communication domain TP1 as GPU1.

[0115] Accelerator clusters running large-scale models (distributed computing) can be set up on server clusters. A server cluster consists of multiple servers and a management module. Servers within the cluster communicate via optical networks, and each server includes an electrical network. Communication between accelerators within the server cluster is achieved through either optical or electrical networks, thus enabling distributed computing. Electrical networks offer simple communication and high bandwidth, but suffer from low reliability and high cost. Low-cost, high-reliability optical networks are becoming the trend in distributed computing development.

[0116] Deploying a large-scale (distributed computing) accelerator cluster in an optical network requires setting up multiple computing nodes and connecting them via the optical network. Each computing node includes an optical module and one or more accelerators. The one or more accelerators on a computing node communicate with accelerators on other computing nodes through the optical module.

[0117] Figure 3 This is a schematic diagram of an optical network structure. The optical network includes multiple servers, and each server includes an optical module and one or more accelerators. Communication between accelerators on different servers is achieved through the optical module. In this embodiment, the server is also referred to as a computing node.

[0118] In a server cluster deploying large models (distributed computing), multiple accelerators within the same communication domain need to communicate with each other to achieve complete training of the parameters for the corresponding dimension of the communication domain. The communication domain to which an accelerator belongs within a server can be categorized into two types:

[0119] One scenario is that all accelerators within a communication domain reside on the same server, in which case that server does not need to interact with other servers regarding data within that communication domain; this type of communication domain is called an intra-server communication domain. For example... Figure 3 In the TP1 communication domain, all accelerators (GPU0, GPU1) are located within server 1, therefore the TP1 communication domain is the server's internal communication domain.

[0120] Another scenario involves multiple accelerators within a communication domain residing on at least two servers. In this case, communication of data within the communication domain needs to be conducted between at least two servers that include the accelerators in that domain; this type of communication domain is called a cross-server communication domain. For example... Figure 3 In the DP1 communication domain, if accelerator GPU0 is located in server 1 and GPU2 is located in server 3, then the DP1 communication domain is a cross-server communication domain.

[0121] Network topology can include accelerator distribution information, indicating the accelerators included in each server. Model information can include parallelism strategies, indicating the communication domain to which an accelerator belongs. Based on the accelerator distribution information and parallelism strategies, it can be determined whether various communication domains in the accelerator cluster (e.g., PP, EP, SP, CP, DP, etc.) belong to intra-server communication domains or cross-server communication domains.

[0122] At least two servers that are part of the same cross-server communication domain are called servers in the same communication domain. For example... Figure 3 In the given information, if server 1 and server 3 include accelerators (GPU0 and GPU2) in the same cross-server communication domain DP1, then server 1 and server 3 are servers in the same communication domain.

[0123] From the perspective of optical network topology, it is impossible to predict the distribution of accelerators within each server (i.e., whether the accelerators within a server belong to a cross-server communication domain, and whether multiple accelerators within a cross-server communication domain are located on at least two servers; or whether the server has servers in the same communication domain, for example, if the accelerators within a server belong to communication domain A, whether the server includes all accelerators in communication domain A; if it only includes some accelerators in communication domain A, then the server also needs to interact with other servers in the same communication domain (including servers with accelerators in communication domain A) to exchange data in communication domain A), and therefore it is impossible to predict the actual communication needs between servers (e.g., whether two servers need to communicate with each other).

[0124] Therefore, to meet the communication needs of all possible accelerator distribution scenarios on different servers in an optical network, it can be assumed that any two servers may have accelerators in the same communication domain, or that any two servers may share a communication domain; that is, communication is required between any two servers. Therefore, multiple servers are typically connected in a full-mesh manner, meaning that any server is connected to all servers in the server cluster except itself. For example... Figure 3 As shown, each server is connected to the other three servers in the optical network. Therefore, in a fullmesh connection scenario, a port for optical communication is reserved between any two servers in the server cluster.

[0125] It is worth noting that, Figure 3 The connections between servers shown are for illustrative purposes only and do not impose a limit on the number of servers' ports. Optical modules can connect to multiple optical fibers through multiple ports. Figure 3 The port is represented by a small square that connects to the optical module. Figure 3 The ports shown can represent one or more ports, for example... Figure 3 Server 1 can connect to server 4 through one or more ports. Figure 3The leftmost port on server 1 is used as an example to represent all ports connected to server 4. This port actually represents one or more ports.

[0126] However, in reality, some servers do not share a common communication domain accelerator, so no traffic passes between these two servers. For example... Figure 3 In this scenario, GPUs 0 and 1 on server 1, and GPUs 6 and 7 on server 4, do not share a communication domain in any dimension (TP, DP, or PP). Therefore, no traffic will pass between server 1 and server 4, resulting in idle ports on the connection between them and wasting communication resources. The same logic applies to the idle ports between server 2 and server 3, which will not be elaborated further here. A shared communication domain can be understood as multiple servers including accelerators within the same communication domain; this communication domain is a cross-server communication domain.

[0127] Inter-accelerator communication during computation can achieve computation masking (i.e., masking the difference between communication and computation time). If the communication ends later than the computation ends, the accelerator will pause computation to wait for communication. The duration of this delay is called the communication exposure time, which leads to a waste of computing resources.

[0128] Each server has limited optical network bandwidth, which is allocated through ports. Unused ports also consume bandwidth. Due to these unused ports, accelerators on the server that need to communicate across nodes receive less allocated bandwidth, leading to increased transmission latency. This can cause communication time to exceed the corresponding accelerator's computation time, thus prolonging the exposure time of communication masking, resulting in wasted computing power and decreased model training / computation efficiency.

[0129] To address the aforementioned issues, this application provides a port allocation method that determines the port connection relationships of multiple servers within a server cluster based on actual communication needs. This avoids reserving server ports for other servers that do not require communication, thereby reducing port vacancy, improving the bandwidth utilization of the optical network, and ultimately enhancing the training / computation efficiency of large models.

[0130] The port allocation method provided in this application embodiment can be implemented by an optical interconnect topology optimization module, which is also called a management module. The optical interconnect topology optimization module can be a control device, network management device, GPU control unit, or software running on the aforementioned devices; this application does not limit this.

[0131] Figure 4a This is an architecture diagram of the optical interconnect topology optimization module provided in an embodiment of this application. Figure 4aAs shown, the optical interconnect topology optimization module includes a model information input interface, an optical interconnect topology information input interface, and an output interface.

[0132] The model information input interface is used to obtain model information. This interface can be open to users, enterprises, server management providers, etc. Server management providers can use this interface to recommend models to users.

[0133] The optical interconnect topology information input interface is used to obtain the network topology of multiple servers in a server cluster. This interface can be an interface exposed to server vendors. Server vendors use it to provide... Figure 3 The example shown is an accelerator cluster service based on optical networks, which enables distributed computing of the model.

[0134] The optical interconnect topology optimization module determines information such as port allocation and communication domain aggregation algorithms based on model information and network topology.

[0135] The optical interconnect topology optimization module also includes sub-modules for optical interconnect port segmentation optimization, optical interconnect communication algorithm optimization, and optical interconnect port multiplexing optimization. The optical interconnect port segmentation optimization sub-module optimizes port segmentation (i.e., the correspondence between ports / between ports and cross-server communication domains) on servers in the optical network. The optical interconnect communication algorithm optimization sub-module optimizes aggregated communication algorithms across server communication domains. The optical interconnect port multiplexing optimization sub-module optimizes port multiplexing schemes on servers.

[0136] The output interface is used to output the aforementioned port allocation and other output information. The output interface can be an interface that is open to users, enterprises, etc.

[0137] In this embodiment, the optical interconnect topology optimization module can be software, and its deployment location can be as follows: Figure 4b As shown. Figure 4b This is a schematic diagram of a server cluster provided in an embodiment of this application.

[0138] like Figure 4b As shown, the server cluster consists of multiple servers, each server including one or more accelerators and optical modules. The sum of the accelerators on all servers constitutes the aforementioned accelerator cluster. The optical modules are used to enable optical network communication between the servers, and each optical module includes multiple ports.

[0139] Optionally, the server cluster may also include optical switching modules, which are used to enable data exchange between multiple servers. Multiple servers and optical switching modules constitute an optical communication network. Optionally, the optical communication network may also include network management / control equipment. The network management / control equipment is used to control the optical switching modules.

[0140] Server vendors can provide server cluster services based on optical networks (i.e., accelerator clusters on multiple servers in an optical communication network). The server vendor's management server may include an accelerator management unit (also known as a unit that controls GPUs) for controlling / managing multiple accelerators in the accelerator cluster.

[0141] The optical interconnect topology optimization module (management module) can be deployed on the accelerator management unit. Optionally, the accelerator management unit can also be deployed on multiple accelerators within the optical communication network. The optical interconnect topology optimization module can also be deployed on network management / control devices in the optical communication network, or on an optical switching module, or on a server in a server cluster; this application does not limit this.

[0142] based on Figure 4a The optical interconnect topology optimization module architecture shown is as follows: Figure 4b The server cluster architecture shown in this application provides a port allocation method. For example... Figure 5 As shown, the method includes:

[0143] 501. The management module obtains model information and network topology of multiple servers. The model information is used to indicate the data communication relationship between the accelerators contained in each server.

[0144] In this embodiment of the application, the management module is... Figure 4a The optical interconnect topology optimization module in the illustrated embodiment will be referred to as the management module in the following text. The server refers to the server in the server cluster.

[0145] The server cluster runs on a model, and the optical interconnect topology optimization module can obtain the model information of the model, as well as the network topology of multiple servers in the server cluster.

[0146] In some embodiments, model information includes the model's parallel strategy (i.e., the parallel domain of the model / model training data partition and the corresponding communication domain). Based on the parallel strategy, the optical interconnect topology optimization module can determine the data communication relationships between the accelerators contained in each server in the server cluster.

[0147] Optionally, the parallel strategy can be the default parallel strategy, with different parallel strategies corresponding to each model. Alternatively, users can select / input their own parallel strategy, which is not limited in this application.

[0148] In some embodiments, the network topology may include information such as the connection relationship between servers in the server cluster, the number of ports (optical ports) contained in each server in the server cluster, the switching latency of the optical switching module in the server cluster, the optical network bandwidth of the server cluster, or the scale or network structure of the optical network.

[0149] The switching delay of an optical switching module indicates the time required for the module to switch optical signal paths. An optical signal path is the transmission path of an optical signal from one server to another in a server cluster. The optical switching module can be an optical cross-connect (OXC) module, where multiple servers in the cluster can connect their ports. In this case, the switching delay refers to the time required for the OXC module to adjust the port connections, and is also the switching delay of the OXC module's internal optical path. It's understandable that the switching delay of an optical switching module may also be excluded from the network topology and be a parameter independent of the network topology.

[0150] The scale of an optical network refers to the number of servers in the optical network, while the network structure refers to the physical connections between the servers.

[0151] In this embodiment of the application, model information can be obtained through Figure 4a The model information input interface of the illustrated embodiment can be obtained, and the network topology can be obtained through... Figure 4a The optical interconnect topology information input interface of the embodiment shown is used to obtain information.

[0152] Specifically, the optical interconnect topology information input interface of the optical interconnect topology optimization module can obtain the network topology from the server vendor's open interface, publicly available network configuration information (including optical and electrical networks), etc. Optionally, the optical interconnect topology information input interface of the optical interconnect topology optimization module can also be connected to any node or network management system of the optical network to determine the network topology of the optical network; this application does not limit this.

[0153] In this embodiment, model information is also used to indicate the parameters required for the model computation process. Model information may include model type, model training configuration parameters, etc. Optionally, model information may also include sample data (e.g., text unit token information). Model information can be obtained by the user. For example, the optical interconnect topology optimization module can display multiple model options to the user (e.g., sparse model, Llama model 2, etc.) and obtain the model selected by the user. Optionally, the optical interconnect topology optimization module can also directly obtain the model-related code information input by the user, and confirm the tensor shape, hierarchical relationship of the model, model configuration file, etc., in the code information as model information. The model training configuration parameters are the specific parameters in the configuration file, such as global batch size, sequence length, zero bubble, recompute, etc. Combining at least some of the above parameters, the model training time can be determined; it can also determine the data communication volume between accelerators when the model runs on a server cluster, etc.

[0154] 502. The management module determines, based on the model information and network topology, whether any server in the server cluster has data communication with the accelerator of the first server through the first port of the first server.

[0155] Model information can indicate the model's parallel strategy. The optical interconnect topology optimization module can determine the multiple communication domains corresponding to multiple accelerators in the model based on the model's parallel strategy. For example, the model's parallel strategy might be as follows: Figure 2 As shown, the multiple communication domains corresponding to the multiple accelerators in the model include data-parallel communication domains DP1-DP4, pipeline-parallel communication domains PP1-PP4, and tensor-parallel communication domains TP1-TP4.

[0156] Accelerators within a server communicate with each other via an electrical network within the server, while accelerators on different servers communicate with each other via an optical network.

[0157] Network topology can include accelerator distribution information, indicating the accelerators on each server. Model information includes parallelism strategies, which can indicate the communication domain to which an accelerator belongs. Based on the accelerator distribution information and parallelism strategies, the optical interconnect topology optimization module can determine the traffic of the electrical network within each server (i.e., the traffic within the communication domain of the server) and the traffic of the optical network between servers (i.e., the traffic across server communication domains). For example... Figure 3Server 1 in the system includes accelerators GPU0 and GPU1. The communication domains of GPU0 and GPU1 are TP1, DP1, DP4, PP3, and PP4. All traffic within the TP1 communication domain is transmitted between GPU0 and GPU1. Therefore, traffic within the TP1 communication domain between GPU0 and GPU1 is transmitted over the internal electrical network of Server 1. TP1 is an intra-server communication domain. Thus, the ports of Server 1 (used for communication with other servers) need to support communication in the DP1, DP4, PP3, and PP4 domains, which are cross-server communication domains.

[0158] Therefore, the optical interconnect topology optimization module can determine that server 1 needs to perform cross-server data communication, meaning that at least one server (outside server 1) in the server cluster has data communication with the accelerator of server 1. Furthermore, this data communication is the cross-server communication domains DP1, DP4 and PP3, PP4 corresponding to the accelerator within server 1.

[0159] Therefore, the optical interconnect topology optimization module needs to determine the port used for cross-server communication domain data communication from among the multiple ports of server 1; this port is also called the first port. For example... Figure 3 In step 502, the first port of server 1 can be the port corresponding to the cross-server communication domains DP1 and DP4 among the multiple ports of server 1, or it can be the port corresponding to the cross-server communication domains PP3 and PP4.

[0160] Optionally, the optical interconnect topology optimization module can determine multiple optional parallel strategies for the model based on model information. Depending on the parallel strategy, the cross-server communication domain corresponding to the port of server 1 may differ.

[0161] Port connections between servers can be achieved through multiple optical switching modules (such as OXC modules) in a server cluster. For example, port 1 of server 1 corresponds to communication domain DP1, and multiple OXC modules can be used to connect port 1 to port 2 of server 2, thereby enabling data communication in communication domain DP1.

[0162] However, the servers included in the cross-server communication domain corresponding to the port of server 1 determined in step 502 (excluding the first server) may be different from the port of server 2 currently connected to the port of server 1. For example, step 502 determines that the communication domain corresponding to the port of server 1 becomes SP1, and the port of server 1 needs to be connected to the port of server 3 within the communication domain SP1. In this case, the OXC module needs to switch the optical signal path so that the optical path connection between the servers changes from the connection between the ports of server 2 and server 2 to the connection between the ports of server 1 and server 3.

[0163] The OXC module internally uses multiple micromirror arrays to reflect optical signals from different ports, thereby achieving connections between specific ports. Before switching optical signal paths, the switching scheme of each micromirror within the micromirror array needs to be determined (the connection relationship between ports can be adjusted by changing the angle of the micromirrors).

[0164] In step 502, based on different parallel strategies, the communication domains corresponding to the ports of server 1 may differ, resulting in different port connection schemes between servers and thus different micromirror switching schemes. Some port connection schemes correspond to simpler micromirror switching schemes, while others correspond to more complex schemes. For example, the current port connection relationships of server 1 include: ports 1 to 4 correspond to the DP communication domain, connecting to ports of servers 2 to 5 respectively to achieve full-mesh connections between servers 1 and 5; port 5 corresponds to the PP communication domain and connects to server 6.

[0165] If port connection scheme 1 is as follows: ports 1 to 4 of server 1 correspond to the DP communication domain; ports 1 and 2 are connected to ports of server 2; and ports 3 and 4 are connected to ports of server 5 to achieve a ring connection between server 1 and server 5; port 5 corresponds to the PP communication domain and is connected to server 6. Then, port connection scheme 1 only requires adjusting the optical signal paths corresponding to ports 2 to 4, making the micromirror switching scheme relatively simple.

[0166] If port connection scheme 2 is as follows: port 1 of server 1 corresponds to the DP communication domain and is connected to the port of server 2; ports 2 to 5 correspond to the PP communication domain and are connected to servers 6 to 9 to achieve full-mesh connection between server 1 and servers 6 to 9, then port connection scheme 2 requires adjusting the optical signal paths corresponding to ports 2 to 5, making the micromirror switching scheme more complex.

[0167] The optical interconnect topology optimization module needs to perform calculations based on the port connection scheme to determine the corresponding micromirror switching scheme. If the micromirror switching scheme corresponding to the port connection scheme is more complex, the calculation time will be longer, and vice versa. In this embodiment, the time required to calculate the switching path of the optical switching module under the corresponding port connection scheme is referred to as the second calculation time.

[0168] Each port connection scheme has a different second computation time. The optical interconnect topology optimization module can determine the optimal port connection scheme based on the shortest computation time among the multiple port connection schemes' corresponding second computation times. Based on this optimal port connection scheme, it then determines the second server and its second port for data communication with the first port. The first server will send data to the second port of the second server through the first port, or the first server will receive data from the second port of the second server through the first port.

[0169] 503. When it is determined that the first server will send data to the second port of the second server through the first port, or when it is determined that the first server will receive data from the second port of the second server through the first port, the management module creates a correspondence between the first port and the second port.

[0170] In step 502, the optical interconnect topology optimization module determines that server 1 needs to perform cross-server data communication, and the corresponding cross-server communication domains, specifically DP1, DP4 and PP1, PP4. Therefore, the optical interconnect topology optimization module can determine from the server cluster other servers (excluding server 1) that need to perform data communication in the same cross-server communication domains (DP1, DP4 and PP1, PP4). In this embodiment, these servers are referred to as the second server.

[0171] For example Figure 3 In this embodiment, step 502 determines that server 1 needs to perform data communication across server communication domains DP1 and DP4. The optical interconnect topology optimization module can determine the other servers (excluding server 1) where the accelerators in the cross-server communication domains DP1 and DP4 are located based on accelerator distribution information in the network topology and a parallel strategy. Specifically... Figure 3 Server 3 in the middle.

[0172] Then, the optical interconnect topology optimization module can determine the first port on server 1 used for data communication across server communication domains DP1 and DP4, and the second port on server 3 (the second server) used for data communication across server communication domains DP1 and DP4.

[0173] Therefore, the optical interconnect topology optimization module can determine whether server 1 will send cross-server communication domain DP1 and DP4 data to the second port of server 3 through the first port, or whether server 1 will receive cross-server communication domain DP1 and DP4 data from the second port of server 3 through the first port, thereby creating a correspondence between the first port and the second port.

[0174] In the embodiments of this application, the correspondence between the first port and the second port can be indicated by first information. The first information can directly indicate the correspondence between the first port and the second port; or it can indirectly indicate the correspondence between the first port / second port and the cross-server communication domain, which is not limited in this application.

[0175] The correspondence between the first port / second port and the cross-server communication domain is also called the port's communication domain information. In this application's embodiments, the communication domain information indicates the cross-server communication domain corresponding to each port on the server. For example... Figure 3 In the example server 1, the communication domain information of some ports is DP1 and DP4, and the communication domain information of some ports is PP3 and PP4.

[0176] In this embodiment, a mapping between the first and second ports is created when it is determined that the server will interact with the second port of the second server via the first port. Port reservations on the server are determined based on actual communication needs, rather than blindly connecting all servers. Compared to the current fullmesh connection method, this avoids wasting port resources caused by servers connecting to other servers that do not need to communicate. It can improve the transmission bandwidth for data interaction between servers, thereby reducing transmission latency between servers, reducing communication exposure time (i.e., the duration where the accelerator's communication time exceeds the computation time), reducing wasted computing power, and thus improving the training / computation efficiency of the model.

[0177] Optionally, in step 502, the number of first ports of the first server determined by the optical interconnect topology optimization module can be multiple. For example, in Figure 3 In the example, there can be multiple DP domain ports and PP domain ports. The number of ports allocated to each cross-server communication domain on server 1 can be determined based on model information and network topology.

[0178] Specifically, model information can indicate the data communication volume between the accelerators contained in each server within a server cluster. The optical interconnect topology optimization module can determine the traffic ratio between each communication domain based on the data communication volume between the accelerators contained in each server. This traffic ratio indicates the communication needs of different communication domains. The data communication volume between accelerators can be determined based on the inherent ratio of communication needs between different communication domains; for example, the traffic in the PP communication domain is typically 4 times that of the DP communication domain, and the traffic in the TP communication domain is typically 10 times that of the PP communication domain. Based on these inherent ratios, in Figure 2 In the parallel strategy shown, the data communication ratio between the DP communication domain accelerator, PP communication domain accelerator and TP communication domain accelerator can be determined to be 4:16:160.

[0179] Based on accelerator distribution information and parallel strategies, the optical interconnect topology optimization module has determined which communication domains are intra-server communication domains and which are cross-server communication domains. Therefore, the optical interconnect topology optimization module can determine the data communication volume ratio between cross-server communication domains based on the traffic ratio between these communication domains. For example... Figure 3 In this study, the traffic ratio between the cross-server communication domains DP and PP can be determined to be 4:16.

[0180] Network topology also includes the number of ports on each server in the server cluster; for example, server 1 has 64 ports. Based on the traffic ratio between different cross-server communication domains and the number of ports on server 1, the bandwidth allocation ratio (also known as the port allocation ratio, i.e., the proportion of ports allocated to different communication domains on the server) between the DP and PP communication domains on server 1 can be determined. Thus, based on the bandwidth allocation (port allocation) between DP and PP on server 1, the transmission of traffic between the DP and PP communication domains is achieved.

[0181] In this embodiment of the application, the cross-server communication domain corresponding to each port on the server is indicated by the communication domain information, thereby indicating the bandwidth allocation ratio (port allocation ratio). For example, on server 1, the communication domain information of ports 1-12 is DP, and the communication domain information of ports 13-64 is PP. The corresponding bandwidth allocation ratio is DP:PP = 12:52.

[0182] It is worth noting that ports on different servers may correspond to the same cross-server communication domain, and the communication domain information may be the same, but the two concepts are not necessarily the same.

[0183] In this embodiment, communication domain information can indicate different parallel domain categories, such as DP communication domain and PP communication domain. Each communication domain category can include multiple communication domains. For example, a DP communication domain can include multiple different DP communication domains of the same category, such as DP1 and DP2. Therefore, in addition to the port allocation on the server (communication domain information corresponding to each port), it is also necessary to determine the servers where multiple accelerators in the same cross-server communication domain reside based on the network topology in order to determine the port connection relationships between the servers and thus the correspondence between ports on different servers.

[0184] In the embodiments of this application, it can be achieved through Figure 4a The output interface of the embodiment shown outputs first information (i.e., the communication domain information corresponding to the port). Figure 6a for Figure 3 The illustrated embodiment shows a schematic diagram of the input and output interfaces. (See diagram for example.) Figure 6aAs shown, all servers are configured with a port allocation of DP:PP = 12:52. For example, on server 1 (optical device 1), ports 1-12 correspond to the DP communication domain, and ports 13-64 correspond to the PP communication domain. Based on the network topology, the accelerators (GPUs 0, 1, 2, and 3) for the DP1 and DP4 communication domains are located on server 1 and server 3, respectively. Furthermore, the accelerators for the DP communication domains on server 1 only include those for the DP1 and DP4 communication domains, and all traffic in the DP communication domains on server 1 is traffic in the DP1 and DP4 communication domains; similarly, the accelerators for the DP communication domains on server 3 only include those for the DP1 and DP4 communication domains, and all traffic in the DP communication domains on server 3 is traffic in the DP1 and DP4 communication domains. Therefore, the 12 ports allocated to the DP communication domains on both server 1 and server 3 are used for communication between the DP1 and DP4 domains. Thus, the 12 DP communication domain ports on server 1 are interconnected with the 12 DP communication domain ports on server 3.

[0185] In this embodiment, the communication domain information can also indicate different communication domains within the same category of cross-server communication domains, such as DP1, DP2, PP1, and PP2. Based on the port allocation on the server (the communication domain information corresponding to each port), the port connection relationship between servers can be directly determined, thereby determining the correspondence between ports on different servers.

[0186] Figure 6b for Figure 3 Another schematic diagram of the input and output interfaces corresponding to the illustrated embodiment. (See diagram below.) Figure 6b As shown, server 1's ports are assigned as DP1, DP4:PP3, PP4 = 12:52, and server 3's ports are assigned as DP1, DP4:PP1, PP2 = 12:52. Therefore, the 12 ports (ports 1-12) of server 1 (optical device 1) assigned to DP1 and DP4 are interconnected with the 12 ports (ports 1-12) of server 3 (optical device 3) assigned to DP1 and DP4. This determines that server 1 will send data to server 2's ports 1-12 through ports 1-12, or server 1 will receive data from server 2's ports 1-12 through ports 1-12. This establishes a mapping between server 1's ports 1-12 and server 2's ports 1-12. Specifically, the traffic of the DP1 and DP4 communication domains on server 1 is shared with the 12 ports allocated to DP1 and DP4 by server 1; the traffic of the DP1 and DP4 communication domains on server 2 is shared with the 12 ports allocated to DP1 and DP4 by server 2.

[0187] Optionally, the network topology may also include the bandwidth supported by the ports of each server in the server cluster. In the process of determining port allocation described above, the number of ports allocated to different communication domains on server 1 can be determined based on the traffic ratio between different cross-server communication domains, combined with the number of ports on server 1 and the bandwidth supported by those ports.

[0188] In server 1, the 12 ports allocated to the DP communication domain can transmit data from both the DP1 and DP4 communication domains. This embodiment does not limit the timing of the transmission of DP1 and DP4 communication domain data from these 12 ports, nor does it specify the particular ports used for transmitting DP1 and DP4 communication domain data. The 52 ports allocated to the PP communication domain are similarly detailed and will not be elaborated upon here.

[0189] In one example, the bandwidth allocation ratio between the cross-server communication domains DP and PP is 12:52. Therefore, based on the bandwidth allocation ratio on server 1, the 64 ports of server 1 can be allocated to the cross-server communication domains DP and PP in a 12:52 ratio. Thus, on server 1, the data-parallel cross-server communication domain DP is allocated 12 ports, and the pipelined cross-server communication domain PP is allocated 52 ports.

[0190] In step 503, 12 of the 64 ports on server 1 correspond to the DP1 and DP4 communication domains, and 52 ports correspond to the PP3 and PP4 communication domains. These 12 ports on server 1 corresponding to the DP1 and DP4 domains are connected to the 12 ports on server 3 corresponding to the DP1 and DP4 domains, thus establishing a correspondence between the 12 ports on server 1 and the 12 ports on server 3 corresponding to the DP1 and DP4 communication domains. Similarly, these 52 ports on server 1 corresponding to the PP3 and PP4 domains are connected to the 52 ports on server 2 corresponding to the PP3 and PP4 domains, thus establishing a correspondence between the 52 ports on server 1 and the 52 ports on server 2 corresponding to the PP3 and PP4 communication domains.

[0191] Optionally, the network topology may include the optical network bandwidth of the server cluster, which indicates the upper limit of the optical network traffic that can be transmitted between multiple servers. If, based on the current parallel strategy, the traffic of the servers in the optical network (i.e., the sum of traffic in each communication domain) is greater than the upper limit of the optical network traffic, the parallel strategy can be adjusted to reduce the transmission demand (traffic volume) of the optical network, thereby improving the transmission efficiency of the optical network.

[0192] Steps 502 and 503 above can be implemented by the optical interconnect port splitting optimization submodule of the optical interconnect topology optimization module. Optionally, after step 503, the optical interconnect topology optimization module can also output the above-mentioned first information. The output of the first information can be implemented by the output interface of the optical interconnect topology optimization module.

[0193] Optionally, the port allocation in step 503 (determining the first port and the second port) can also be optimized based on a collective communication algorithm. In this embodiment, in addition to the port allocation based on the bandwidth allocation ratio between each cross-server communication domain, the port allocation can also be determined based on the collective communication algorithm of each cross-server communication domain. According to the network topology, the connection relationship between multiple servers where multiple accelerators in the same cross-server communication domain are located can be determined, thereby determining the collective communication algorithm between multiple servers in that cross-server communication domain.

[0194] For example Figure 7 The parallel strategy shown corresponds to the network topology as follows: Figure 8 As shown. Taking the PP1 communication domain where GPU14 is located in server 8 as an example, as... Figure 7 As shown, the PP1 communication domain includes accelerators GPU2, GPU6, GPU10, and GPU14. Figure 8 As shown, accelerators GPU2, GPU6, GPU10, and GPU14 are located on servers 5, 6, 7, and 8, respectively. The current connection between servers 5, 6, 7, and 8 is the default fullmesh connection. In step 503, after determining the number of ports allocated to each cross-server communication domain on the servers (the number of first ports), the optical interconnect topology optimization module can select a suitable collective communication algorithm for the cross-server communication domain based on this number of ports. A collective communication algorithm can only be assigned to a cross-server communication domain if the number of ports allocated to the cross-server communication domain on the servers meets the port requirements of that algorithm. The port requirements for each collective communication algorithm are shown in Table 1, where n represents the number of servers included in the cross-server communication domain.

[0195] Table 1 Port Quantity Requirements for Aggregate Communication Algorithms

[0196] Collective Communication Algorithm Ring Mesh HCCL-HD Time-division multiplexing HD Port requirements 2 n-1 <![CDATA[log2n]]> n / 2 Bandwidth utilization 100％ 100％ <![CDATA[1 / log2n]]> 100％

[0197] For example Figure 8In the illustrated embodiment, it is assumed that each server is allocated 3 ports to the PP communication domain, and the number of servers within the PP communication domain is 4. Therefore, the currently allocated number of ports meets the port requirements of the ring, mesh, HCCL-HD, and time-division multiplexing HD collective communication algorithms, and any one of these algorithms can be selected. Since the preferred collective communication algorithm for the PP communication domain is the ring algorithm, the ring algorithm is chosen as the collective communication algorithm between servers 5, 6, 7, and 8 in the PP1 communication domain.

[0198] It is worth noting that in other communication domains (such as DP, TT, EP, etc.), aggregate communication algorithms such as fullmesh, ring, halve and double (HD), nonuniform hierarchical ring (NHR), nonuniform bruck (NB), and pipeline parallelism can be selected, and this application does not limit them.

[0199] Because in the ring algorithm, a single server only needs to connect to two adjacent servers within the same cross-server communication domain, therefore... Figure 9 As shown, after the aggregation communication algorithm within the PP1 communication domain is changed from fullmesh to ring, the ports originally connected to server 6 on server 8 that were allocated to PP1 can be reassigned to connect to server 5 or server 7. In one example, the PP1 communication domain on server 8 is allocated 16 ports. Figure 9 As shown in the upper middle diagram, if it is a fullmesh collective communication algorithm, the ports of the 16 PP1 communication domains on server 8 are allocated to servers 5, 6, and 7 in the PP1 communication domain in a 5:5:5 ratio. The remaining port in the PP1 communication domain on server 8 is left unused, resulting in a waste of port and bandwidth.

[0200] like Figure 9 As shown in the figure below, if the collective communication algorithm in the PP1 communication domain is adjusted to the ring algorithm, the 16 ports of the PP1 communication domain on server 8 will be allocated to the adjacent servers 5 and 7 in an 8:8 ratio. All ports in the PP1 communication domain on server 8 will be used for signal transmission, which can reduce the waste of ports and bandwidth.

[0201] Similarly, the collective communication algorithms for the DP4, DP5, and PP2 communication domains on server 8 can be determined, thereby identifying the servers connected to the multiple ports allocated to DP4, DP5, and PP2 on server 8 and their corresponding collective communication algorithms. This leads to the determination of the port allocation scheme for all servers on server 8. The same logic applies to other servers, thus obtaining the port allocation scheme for all servers in the accelerator cluster.

[0202] like Figure 10 As shown, in another example, the DP1 communication domain includes accelerators GPU0-GPU7, which are located on servers 1-8 respectively. Figure 10 As shown in the diagram above, servers 1-8 currently use the default fullmesh connection method, with each server connected to the other 7 servers. In this embodiment, the aggregate communication algorithm for the DP1 communication domain can be selected as the halving and doubling HD algorithm; therefore, the aggregate communication algorithm between servers 1-8 in the DP communication domain is selected as the HD algorithm.

[0203] In the HD algorithm, a single server only needs to connect to log₂n (where n is the number of servers housing multiple accelerators within the same cross-server communication domain) servers. In this embodiment, n = 8, therefore a single server only needs to connect to 3 servers within the same cross-server communication domain. Therefore, as... Figure 10 As shown by the dashed line in the lower part of the diagram, after the aggregation communication algorithm within the DP1 communication domain is changed from fullmesh to HD algorithm, the ports originally connected to servers 4, 6, 7, and 8 on server 1 can be reassigned to connect to servers 2, 3, or 5. In one example, the DP1 communication domain on server 1 is allocated 9 ports. Figure 10 As shown in the upper diagram, if it is a fullmesh aggregation communication algorithm, the ports of the 9 DP1 communication domains on server 1 are allocated to servers 2-8 in the DP1 communication domain in a ratio of 1:1:1:1:1:1:1. The remaining 2 ports in the DP1 communication domain on server 1 are left unused, resulting in a waste of ports and bandwidth.

[0204] like Figure 10 As shown in the diagram below, if the aggregated communication algorithm within the DP1 communication domain is adjusted to the HD algorithm, the nine ports of the DP1 communication domain on server 1 will be allocated to servers 2, 3, and 5 within the DP1 communication domain in a 3:3:3 ratio. All ports within the DP1 communication domain on server 1 will be used for signal transmission, reducing port and bandwidth waste. The same logic applies to the other servers, thus obtaining the port allocation scheme for all servers in the accelerator cluster.

[0205] The optical interconnect topology optimization module can simulate the computation time of each accelerator in the accelerator cluster of the optical network, as well as the communication time between each accelerator (including the electrical communication time between accelerators in the server and the optical communication time between accelerators in the server) based on the model training information (model training parameters, etc.) in the model information, thereby determining the model training time corresponding to the port allocation scheme.

[0206] The optical interconnect communication algorithm optimization submodule of the optical interconnect topology optimization module is used to optimize port allocation based on the cross-server communication domain aggregated communication algorithm. That is, the process of determining the aggregated communication algorithm in step 502 can be implemented by the optical interconnect port segmentation optimization submodule.

[0207] In the process of determining the bandwidth allocation ratios and the collective communication algorithms for different communication domains, we can list all possible combinations of bandwidth allocation ratios (also known as port allocation ratios) and collective communication algorithms for different server communication domains, thus obtaining multiple port allocation schemes. The optical interconnect topology optimization module can determine the model training time based on different port allocation schemes, resulting in the model training time table shown in Table 2.

[0208] Table 2 shows the model training time corresponding to different bandwidth allocation ratios and port allocation schemes for the ensemble communication algorithm on server 5.

[0209]

[0210] Table 1 shows the port allocation scheme corresponding to the minimum model training time (in bold in Table 1), which is then used as the port allocation scheme output by the optical interconnect topology optimization module. This process is called comprehensive optimization of the bandwidth allocation ratio and the ensemble communication algorithm, which involves multi-faceted adjustments to the bandwidth allocation ratio and the ensemble communication algorithm with the model training time as the optimization objective. The optical interconnect port segmentation optimization submodule and the optical interconnect port segmentation optimization submodule achieve optimization of port allocation and the ensemble communication algorithm through the above comprehensive optimization.

[0211] In determining the model training duration, the initial computation time for each server in the server cluster can be determined based on the model information and the computing power information of the server cluster. The computing power information represents the number of accelerators contained in each server within the server cluster and the operating frequency of the accelerators.

[0212] The operation of the accelerator within the server includes both computation and communication processes. Some data communication needs to be completed before the computation begins. If this data communication process is performed concurrently with the computation process of the previous cycle, the accelerator's waiting time for this data communication can be reduced, allowing the computation process of the current cycle to begin earlier. In this embodiment, the cycle length of the computation process is referred to as the first computation duration. When the communication process duration is less than or equal to the first computation duration, complete computational masking (i.e., communication time is masked within the computation time, allowing for continuous computation) can be achieved. The first computation duration is also called the computational masking duration, during which model computation and cross-server data communication can be performed simultaneously.

[0213] In this embodiment, the data communication volume refers to the actual amount of data to be transmitted, and the first computation duration is the duration of each computation cycle of the accelerator. Based on the data communication volume and the first computation duration, the bandwidth requirement for achieving complete computational masking can be determined. The first computation duration is the window duration for achieving computational masking of the corresponding communication domain. As long as the communication duration of the communication domain is controlled within this window duration, computational masking can be achieved. The communication duration of the communication domain depends on the number of corresponding ports; the larger the number of ports, the smaller the corresponding communication duration. The size of the first computation duration is determined by the proportion of computational load within the computation cycle to the chip's computational capacity. The larger the computational load corresponding to the communication domain, the larger the first computation duration, and the longer the computational masking window time, allowing for the allocation of fewer ports to that communication domain. Conversely, the larger the first computation duration of the communication domain, the shorter the corresponding communication masking window time, requiring the allocation of more ports. Through computational masking, the computational resources wasted by the accelerator due to waiting for communication (these computational resources are actually the accelerator's time resources) are reduced, thereby improving the computational efficiency of the server cluster.

[0214] Therefore, after determining the number of ports corresponding to each communication domain in step 503, the optical interconnect topology optimization module can also optimize the number of first ports of the first server based on the data communication volume between accelerators, the number of ports contained in each server, and the first computing time of the first server.

[0215] In reality, since the number of server ports may not be divisible by the determined optimal port allocation, the actual bandwidth allocation ratio may be equal to or close to the determined optimal port allocation ratio (bandwidth allocation ratio).

[0216] In this embodiment, the port allocation of each server is associated with the cross-server communication domain to which the accelerator within the server belongs. This ensures that the server's port only connects to other servers within the same communication domain, avoiding the waste of port resources caused by connecting servers to servers in different communication domains. This can increase the transmission bandwidth of accelerators in the shared communication domain between servers, thereby reducing transmission latency between servers, reducing communication exposure time, reducing wasted computing power, and thus improving the training / computation efficiency of the model.

[0217] For example Figure 3 As shown, in the original fullmesh optical network connection scheme, no traffic passes between server 1 and server 4, and between server 2 and server 3. The ports connecting server 1 and server 4, and between server 2 and server 3, are left unused, resulting in wasted communication resources. Based on the port allocation scheme provided in this application embodiment, all servers connected to server 1 share the same communication domain as server 1 (i.e., share the same communication domain as the accelerator within server 1). Server 1 will not connect to server 4, thus avoiding port idleness. The same applies to servers 2 and 3, which will not be elaborated here.

[0218] In this embodiment, based on port allocation according to the cross-server communication domain corresponding to the server (i.e., the cross-server communication domain to which the accelerator within the server belongs), the port allocation scheme is optimized through multi-dimensional tuning based on the bandwidth allocation ratio between cross-server communication domains and the collective communication algorithm of each communication domain to obtain the optimal port allocation scheme. Specifically, tuning the bandwidth allocation ratio between cross-server communication domains can meet the transmission needs of each cross-server communication domain (allocating sufficient ports for transmission to each cross-server communication domain), thereby achieving targeted resource allocation and improving the utilization rate of communication resources. Tuning the collective communication algorithm of each cross-server communication domain can optimize the traffic distribution between servers within the cross-server communication domain, thereby improving the communication efficiency between servers within the cross-server communication domain.

[0219] For example Figure 8 As shown, the PP1 communication domain includes accelerators GPU2, GPU6, GPU10, and GPU14, located within servers 5, 6, 7, and 8, respectively. Therefore, servers 5, 6, 7, and 8 need to exchange data within the PP1 communication domain. Figure 8 In the architecture shown, servers 5, 6, 7, and 8 are connected in a full-mesh configuration, meaning each server is connected to the other three servers within the PP1 communication domain. See the corresponding connection structure for details. Figure 9The diagram above illustrates this. For a single server, the port allocated to PP1 needs to be divided into three parts to connect to three servers within the PP1 communication domain. If the aggregate communication algorithm within the PP1 communication domain is adjusted to the ring algorithm, then a single server only needs to connect to the two nearest servers within the PP1 communication domain. Figure 9 As shown in the diagram below, the ports allocated to the PP1 communication domain only need to be divided into two parts to connect to the servers within the PP1 communication domain. Increasing the number of ports allocated to a single server within the PP1 communication domain can improve the communication bandwidth within the PP1 communication domain.

[0220] It is worth noting that in this embodiment of the application, multiple servers in the optical network are interconnected via optical switches. The optical switches may include optical cross-connect (OXC) modules. Figure 3 The optical network architecture shown, and the corresponding OXC connection relationships are as follows: Figure 11 As shown in the diagram above, the default connection between different servers in the current accelerator cluster's optical network is a fullmesh connection. Figure 11 Ports 1 through 12 in the diagram can each represent one or more ports. The connection relationship between any two ports from 1 to 12 is used to schematically represent one or more communication links between servers. For example... Figure 11 As shown in the figure below, the optical path inside the OXC module can be adjusted based on the port allocation scheme provided in the embodiments of this application, thereby adjusting the connection relationship between ports of different servers and realizing the connection between ports in the same communication domain.

[0221] For example Figure 3 As shown, there is no traffic between server 1 and server 4, and no traffic between server 2 and server 3. Traffic flows between server 1 and server 2, server 3 and server 4, server 1 and server 3, and server 2 and server 4. Therefore... Figure 11 As shown in the diagram below, by switching the connection relationship between the ports using OXC, port 1, which was originally connected to server 4, is switched to port 6, which is connected to server 2, thereby enabling communication between server 1 and server 2; port 12, which was originally connected to server 1, is switched to port 7, which is connected to server 3, thereby enabling communication between server 4 and server 3.

[0222] In step 503 above, the optical interconnect topology optimization module can determine the connection relationship between ports on different servers (i.e., the actual connection relationship between the first port and the second port) based on the network topology and the communication domain information corresponding to each port.

[0223] Optionally, this step can also be implemented by the OXC module. After determining the communication domain information of the server's ports, the optical interconnect topology optimization module sends the communication domain information to the OXC module. The OXC module combines the communication domain information with the network topology to determine the connection relationships between multiple ports of different servers (i.e., the actual connection relationship between the first port and the second port).

[0224] Optional, Figure 5 Step 503 of the illustrated embodiment can also be optimized based on port multiplexing. In this embodiment, traffic from multiple cross-server communication domains can be multiplexed and transmitted on the same port to save port (bandwidth) resources. Specifically, in step 503, a traffic multiplexing scheme for multiple cross-server communication domains (i.e., traffic from multiple cross-server communication domains is multiplexed and transmitted on one or more ports) can be determined according to the network topology of the optical network. In the table shown in Table 1, the dimension of traffic multiplexing scheme is added, and the bandwidth allocation ratio, the aggregated communication algorithm, and the traffic multiplexing scheme are comprehensively optimized. Based on the optimized bandwidth allocation ratio, the optimized aggregated communication algorithm, and the optimized traffic multiplexing scheme, the communication domain information corresponding to each of the multiple ports on the server is obtained.

[0225] For example Figure 12 In this approach, the accelerator cluster is divided into upper and lower layers based on the training data dimension. These upper and lower layers represent different data parallel domains. The upper and lower directions, orthogonal to the arrangement direction of multiple accelerators within the data parallel domain (upper and lower layer directions), constitute the direction of the DP communication domain. Two accelerators in the same upper and lower column are in the same DP communication domain.

[0226] By segmenting the accelerator cluster along the context dimension, it is divided into four layers, left and right, which are then different context parallel domains. The left and right directions, orthogonal to the arrangement direction of multiple accelerators within the context parallel domain (left and right layer directions), are the directions of the CP communication domain. Multiple accelerators in the same row along the left and right directions are in the same CP communication domain.

[0227] By partitioning the accelerator cluster along the tensor dimension into two layers, the front and back layers become different tensor parallel domains. The front-back direction, orthogonal to the arrangement direction of multiple accelerators within the tensor parallel domain (front-back direction), is the direction of the TP communication domain. Multiple accelerators in the same row along the front-back direction are in the same TP communication domain.

[0228] The accelerator cluster is segmented along the expert dimension into eight columns of accelerator groups arranged in a front-to-back pattern (the eight columns are: GPU0+GPU1, GPU2+GPU3, GPU4+GPU5, ...). These eight columns represent different expert parallel domains. The front-to-back layer direction orthogonal to the arrangement direction (front-to-back direction) of multiple accelerators within the expert parallel domain (e.g., the direction of the layer containing GPU0, GPU2, GPU4, GPU6, GPU8, GPU10, GPU12, and GPU14) is the direction of the expert parallel communication domain. Multiple accelerators in the same layer along the front-to-back layer direction are in the same expert parallel communication domain (EP).

[0229] If server 1 includes Figure 12 The accelerators are GPU0 and GPU2. Since GPU0 resides in the communication domain DP1+CP3+EP1+TP1, it is simultaneously in both communication domains DP1 and EP1, and the data transmission direction in both domains is consistent. Therefore, data from GPU0 within communication domains DP1 and EP1 can be merged and transmitted. For example... Figure 13 As shown, the communication domain corresponding to port 1 in server 1 (opticaldevice 1) is DP+EP, which is an overlap of the DP communication domain and the EP communication domain. port1 transmits data from both the DP and EP communication domains.

[0230] If the number of accelerators in the EP communication domain equals the number of accelerators in the DP communication domain within the same layer multiplied by the number of accelerators in the CP communication domain. For example... Figure 12 In this context, the number of accelerators in EP1 = 8 = 4 (number of accelerators in CP1 communication domain / number of accelerators in CP3 communication domain) * 2 (number of accelerators in DP1 communication domain / DP2 communication domain / DP3 communication domain / DP4 communication domain). This situation is called complete overlap, meaning that all accelerators in the DP communication domain / CP communication domain are also in the EP communication domain. Taking server 1 as an example, if the ports on server 1 are fully multiplexed, then the communication domain information corresponding to all ports on server 1 is either DP communication domain + EP communication domain or CP communication domain + EP communication domain.

[0231] Ports on the server may also be partially reused. If there are a large number of experts, for example... Figure 12There are four experts. Expert 1's communication domain EP1 contains GPU0, GPU4, GPU8, and GPU12; Expert 2's communication domain EP2 contains GPU1, GPU5, GPU9, and GPU13; Expert 3's communication domain EP3 contains GPU2, GPU6, GPU10, and GPU14; and Expert 4's communication domain EP4 contains GPU3, GPU7, GPU11, and GPU15. Therefore, the number of accelerators in the EP1 communication domain is 4 < 4 (the number of accelerators in the CP1 / CP3 communication domains at the same layer as EP1) * 2 (the number of accelerators in the DP1 / DP2 / DP3 / DP4 communication domains at the same layer as EP1). This situation is called partial overlap, meaning that some accelerators in the DP / CP communication domains are also located in the EP communication domain.

[0232] Corresponding to partial multiplexing of ports, Figure 13 for Figure 4a This is a schematic diagram of the input and output interfaces of the optical interconnect topology optimization module. (See diagram for example.) Figure 13 As shown, port 1 within server 1 (optical device 1) corresponds to the DP+EP communication domain, which is an overlap of the DP and EP communication domains. Port 1 transmits data from both the DP and EP communication domains. Port 2 corresponds to the DP communication domain and transmits data from the DP communication domain; this port is not shared with the EP communication domain. Therefore... Figure 13 The server 1 shown is a partial port multiplexing (i.e., some ports are multiplexed for EP communication domain and DP / CP communication domain).

[0233] like Figure 13 As shown, the input interfaces include a model information input interface and an optical interconnect topology information input interface. The model information input interface is used to obtain model information, including model type, model training configuration parameters, sample data, etc. The optical interconnect topology information input interface is used to obtain the network topology of the optical network. The network topology includes the connection relationships between servers in the optical network, or the number of ports (portsize) of each server, or the port bandwidth (bandwidth) of each server, or the accelerators included in each server, or the optical path switching time (switchtime), or wavelength, etc.

[0234] The output interface includes port splitting output ports and communication algorithm output ports. The port splitting output ports are used to output the aforementioned first information, namely, the communication domain information corresponding to multiple ports on each server, for example... Figure 13In this diagram, port 1 of server optical device 1 corresponds to both the DP and EP communication domains. The communication domain information corresponding to a port indicates the port's connectivity; that is, when ports on different servers correspond to the same communication domain, the ports are interconnected. The connectivity between ports on different servers can be determined based on the servers housing multiple accelerators in the same communication domain within the network topology.

[0235] Figure 13 This is a schematic diagram of the input and output interfaces. Optional, such as... Figure 13 As shown, the output interface may also include a parallel strategy output interface, used to output the number of communication domains used to transmit data from each parallel domain. For example... Figure 13 In this context, the number of TP communication domains is 8, the number of PP communication domains is 11, and so on.

[0236] In this embodiment, data from different communication domains that are transmitted in the same direction on the same server are allocated to the same port for transmission (referred to as port multiplexing). Traffic from multiple communication domains can share the bandwidth of the same port, thereby reducing bandwidth waste.

[0237] In this embodiment, multiple ports on the server can be reused, meaning all ports on the server correspond to multiple communication domains. For example, some ports on the server correspond to the data-parallel communication domain DP, some ports correspond to the context-parallel communication domain CP, and all ports correspond to the expert-parallel communication domain EP. Therefore, all ports can be reused (some ports are reused in both DP and EP communication domains, and some ports are reused in both CP and EP communication domains). Since the EP communication domain has high traffic and low memory overhead, setting all ports to reuse both EP and DP / CP communication domains reduces the amount of communication data and improves load balancing.

[0238] In this embodiment, some ports on the server can be multiplexed, meaning some ports on the server correspond to multiple communication domains, while others correspond to only one communication domain. For example, some ports on the server may be multiplexed using EP and DP communication domains or EP and CP communication domains, while others may correspond to only one communication domain. On the one hand, the EP communication domain has a large traffic volume, so multiplexing the EP communication domain with the DP / CP communication domain can distribute most of the traffic of the EP communication domain to the optical network outside the server, reducing the consumption of high-speed bandwidth in the server's internal electrical network. On the other hand, since the number of EP communication domains is limited, the number of EP communication domains is relatively small, therefore only some ports can be multiplexed using the EP communication domain with the DP / CP communication domain.

[0239] The optical interconnect port multiplexing optimization submodule of the optical interconnect topology optimization module is used to optimize port allocation based on the port multiplexing scheme of the ports on the server. That is, the process of determining the port multiplexing scheme in step 503 can be implemented by the optical interconnect port multiplexing optimization submodule.

[0240] Optionally, port multiplexing can also be implemented based on the mutual isolation of communication times between different communication domains. For example... Figure 14a As shown, the server cluster includes servers 0 through 5. Server 0 includes accelerator GPU 0, server 1 includes accelerator GPU 1, and so on. The cross-server communication domain SP1 includes GPU 0, GPU 1, and GPU 2; SP2 includes GPU 3, GPU 4, and GPU 5; the cross-server communication domain DP1 includes GPU 0 and GPU 3; DP2 includes GPU 1 and GPU 4; DP3 includes GPU 2 and GPU 5; and the cross-server communication domain EP1 includes the aforementioned GPUs 0 through 5.

[0241] Model computation can include an expert phase and an attention (Attn) phase, such as... Figure 14b As shown, taking GPU 0 as an example, the expert phase is time-isolated from the Attn phase. During the Attn phase, communication in communication domain EP1 takes place; during the expert phase, communication in communication domain SP1 takes place; and after both the expert and Attn phases are completed, communication in communication domain DP1 takes place. Therefore, the communication times of communication domains EP1 and SP1 on GPU 0 are isolated, and the communication times of communication domains EP1 and DP1 are also isolated; that is, communication domain EP1 will not communicate with SP1 or DP1 simultaneously.

[0242] Therefore, the ports on server 1, where accelerator 0 is located, allocated to communication domains SP1 and DP1, can be multiplexed to communication domain EP1. For example, if ports 1-4 on server 1 correspond to communication domain SP1, and ports 5-8 correspond to communication domain DP1, then ports 1-8 can also correspond to communication domain EP1. Outside of the time slots when ports 1-4 transmit data for communication domain SP1, data for communication domain EP1 is transmitted via ports 1-4; similarly, outside of the time slots when ports 5-8 transmit data for communication domain DP1, data for communication domain EP1 is transmitted via ports 5-8. Since data transmission in the EP1 communication domain utilizes the transmission paths (i.e., ports) of the SP1 and DP1 communication domains, and the communication time of the EP1 communication domain is isolated from the communication times of the SP1 and DP1 communication domains, this scheme can utilize previously idle communication resources, improve transmission efficiency, and reduce overall training time.

[0243] In this embodiment of the application, if the optical switch connecting multiple servers is an OXC module, the communication time periods of different communication domains can be distinguished. Through the optical path switching of the OXC module, time-division multiplexing communication of different communication domains can be realized between multiple servers.

[0244] For example Figure 15 As shown, communication between multiple servers is divided into multiple sequentially connected time slots, with different communication domains corresponding to multiple ports of the server in different time slots. For example... Figure 15 In the accelerator cluster, all ports of multiple servers in time slot 1 correspond to the DP communication domain, and all ports of multiple servers in time slot 2 correspond to the PP communication domain. At the beginning of time slot 1, the OXC module adjusts the connection relationships between servers based on the fullmesh algorithm for the DP communication domain, thus enabling data communication in the DP communication domain within time slot 1. At the beginning of time slot 2, the OXC module adjusts the connection relationships between servers based on the P2P algorithm for the PP communication domain, thus enabling data communication in the PP communication domain within time slot 2, and so on, achieving data communication across all communication domains within the accelerator cluster.

[0245] It is worth noting that, Figure 15 This is merely an example and does not constitute a limitation on the communication domain corresponding to a port within a single time slot. For example, within a certain time slot, multiple ports of the server may partially correspond to the DP communication domain and partially to the TP communication domain; this application does not impose any limitations on this.

[0246] In this embodiment of the application, the optimization device containing the optical interconnect topology optimization module can determine its own parallel strategy. For example... Figure 16 As shown, the specific operation process of the optimization module includes:

[0247] 1. The user selects a training model; at the same time, they configure and read information such as the physical topology network.

[0248] The user inputs the model and reads the physical environment configuration; the device reads the network topology and configures the optical interconnect domain parameters (i.e., the network topology of the optical network).

[0249] 2. Input parameterization to build basic AI models and network abstraction models.

[0250] Input parametric modeling involves parametrically processing the user-inputted model for subsequent AI modeling and analysis input.

[0251] The read network topology is parameterized to construct basic network topology models, such as Fat-Tree and Full Mesh.

[0252] 3. Input the model information into the parallel policy analysis module to simulate the end-to-end training process.

[0253] Parallel strategy analysis, through fine-grained modeling of computation and communication operators, concatenates these operators to simulate the training process. This allows for the optimization of the parallel strategy and the output of a superior parallel strategy.

[0254] Specifically, based on the model information and network topology, multiple parallel domains corresponding to the model information, as well as the corresponding communication domains, can be determined.

[0255] Steps 1-3 above can be implemented using the input interface of the optical interconnect topology optimization module. Specifically, this can be achieved through the model information input structure and the optical interconnect topology information input structure.

[0256] 4. Based on the output of the previous steps, optimize the optical switching configuration by combining the optical topology information.

[0257] Based on the forward output, optical interconnect characteristics are modeled, and a parallel strategy with a degradation rate within a tolerable range is selected for optimization.

[0258] Based on the network topology and model characteristics, determine the number of ports, outgoing bandwidth, and port multiplexing. Select the segmentation ratio based on the number of ports, and choose the communication algorithm based on the outgoing bandwidth and multiplexed ports.

[0259] Output the optimal training scheme. This includes: parallel strategies, bandwidth allocation, communication algorithms, port multiplexing strategies, etc.

[0260] Optionally, the parallel domain information corresponding to each of the multiple ports on the server corresponding to the first parallel domain combination can be output; and the parallel domain information corresponding to each of the multiple ports on the server corresponding to the second parallel domain combination can be output.

[0261] Step 4 above can be implemented by the optical interconnect port segmentation optimization submodule, the optical interconnect communication algorithm optimization submodule, and the optical interconnect port multiplexing optimization submodule within the optical interconnect topology optimization module. Specifically, the optical interconnect port segmentation optimization submodule selects the segmentation ratio based on the number of ports; the optical interconnect communication algorithm optimization submodule selects the communication algorithm based on the outgoing frame bandwidth and the multiplexed ports; and the optical interconnect port multiplexing optimization submodule determines port multiplexing.

[0262] 5. Generate resource configuration information based on the search results and compile it into information that can be distributed.

[0263] Based on comprehensive judgment and analysis, the search results are used to generate configuration files that can be distributed, such as the port connection relationships within OXC.

[0264] Once the basic configuration is distributed, its correctness is checked. If a configuration error is found, the user is prompted to modify the input parameters.

[0265] 6. Present the configuration information to the user and allow them to choose whether to start training.

[0266] Detailed configuration information is presented to the user, who then confirms and chooses whether to start the training task.

[0267] The command is issued to send the above results to the PyTorch framework, communication algorithm library, OXC configuration management unit, etc.

[0268] Steps 5-6 above can be implemented by the output interface of the optical interconnect topology optimization module.

[0269] Please see Figure 17 , Figure 17 This is a schematic diagram of the port allocation device provided in an embodiment of this application. If the aforementioned optical interconnect topology optimization module (i.e., management module) is software, then the port allocation device 1600 can be used to run the optical interconnect topology optimization module (management module). Figure 17 As shown, the port allocation device 1600 includes a receiving unit 1601 and a processing unit 1602.

[0270] The receiving unit 1601 is used to: acquire model information of the model and the network topology of multiple servers, wherein the model information is used to indicate the data communication relationship between the accelerators contained in each server.

[0271] The computing unit 1602 is used to: determine, based on model information and network topology, whether any server in the server cluster (in this embodiment, any server refers to any server in the server cluster other than the first server) has data communication with the accelerator of the first server through the first port of the first server.

[0272] The arithmetic unit 1602 is also used to: when it is determined that the first server will send data to the second port of the second server through the first port, or when it is determined that the first server will receive data from the second port of the second server through the first port, create a correspondence between the first port and the second port.

[0273] Optionally, the port allocation device 1600 may further include an output unit 1603. The output unit 1603 is used to output first information. The first information is used to indicate the correspondence between the first port and the second port.

[0274] The receiving unit 1601 is used to perform Figure 5 Step 501 of the illustrated embodiment; the arithmetic unit 1602 is used to execute Figure 5 Steps 502 and 503 of the illustrated embodiment; the output unit 1603 is used to execute Figure 5The process of outputting the first information after step 503 in the illustrated embodiment. The receiving unit 1601 corresponds to the model information input interface and the optical interconnect topology information input interface of the optical interconnect topology optimization module, the computing unit 1602 corresponds to the optical interconnect port segmentation optimization submodule, the optical interconnect communication algorithm optimization submodule (optional), and the optical interconnect port multiplexing optimization submodule (optional) of the optical interconnect topology optimization module, and the output unit 1603 corresponds to the output interface of the optical interconnect topology optimization module.

[0275] Port allocation device 1600 is used to implement the aforementioned Figures 5 to 15 The port allocation method shown.

[0276] In one optional implementation, the computing unit 1602 is specifically used to: determine, based on the model information, whether any server in the server cluster has data communication with the accelerator of the first server; and, based on the existence of any server having data communication with the accelerator of the first server, determine, based on the network topology, whether any server communicates with the accelerator of the first server through the first port.

[0277] In an optional implementation, the computing unit 1602 is further configured to: determine the aggregate communication algorithm used by the accelerators that need to communicate with the first server through the first port of the first server, based on the fact that the second server has data communication with the accelerators of the first server through the first port of the first server; and determine, based on the network topology and the aggregate communication algorithm, whether the first server will send data to the second port of the second server through the first port, or whether the first server will receive data from the second port of the second server through the first port.

[0278] In an optional implementation, the model information is further used to indicate the data communication volume between the accelerators contained in each server; the network topology includes the number of ports contained in each server in the server cluster; the computing unit 1602 is further used to: based on the existence of any server having data communication with the accelerator of the first server, determine the number of first ports used for data communication with the arbitrary server according to the data communication volume between the accelerators and the number of ports contained in each server, wherein the number of first ports is multiple; the computing unit 1602 is specifically used to: create a correspondence between multiple first ports and second ports.

[0279] In an optional implementation, the network topology further includes the bandwidth supported by the ports contained in each server in the server cluster; the computing unit 1602 is specifically used to: based on the existence of data communication between any server and the accelerator of the first server, determine the number of first ports used for data communication with the arbitrary server according to the data communication volume between the accelerators, the number of ports contained in each server, and the bandwidth supported by the ports contained in each server.

[0280] In an optional implementation, the computing unit 1602 is further configured to: determine a first computing time for each server in the server cluster based on the model information and the computing power information of the server cluster, wherein the model calculation and cross-server data communication can be performed synchronously during the first computing time; specifically, the computing unit 1602 is configured to: determine the number of first ports for data communication with the first server based on the existence of data communication between any server and the accelerator of the first server, according to the data communication volume between the accelerators, the number of ports contained in each server, and the first computing time of the first server.

[0281] In one alternative implementation, the server cluster further includes a first optical switching module, which is used to switch the state between the first port and the second port to a communicable state.

[0282] In an optional implementation, the server cluster is characterized by further including multiple optical switching modules; the computing unit 1602 is further configured to: determine multiple port connection schemes based on the existence of any server communicating with the accelerator of the first server through the first port of the first server, wherein different port connection schemes indicate that the first port of the first server is connected to different servers in the server cluster; determine a second computation time under each port connection scheme; the second computation time is used to indicate the time required to calculate the switching path of the optical switching module under the corresponding port connection scheme; and, based on the port connection scheme corresponding to the shortest computation time, determine that the first server will send data to the second port of the second server through the first port, or determine that the first server will receive data from the second port of the second server through the first port.

[0283] Please see Figure 18 , Figure 18 This is a schematic diagram of another port allocation device provided in an embodiment of this application. The port allocation device 1700 can be the aforementioned OXC module. For example... Figure 18 As shown, the port allocation device 1700 includes a receiving unit 1701 and a processing unit 1702.

[0284] The receiving unit 1701 is configured to: receive first information, the first information being used to indicate the correspondence between the first port and the second port;

[0285] The arithmetic unit 1702 is used to: create a communication connection between the first port and the second port based on the first information.

[0286] Port allocation device 1700 is used to implement the aforementioned Figure 11 The actions performed by the OXC module in the illustrated embodiment are to implement the corresponding port allocation method.

[0287] Figure 19 This is a schematic diagram of the structure of a port allocation device provided in an embodiment of this application. The port allocation device 1800 may include one or more CPUs 1801 and memory 1802, and the memory 1802 stores one or more applications or data.

[0288] The memory 1802 can be volatile or persistent storage. The program stored in the memory 1802 can include one or more modules, each module including a series of instruction operations on the port allocation device 1800. Furthermore, the central processing unit 1801 can be configured to communicate with the memory 1802 and execute the series of instruction operations in the memory 1802 on the port allocation device 1800.

[0289] The port allocation device 1800 may also include one or more communication interfaces 1803, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0290] The port allocation device 1800 can perform the aforementioned... Figures 5 to 15 The specific operations performed by the optical interconnect topology optimization module in the illustrated embodiment will not be described here.

[0291] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 5 to 15 The steps performed by the optical interconnect topology optimization module in the illustrated embodiment.

[0292] This application embodiment also provides a computer-readable storage medium storing a program for performing signal processing, which, when run on a computer, causes the computer to perform the aforementioned actions. Figures 5 to 15 The steps performed by the optical interconnect topology optimization module in the illustrated embodiment.

[0293] The port allocation device or optical interconnect topology optimization module provided in this application embodiment can specifically be a chip. The chip includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip in the training device to perform the aforementioned operations. Figures 5 to 15 The steps performed by the optical interconnect topology optimization module in the illustrated embodiment are as follows. Optionally, the storage unit can be an in-chip storage unit, such as a register or cache. Alternatively, the storage unit can be an external storage unit located within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0294] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0295] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0296] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0297] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0298] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A server cluster, characterized in that, The server cluster operates on a model, which includes multiple servers and a management module. The multiple servers include a first server and a second server. The first server includes multiple accelerators and a first port, and the second server includes multiple accelerators and a second port. The management module is used to obtain the model information of the model and the network topology of the multiple servers. The model information is used to indicate the data communication relationship between the accelerators contained in each server. The management module is also used to determine, based on the model information and the network topology, whether any server in the server cluster has data communication with the accelerator of the first server through the first port of the first server; The management module is further configured to create a correspondence between the first port and the second port when it is determined that the first server will send data to the second port of the second server through the first port, or when it is determined that the first server will receive data from the second port of the second server through the first port.

2. The server cluster according to claim 1, characterized in that, The management module is specifically used for: Based on the model information, determine whether any server in the server cluster has data communication with the accelerator of the first server; Based on the premise that any server has data communication with the accelerator of the first server, it is determined according to the network topology whether any server communicates with the accelerator of the first server through the first port.

3. The server cluster according to claim 1 or 2, characterized in that, The management module is also used for: Based on the fact that the second server communicates with the accelerator of the first server through the first port of the first server, the set communication algorithm used by the accelerator that needs to communicate with the first server is determined. Based on the network topology and the aggregated communication algorithm, it is determined that the first server will send data to the second port of the second server through the first port, or that the first server will receive data from the second port of the second server through the first port.

4. The server cluster according to any one of claims 1 to 3, characterized in that, The model information is also used to indicate the amount of data communication between the accelerators contained in each server; the network topology includes the number of ports contained in each server in the server cluster. The management module is further configured to, based on the existence of data communication between any server and the accelerator of the first server, determine the number of first ports used for data communication with any server, according to the data communication volume between the accelerators and the number of ports contained in each server, wherein the number of first ports is multiple. The management module is specifically used to create multiple correspondences between the first port and the second port.

5. The server cluster according to claim 4, characterized in that, The network topology also includes the bandwidth supported by the ports of each server in the server cluster. The management module is specifically used to determine the number of first ports for data communication with any one of the accelerators, based on the existence of data communication between any one of the accelerators, the number of ports contained in each server, and the bandwidth supported by the ports contained in each server, according to the data communication volume between the accelerators, the number of ports contained in each server, and the bandwidth supported by the ports contained in each server.

6. The server cluster according to claim 4 or 5, characterized in that: The management module is also used to determine the first computing time of each server in the server cluster based on the model information and the computing power information of the server cluster, and the model can be computed and cross-server data communication can be performed synchronously under the first computing time. The management module is specifically used to determine the number of first ports used for data communication with any one of the accelerators, based on the existence of data communication between any one of the accelerators, the number of ports contained in each of the accelerators, and the first computing time of the first server.

7. The server cluster according to any one of claims 1 to 6, characterized in that: It also includes a first optical switching module, which is used to switch the state between the first port and the second port to a communicable state.

8. The server cluster according to any one of claims 1 to 7, characterized in that, It also includes multiple optical switching modules; the management module is further used for: Based on the existence that any server communicates with the accelerator of the first server through the first port of the first server, multiple port connection schemes are determined, and different port connection schemes indicate that the first port of the first server is connected to different servers in the server cluster. Determine the second calculation time for each port connection scheme; the second calculation time is used to indicate the time required to calculate the switching path of the optical switching module under the corresponding port connection scheme; Based on the port connection scheme corresponding to the shortest computation time, it is determined that the first server will send data to the second port of the second server through the first port, or it is determined that the first server will receive data from the second port of the second server through the first port.

9. A port allocation method, characterized in that, include: The model information and network topology of multiple servers are obtained. The model runs on a server cluster, which includes multiple servers, including a first server and a second server. The first server includes multiple accelerators and a first port, and the second server includes multiple accelerators and a second port. The model information is used to indicate the data communication relationship between the accelerators contained in each server. Based on the model information and the network topology, determine whether any server in the server cluster has data communication with the accelerator of the first server through the first port of the first server; When it is determined that the first server will send data to the second port of the second server through the first port, or when it is determined that the first server will receive data from the second port of the second server through the first port, a correspondence is created between the first port and the second port.

10. The method according to claim 9, characterized in that, The step of determining whether any server in the server cluster has data communication with the accelerator of the first server through the first port of the first server, based on the model information and the network topology, includes: Based on the model information, determine whether any server in the server cluster has data communication with the accelerator of the first server; Based on the premise that any server has data communication with the accelerator of the first server, it is determined according to the network topology whether any server communicates with the accelerator of the first server through the first port.

11. The method according to claim 9 or 10, characterized in that, After determining whether any server in the server cluster has data communication with the accelerator of the first server through the first port of the first server based on the model information and the network topology, the method further includes: Based on the fact that the second server communicates with the accelerator of the first server through the first port of the first server, the set communication algorithm used by the accelerator that needs to communicate with the first server is determined. Based on the network topology and the aggregated communication algorithm, it is determined that the first server will send data to the second port of the second server through the first port, or that the first server will receive data from the second port of the second server through the first port.

12. The method according to any one of claims 9 to 11, characterized in that, The model information is also used to indicate the amount of data communication between the accelerators contained in each server; the network topology includes the number of ports contained in each server in the server cluster. Before creating the correspondence between the first port and the second port, the method further includes: Based on the existence of data communication between any server and the accelerator of the first server, the number of first ports used for data communication with any server is determined according to the data communication volume between the accelerators and the number of ports contained in each server. The number of first ports is multiple. Creating the correspondence between the first port and the second port includes: Create multiple mappings between the first port and the second port.

13. The method according to claim 12, characterized in that, The network topology also includes the bandwidth supported by the ports of each server in the server cluster. The step of determining the number of first ports used for data communication with any one of the servers, based on the data communication volume between the accelerators and the number of ports contained in each server, includes: The number of first ports for data communication with any one of the servers is determined based on the data communication volume between the accelerators, the number of ports contained in each server, and the bandwidth supported by the ports contained in each server.

14. The method according to claim 12 or 13, characterized in that, Before determining the number of first ports used for data communication with any one of the servers based on the data communication volume between the accelerators and the number of ports contained in each server, the method further includes: Based on the model information and the computing power information of the server cluster, a first computing time is determined for each server in the server cluster. During the first computing time, the model can be computed and cross-server data communication can be performed synchronously. The step of determining the number of first ports used for data communication with any one of the servers, based on the data communication volume between the accelerators and the number of ports contained in each server, includes: The number of first ports used for data communication with any one of the accelerators is determined based on the data communication volume between the accelerators, the number of ports contained in each server, and the first computing time of the first server.

15. The method according to any one of claims 9 to 14, characterized in that, The server cluster further includes a first optical switching module, which is used to switch the state between the first port and the second port to a communicable state.

16. The method according to any one of claims 9 to 15, characterized in that, The server cluster also includes multiple optical switching modules; Before creating the correspondence between the first port and the second port, the method further includes: Based on the existence that any server communicates with the accelerator of the first server through the first port of the first server, multiple port connection schemes are determined, and different port connection schemes indicate that the first port of the first server is connected to different servers in the server cluster. Determine the second calculation time for each port connection scheme; the second calculation time is used to indicate the time required to calculate the switching path of the optical switching module under the corresponding port connection scheme; Based on the port connection scheme corresponding to the shortest computation time, it is determined that the first server will send data to the second port of the second server through the first port, or it is determined that the first server will receive data from the second port of the second server through the first port.

17. A port allocation method, characterized in that, An optical switching module is used in a server cluster. The server cluster has a running model and includes multiple servers and a management module. The multiple servers include a first server and a second server. The first server includes multiple accelerators and a first port, and the second server includes multiple accelerators and a second port. The method includes: Receive first information, which is used to indicate the correspondence between the first port and the second port; Based on the first information, a communication connection is created between the first port and the second port.

18. A port allocation device, characterized in that, The device includes one or more units for implementing the port allocation method according to any one of claims 9 to 17.

19. A port allocation device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory; The memory is used to store programs; The processor is configured to execute a program in the memory, causing the port allocation device to perform the method as described in any one of claims 9 to 17.

20. A computer program product, characterized in that, When it is run on a computer, it causes the computer to perform the method as described in any one of claims 9 to 17.

Images