Path planning method and system for data center network cross-host set communication

Abstract

The application discloses a path planning method and system for cross-host set communication of a data center network, and the method comprises the following steps: receiving peer-to-peer relationships of each working node in a set communication initialization stage, obtaining a communication pair set across hosts and a candidate path mapping of each pair, and selecting an optimal path for each pair based on link load; subsequently, calling a host-side port synthesis mechanism for the selected optimal path, generating a plurality of effective source ports for the pair to be used in a cycle, and enabling each node task executor to perform cross-host set communication according to a received port allocation result, so that the hash selection of equivalent multi-path is controllably guided to realize load balancing without changing a switch or set communication library logic. The application completes the mapping of a data flow source port through one-time initialization, enables the data flow to be forwarded along a path with a small overlap degree, effectively reduces the path overlap degree, and shortens the flow completion time and execution time of the set communication.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data center network and distributed system optimization, and particularly relates to a path planning method and system for cross-host set communication of a data center network. BACKGROUND

[0002] In a data center network (DCN), collective communication (CC) usually involves transferring large flows between different host nodes, which often span multiple logical compute units (such as graphics processing units, tensor processing units, neural network processing units, etc.) to implement distributed model (such as large language models, visual models, etc.) training. This kind of cross-host CC is prevalent because training usually adopts multiple parallel strategies (PS) to coordinate the load of each host's task in the full network scale. These flows will traverse the logical host topology (such as organized in a ring structure) to span the next hop selection relying on the equal-cost multi-path (ECMP) mechanism to select the switch on the path. The ECMP forwarding is usually a black box to users because the ECMP hash function f that determines the corresponding egress port of the packet header h relies on XOR, bit shift, and mask operations. However, the randomness of ECMP based on hash often causes the cross-spine flows to concentrate on part of the spine-leaf links at the same time, resulting in path overlap and bandwidth competition. These hotspots will cause flow completion delay at the compute unit level, reducing the efficiency of CC. A lightweight method to solve this problem is to plan the cross-host path in the initialization phase after the logical topology is established and before the CC primitive is started, which can guide the cross-spine flow to reduce contention and thus improve the overall communication efficiency.

[0003] This path planning can utilize the technology of the recent host-based path selector (HPS), which allows the host to modify the header field (such as the source port) to linearly predict the exact trajectory of the packet on the switch according to the ECMP hash. This hash linearity can be expressed as: wherein is the modification of the data packet header, represents the XOR operation. This hash linearity allows the host to predictably affect the output port by applying a bitwise XOR offset to the packet header. Specifically, if the original hash and the desired port offset are known, then can be derived. The packet header is modified to After that, the data packet can be guaranteed to be forwarded along the desired path. This modification is small and feasible, while other methods require modification of the routing and transport layer, adjustment of the network topology, or rely on complex rules for replication and reconfiguration.

[0004] Based on this principle, existing methods implement path planning for multi-hop, multi-layer DCNs based on host-side path selection techniques. HPS requires prior knowledge of the DCN network topology information and black-box testing of switches to obtain a single hash table (SHT), which is a modification of each packet header Mapping to the port offset it generates HPS further generalizes the measured SHT to XOR-combinable And Pairs, HPS implements flexible bit-level modifications. Using these modifications, HPS defines deterministic criteria to verify whether the source port will be forwarded along a given path and performs actual probe verification when necessary, thereby achieving precise end-to-end host-side path control without modifying switches or routing protocols. HPS allows up to Internal attempts within a single external call to reduce failure rates. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art and provide a path planning method and system for cross-host set communication in a data center network. The present application can alleviate the path overlap in the network in existing cross-host set communication techniques, solve the problem of uneven link load, and improve the efficiency of set communication.

[0006] The purpose of the present application is achieved by the following technical solutions: the first aspect of the embodiment of the present application provides a path planning method for cross-host set communication in a data center network, comprising the following steps:

[0007] (1) Based on the topology structure of the data center network, the path planner collects the peer-to-peer relationship of inter-host set communication of each participating node task executor to obtain a cross-host communication pair set;

[0008] (2) Initialize the port allocation mapping for recording the allocation of communication port resources;

[0009] (3) For each parallel strategy in the cross-host communication pair set, perform path selection and port allocation processes to obtain the final port allocation mapping;

[0010] (4) After the path selection and port allocation for all parallel strategies and their corresponding each communication pair are completed, the allocation results in the port allocation mapping related to each source logical computing unit are distributed to the node task executor corresponding to the source logical computing unit;

[0011] (5) Each node task executor performs the collective communication across hosts according to the received allocation results when running the collective communication operation.

[0012] Further, the set of communication pairs across hosts is specifically obtained by the following method:

[0013] The peer-to-peer relationship is determined and reported by each participating node task executor in the initialization phase of the execution of the collective communication operation; the peer-to-peer relationship is represented by a mapping T, wherein the key of the mapping T is a parallel strategy identifier ps, and the value of the mapping T is an ordered set of all communication pairs across hosts to be established under the parallel strategy corresponding to the parallel strategy identifier ps ; the communication pair across hosts is an ordered pair , is a source logical computing unit node identifier, is a destination logical computing unit node identifier;

[0014] The peer-to-peer relationship from all participating node task executors is aggregated by the centralized path planner to form a global set of communication pairs across hosts .

[0015] Further, the parallel strategy includes a data parallel strategy and a model parallel strategy, and the model parallel strategy includes an expert parallel strategy, a pipeline parallel strategy and a tensor parallel strategy.

[0016] Further, the step (2) specifically includes:

[0017] The key of the port allocation mapping R is an identifier of all logical computing units currently participating in the collective communication, and the set of the identifier is denoted as V. For each identifier v in the set V, the corresponding value R(v) in the port allocation mapping R is initialized as an empty mapping.

[0018] Further, the path selection and port allocation process specifically includes the following sub-steps:

[0019] (3.1) Initialize the link load mapping L as an empty mapping, wherein L(e) represents the current load value of the link e, i.e., the current cumulative path count of the link e, and the initial value is 0;

[0020] (3.2) For each ordered set of communication pairs across hosts corresponding to the parallel strategy ps​​​ each communication pair , the following sub-steps are executed:

[0021] (3.2.1) judging whether the candidate path mapping of the corresponding communication pair is empty, if empty, skipping the current communication pair; if not empty, continuing to execute the next step;

[0022] (3.2.2) selecting an optimal path from the candidate path mapping based on the link load mapping L;

[0023] (3.2.3) updating the cumulative path count of all links of the optimal path in the link load mapping L according to the optimal path ;

[0024] (3.2.4) generating a port set O containing M source ports for the communication pair based on the optimal path , for use in run-time cyclic multiplexing;

[0025] (3.2.5) recording the generated port set O into the port allocation mapping R.

[0026] Further, the step (3.2.2) specifically comprises:

[0027] for each candidate path in the candidate path mapping , the following is executed:

[0028] obtaining the current load value of each link e in the candidate path , and sorting the load values in descending order to obtain a load value sequence ;

[0029] calculating the binary exponential weighted load score of the candidate path according to the load value sequence ;

[0030] selecting the candidate path with the minimum binary exponential weighted load score from all candidate paths as the optimal path .

[0031] Further, the step (3.2.4) specifically comprises:

[0032] first initializing the port set O and the selected port record set to be empty; then, repeatedly executing the following port generation sub-step until the length of the port set O reaches M:

[0033] a) randomly select a candidate source port from the available port number range which has not been recorded , and update the selected port record set ;

[0034] b) based on the source IP address and the destination IP address of the communication pair , the candidate source port and the fixed destination port , construct a network packet header h to be verified;

[0035] c) according to the pre-measured single hash table of the switch, based on the network packet header h to be verified and the optimal path , verify and generate a valid source port sp in the data center network; if the source port sp is successfully generated and the source port sp does not exist in the port set O, the source port sp is added to the port set O.

[0036] Further, the step (5) specifically includes:

[0037] For each parallel strategy ps and each destination logical computing unit that needs to communicate under the parallel strategy ps, the node task executor obtains a source port from the port set indicated by the allocation result ;

[0038] The data stream is sent using the obtained source port, so that the data stream can be forwarded along the optimal path determined for the communication pair and the parallel strategy ps.

[0039] Further, the way of obtaining the source port from the port set indicated by the allocation result includes at least one of the following: cyclic use, random use, sequential use or use based on flow characteristics.

[0040] The second aspect of the embodiment of the present application provides a path planning system for data center network cross-host set communication, including one or more processors and a memory, the memory is coupled with the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to realize the path planning method for data center network cross-host set communication described above.

[0041] Compared with the prior art, the present application has the following beneficial effects:

[0042] (1) The application realizes dynamic cooperation of path selection and flow distribution by introducing a link load-aware based path planning and source port mapping mechanism in cross-host collective communication, can significantly reduce the path overlap degree of cross-host communication flow in a multi-path data center network, realize link level load balancing, and effectively improve the bandwidth utilization and overall transmission performance of collective communication.

[0043] (2) The application has good universality and scalability, can seamlessly adapt to existing RDMA / RoCE communication stacks and mainstream deep learning frameworks, and has significant engineering application value and promotion potential. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a flow chart of the path planning method for cross-host collective communication of a data center network of the application;

[0045] Figure 2 is an architectural flow diagram of the path planning method for cross-host collective communication of a data center network of the application;

[0046] Figure 3 is a path planning deployment example diagram for cross-host collective communication of a data center network of the application;

[0047] Figure 4 is a structural diagram of the path planning system for cross-host collective communication of a data center network of the application. DETAILED DESCRIPTION

[0048] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to designate the same elements, unless otherwise indicated. It is apparent that the drawings used in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] The terms used in the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that, although the terms first, second, third, etc. can be employed in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to differentiate one piece of information from another piece of information. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if' as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0051] The present application will be described in detail below with reference to the accompanying drawings. The features in the following examples and embodiments can be combined with each other without conflict.

[0052] Referring to Figure 1 and Figure 2 The path planning method for data center network cross-host set communication of the present application specifically includes the following steps:

[0053] (1) Based on the topology structure of the data center network (DCN), the path planner (PathPlanner) collects the peer-to-peer relationship of the inter-host collective communication (CC) of each participating node task executor (worker) to obtain the cross-host communication pair set T. In the present application, the path planning system for cross-host collective communication can be regarded as the path planner.

[0054] Further, the cross-host communication pair set T is specifically obtained by the following method steps:

[0055] (1.1) Determine and report the peer-to-peer relationship by each participating node task executor in the initialization phase of the collective communication operation (such as calling the ncclCommInitRank command or equivalent operation); represent the peer-to-peer relationship as a mapping T, where the key of the mapping T is the parallel strategy identifier ps, and the value of the mapping T is the ordered set of all cross-host communication pairs to be established under the parallel strategy corresponding to the parallel strategy identifier ps ; the cross-host communication pair is an ordered pair , where is the source logical computing unit node identifier, is the destination logical computing unit node identifier.

[0056] The parallel strategies include a data parallelism (DP) strategy and a model parallel strategy, and the model parallel strategy includes an expert parallelism (EP) strategy, a pipeline parallelism (PP) strategy and a tensor parallelism (TP) strategy.

[0057] (1.2) Each participating node task executor reports its corresponding ordered set and its corresponding parallel strategy identifier ps to the centralized path planner in a single message, and the centralized path planner aggregates the peer-to-peer relationships from all participating node task executors to form a global cross-host communication pair set T, denoted as .

[0058] (2) The port allocation mapping R used to record the communication port resource allocation situation is initialized.

[0059] Specifically, the key of the port allocation mapping R is the identifier of all logical computing units currently participating in set communication, and the set of the identifier is denoted as ; for each identifier v in the set , the corresponding value of the port allocation mapping R is initialized as an empty mapping. Specifically, it can be denoted as , that is, for all , let . Wherein, the port allocation mapping R when used to record the allocation situation of the communication port resource, can be specifically denoted as , which indicates the source port list allocated for the communication link from the source logical computing unit node to the destination logical computing unit node under the parallel strategy corresponding to the parallel strategy identifier ps.

[0060] (3) For each parallel strategy ps in the cross-host communication pair set T, the path selection and port allocation process is performed to obtain the final port allocation mapping R.

[0061] In this embodiment, the path selection and port allocation process specifically includes the following sub-steps:

[0062] (3.1) Initialize the link load mapping L as an empty mapping, wherein represents the current load value of the link e, that is, the current cumulative path count of the link e, which is used to measure the occupation of the current allocated path to the link, , and the initial value of

[0063] It should be noted that the link e can be obtained based on the physical topology of the data center network. Specifically, the candidate paths between all source nodes and destination nodes in the data center network can be represented as a mapping P, which can be specifically represented as , represents a candidate end-to-end path set from the source node to the destination node , and each path is represented as , , , represents the link , the link , and the link on the path , represents the length of the path . P can be statically provided by an administrator or obtained by enumerating all candidate end-to-end paths based on the physical topology of the data center network. The physical topology of the data center network can be represented as , where V is a set of nodes (hosts, switches, logical computing units, etc.), and E is a set of links.

[0064] (3.2) For each communication pair in the ordered set of all cross-host communication pairs corresponding to the parallel strategy ps, the following sub-steps are performed:

[0065] (3.2.1) Determine whether the candidate path mapping corresponding to the communication pair is empty. If it is empty, there is no candidate path for the communication pair, and step (3.2.2) is not continued, the current communication pair is skipped, and the next communication pair in the ordered set is processed. If it is not empty, the next step (3.2.2) is continued.

[0066] (3.2.2) Select an optimal path from the candidate path mapping based on the link load mapping L.

[0067] Specifically, first, the minimum load is initialized, and the optimal path is initialized. Then, for each candidate path in the candidate path mapping , the following sub-steps are performed:

[0068] (3.2.2.1) Obtain the current load value of each link e in the candidate path The load value sequence is obtained by sorting the load values ​​in descending order. It can be formally expressed as , Indicates the candidate path The current load value of its uplink The sequence obtained by sorting in descending order.

[0069] (3.2.2.2) Based on the load value sequence Calculate candidate paths Binary exponential weighted load score The calculation formula is as follows:

[0070]

[0071] In the formula, Indicate candidate path The binary exponential weighted load score, The sequence of load values ​​obtained in step (3.2.2.1) represents the total load value. The i-th load value in the data. Indicates the calculation of candidate paths The length.

[0072] (3.2.2.3) If ,but ,and That is, select the candidate path with the smallest binary exponential weighted load score from all candidate paths as the optimal path. .

[0073] (3.2.3) Based on the optimal path Update the optimal path in the link load mapping L. The cumulative path count of all links can be formally represented as This update reflects the link occupancy already reserved for this communication pair, which is beneficial for subsequent load balancing selection for other pairs.

[0074] (3.2.4) Based on the optimal path For communication pairs A port set O containing M source ports is generated for runtime cyclic reuse, which improves the efficiency of set communication. Here, M represents the number of ports for each communication pair. The number of source ports allocated ensures that each communication pair M valid source ports need to be obtained for cyclical use during runtime.

[0075] (3.2.4.1) Initialize the port set O into an empty list, i.e. It is initialized as a set of selected port records used to store the port numbers already in use. Empty.

[0076] (3.2.4.2) Repeat the following port generation sub-steps until the length of the port set O reaches M:

[0077] a) Randomly select a candidate source port from the available port number range which has not been recorded yet , and update the selected port record set , i.e. insert it into the selected port record set . Since the source port number is an unsigned integer of 16-bit length (0~65535), and the source ports of 0~1023 are usually reserved for system critical services themselves, they cannot be selected as the required ports of the application, so the available port number range is {1024,...,65535}.

[0078] b) Based on the source IP address and the destination IP address of the communication pair , the candidate source port , and the fixed destination port , construct the network packet header to be verified h, which can be formally represented as: , where is the host IP identifier.

[0079] c) According to the pre-measured single hash table (SHT) of the switch, based on the network packet header to be verified h and the optimal path , verify and generate a valid source port sp in the data center network, which can enable h to be forwarded along the optimal path ; if the source port sp is successfully generated and the source port sp does not exist in the port set O, add the source port sp to the port set O; if the source port sp is not successfully generated or the source port sp exists in the port set O, repeat times until the source port sp is successfully generated, where can be specified by the user.

[0080] It should be understood that the pre-measured single hash table of the switch can facilitate the subsequent call of the host-based path selector (HPS). In addition, in order to reduce the measurement burden, usually only 1~2 representative switches are measured for SHT at each network layer, unless the in-house switch configuration is inconsistent.

[0081] (3.2.5) Record the generated port set O in the port allocation mapping R, which can be specifically represented as , so that the final port allocation mapping R is obtained.

[0082] (4) When the path selection and port allocation for all parallel strategies ps and each corresponding communication pair are completed, the allocation result in the port allocation mapping R related to each source logical computing unit is distributed to the node task executor corresponding to the source logical computing unit.

[0083] (5) Each node task executor performs the cross-host collective communication according to the received allocation result when running the collective communication operation.

[0084] Specifically, for each parallel strategy ps and each destination logical computing unit that needs to communicate under the parallel strategy ps, the node task executor acquires the source port sp from the port set O indicated by the allocation result ; sends the data stream using the acquired source port sp so that the data stream can be forwarded along the optimal path determined for the communication pair and the parallel strategy ps , thereby improving the overlap of network paths and improving the performance of the collective communication.

[0085] Further, the way of acquiring the source port sp from the port set O indicated by the allocation result includes at least one of the following: cyclic use, random use, sequential use, or use based on flow characteristics.

[0086] The path planning method for cross-host collective communication of a data center network according to the present application will be described in detail below according to an embodiment, and the purposes and effects of the present application will become more apparent.

[0087] The DCN topology in the present embodiment is a typical spine-leaf structure, as shown in Figure 3 . In the DCN, there are 4 leaf layer switches numbered 32-35, and 4 spine layer switches numbered 36-39, each leaf layer switch is connected to all 4 spine layer switches, and each leaf layer switch is connected to 8 logical computing unit (rank) nodes, a total of 32, numbered 0-31. Every 4 rank nodes form a host, interconnected by an intra-machine network such as NVIDIA NVSwitch or NVLink, for example, host 0 contains rank nodes 0-3, host 1 contains rank nodes 4-7, host 2 contains rank nodes 8-11, and so on, a total of 8 hosts.

[0088] ​​​In the DCN, the collective communication includes DP strategy, adopts ring topology structure, and the communication pairs are formed in adjacent relationship, that is , there are 32 communication pairs in total, of which 8 are cross-host communication pairs, which are (3, 4), (7, 8), (11, 12), (15, 16), (19, 20), (23, 24), (27, 28), and (31, 0). These cross-host communication pairs are the main planning objects of the method described in the application. It is assumed that the number of source ports M allocated for each communication pair is 16, that is, 16 source port numbers are generated for each communication pair; the UDP destination port number of RDMA / RoCE is 47000, which means that when using the RoCE (RDMA over Converged Ethernet, RDMA based on converged Ethernet) protocol, the fixed destination port number of the UDP (User Datagram Protocol) encapsulating the RDMA (Remote Direct Memory Access) traffic is 47000.

[0089] In this embodiment, the path planning method specifically includes the following steps:

[0090] (1) Collect the set T of cross-host communication pairs. Specifically, the path planner collects all communication pair equalities under the ring topology structure. As shown in Figure 3 , in the ring topology structure, there are 32 communication pairs under the current DP strategy, of which 8 are cross-host communication pairs, that is .

[0091] (2) Initialize the port mapping result R to be empty, that is , that is, for all , let . Specifically, , , , …, . Then, for the DP strategy of the present embodiment, .

[0092] (3) For each parallel strategy ps (in this embodiment, ps=DP) in the set T of cross-host communication pairs, perform the following path selection and port allocation process:

[0093] (3.1) Initialize the link load mapping L to be an empty mapping, that is: for each link in the data center network, initialize .

[0094] (3.2) For each communication pair in That is, (3, 4), (7, 8), (11, 12), (15, 16), (19, 20), (23, 24), (27, 28), (31, 0), the following sub-steps are executed to generate the relevant source port:

[0095] (3.2.1) Determine the communication pair corresponding candidate path mapping is not empty, for example, for the current communication pair (7, 8), the communication pair (7, 8) corresponding candidate path mapping is {(7, 32, 36, 33, 8), (7, 32, 37, 33, 8), (7, 32, 38, 33, 8), (7, 32, 39, 33, 8)}, which is not empty, then the next step (3.2.2) is executed.

[0096] (3.2.2) Initialize the minimum load and initialize the optimal path ; then, for each candidate path in the candidate path mapping , for example, for the 4 candidate paths in the candidate path mapping , the following sub-steps are executed, that is, the following 4 loops are executed:

[0097] First loop: , the following sub-steps are executed:

[0098] (3.2.2.1) Obtain the current load value of each link e in the candidate path , that is , then sort the load value sequence in descending order .

[0099] (3.2.2.2) Calculate the binary exponential weighted load score of the candidate path .

[0100] (3.2.2.3) Compare and , since , the current candidate path (7, 32, 36, 33, 8) is taken as the optimal path , that is ; at the same time, the binary exponential weighted load score of the current candidate path (7, 32, 36, 33, 8) is used to update the minimum load , that is .

[0101] Second loop:​​ perform the following sub-steps:

[0102] (3.2.2.1) Obtain the current load value of each link e in the candidate path i.e. the sequence of load values in descending order is .

[0103] (3.2.2.2) Calculate the binary exponential weighted load score of the candidate path . .

[0104] (3.2.2.3) Compare the calculated with the current , since , the previously selected optimal path is kept .

[0105] 3rd loop: perform the following sub-steps:

[0106] (3.2.2.1) Obtain the current load value of each link e in the candidate path i.e. the sequence of load values in descending order is .

[0107] (3.2.2.2) Calculate the binary exponential weighted load score of the candidate path . .

[0108] (3.2.2.3) Compare the calculated with the current , since , the previously selected optimal path is kept .

[0109] 4th loop: perform the following sub-steps:

[0110] (3.2.2.1) Obtain the current load value of each link e in the candidate path i.e. the sequence of load values in descending order is .

[0111] (3.2.2.2) Calculate the binary exponential weighted load score of the candidate path . .

[0112] ​​​​​​(3.2.2.3) the calculated and the current are compared, and if , the previously selected optimal path is kept .

[0113] After 4 cycles, the binary exponentially weighted load scores of the 4 candidate paths are all 0, and the first decided optimal path is , .

[0114] (3.2.3) after the optimal path is determined, an occupancy count update operation is performed on the cumulative path count of each link on the optimal path : i.e. , , , are all incremented by 1, and here the counter is 1, and the rest of the elements in the link load map L remain unchanged, and here the counter is 0.

[0115] (3.2.4) based on the optimal path , a port set O containing M source ports is generated for the communication pair :

[0116] (3.2.4.1) initialize the port set , and the selected port record set .

[0117] (3.2.4.2) repeat the following port generation sub-steps until the length of the port set O reaches M, i.e. :

[0118] a) randomly select a candidate source port from {1024,...,65535} that has not been recorded, such as 32561, and insert it into .

[0119] b) based on the source IP address and the destination IP address of the communication pair , the candidate source port , and the fixed destination port , construct the to-be-verified network packet header h, for example, construct the to-be-verified network packet header for the communication pair (7,8), which respectively represent the IP of node 7, the IP of node 8, , and the destination port number of RDMA / RoCE.

[0120] c) in order to enable h to follow the optimal path In the data center network forwarding, according to the pre-measured SHT of the switch, based on the to-be-verified network packet header h and the optimal path , an effective source port sp=68361 is verified and generated in the data center network. Since the sp generation is successful and , the sp is added to O, i.e. .

[0121] By repeating the above steps, the final port set O with a length of 16 can be obtained .

[0122] (3.2.5) The generated port set O is recorded in the port allocation mapping R, i.e. .

[0123] Through the calculation for all communication pairs (i.e. 8 cross-host communication pairs corresponding to the DP), the final port allocation mapping R is obtained:

[0124]

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[0131]

[0132] (4) When each communication pair in completes path selection and port allocation, the allocation result of each is distributed to the node task executor corresponding to , for example, the allocation result R[7], i.e. , is sent to the node task executor corresponding to node 7.

[0133] (5) For example, when , , , the worker 7 can select one of the following port numbers in sequence or randomly from the allocation result when sending the corresponding data stream: ​By binding the selected source port number, the worker 7 can ensure that the data stream follows the optimal path determined in step (3.2.2.3) Forwarding. By implementing port-level path binding at the source end, cross-host traffic between different communication pairs is dispersed on multiple spine links, significantly reducing path overlap and improving the overall throughput and stability of collective communication.

[0134] Corresponding to the above-mentioned embodiments of the path planning method for cross-host collective communication of a data center network, the present application also provides embodiments of a path planning system for cross-host collective communication of a data center network.

[0135] Referring to Figure 4 , the embodiments of the present application provide a path planning system for cross-host collective communication of a data center network, which includes one or more processors and a memory, the memory being coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to realize the path planning method for cross-host collective communication of a data center network in the above-mentioned embodiments.

[0136] The embodiments of the path planning system for cross-host collective communication of a data center network can be applied to any data processing device, which can be a device or apparatus such as a computer. The system embodiments can be realized by software, or by hardware or a combination of software and hardware. Taking software realization as an example, as a logical system, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for execution by the processor of the data processing device where the system is located. From the hardware level, as shown in Figure 4 , it is a hardware structure diagram of the data processing device where the path planning system for cross-host collective communication of a data center network is located, in addition to the processor, graphics processor, tensor processor, neural network processor, memory, network interface, and non-volatile memory shown in Figure 4 , the data processing device where the system is located in the embodiments usually includes other hardware according to the actual functions of the data processing device, and details are not described here.

[0137] The implementation process of the functions and roles of each unit in the above-mentioned system is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and is not described here.

[0138] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts are described in the method embodiments. The system embodiments described above are only illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the technical solutions of the present application according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0139] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A path planning method for cross-host aggregate communication in a data center network, characterized in that, Includes the following steps: (1) Based on the topology of the data center network, the path planner collects the peer-to-peer relationships of the host set communication of each participating node task executor in order to obtain the cross-host communication pair set; (2) Initialize the port allocation mapping used to record the allocation of communication port resources; (3) For each parallel strategy in the cross-host communication pair set, execute the path selection and port allocation process to obtain the final port allocation mapping; The path selection and port allocation process specifically includes the following sub-steps: (3.1) Initialize the link load mapping L as an empty mapping, where This represents the current load value of link e, which is the current cumulative path count of link e. Its initial value is 0. (3.2) For the ordered set of all cross-host communication pairs corresponding to the parallel strategy ps Each communication pair in ,in This serves as the identifier for the source logic computation unit node. To identify the target logical computation unit node, perform the following sub-steps: (3.2.1) Determine the communication pair Corresponding candidate path mapping Is it empty? If it is empty, skip the current communication pair. If it is not empty, proceed to the next step; (3.2.2) Based on the link load mapping L, map from candidate paths Choose an optimal path ; Step (3.2.2) specifically includes: For candidate path mapping Each candidate path in ,implement: Get candidate paths Current load value of each link e in the middle The load value sequence is obtained by sorting the load values ​​in descending order. ; Based on the load value sequence Calculate candidate paths Binary exponential weighted load score; Select the candidate path with the smallest binary exponential weighted load score from all candidate paths as the optimal path. ; (3.2.3) Based on the optimal path Update the optimal path in the link load mapping L. The cumulative path count of all links on the network; (3.2.4) Based on the optimal path For communication pairs Generate a port set O containing M source ports for cyclic reuse at runtime; step (3.2.4) specifically includes: First, initialize the port set O and the selected port record set. If empty; then, repeat the following port generation sub-step until the length of the port set O reaches M: a) Randomly select an unrecorded candidate source port from the range of available port numbers. And update the set of selected port records. ; b) Based on communication pairs Source IP address and destination IP address, candidate source port With and fixed destination port Construct the network packet header h to be verified; c) Based on the pre-measured single hash table of the switch, and using the network packet header h to be verified and the optimal path... In the data center network, a valid source port sp is verified and generated; if the source port sp is successfully generated and does not exist in the port set O, the source port sp is added to the port set O. (3.2.5) Record the generated port set O into the port allocation mapping R; (4) After path selection and port allocation are completed for all parallel strategies and their corresponding communication pairs, the allocation results related to each source logical computing unit in the port allocation mapping are distributed to the node task executor corresponding to the source logical computing unit. (5) When each node task executor runs the collection communication operation, it performs cross-host collection communication based on the received allocation results.

2. The path planning method for cross-host aggregate communication in a data center network according to claim 1, characterized in that, The cross-host communication pair set is obtained through the following method: The peer relationship is determined and reported by each participating node's task executor during the initialization phase of the set communication operation; the peer relationship is represented by a mapping T, where the key of the mapping T is the parallel policy identifier ps, and the value of the mapping T is an ordered set of all cross-host communication pairs to be established under the parallel policy corresponding to the parallel policy identifier ps. The cross-host communication pairs are ordered pairs. ; A centralized path planner aggregates peer relationships from all participating node task executors to form a global set of cross-host communication pairs. .

3. The path planning method for cross-host aggregate communication in a data center network according to claim 2, characterized in that, The parallel strategies include data parallel strategies and model parallel strategies. The model parallel strategies include expert parallel strategies, pipeline parallel strategies, and tensor parallel strategies.

4. The path planning method for cross-host aggregate communication in a data center network according to claim 1, characterized in that, Step (2) specifically includes: The key of the port allocation mapping R is the identifier of all logical computation units currently participating in the set communication, and the set of this identifier is denoted as . For sets Each identifier v in the mapping will assign the corresponding value in the port to the mapping R. Initialize to an empty mapping.

5. The path planning method for cross-host aggregate communication in a data center network according to claim 1, characterized in that, Step (5) specifically includes: For each parallel strategy ps, and for each destination logical computation unit that needs to communicate under that parallel strategy ps. The node task executor receives the allocation results. Obtain the source port from the specified port set; Send the data stream using the acquired source port, so that the data stream can be used for communication. Forward the route using the optimal path determined by the parallel strategy ps.

6. The path planning method for cross-host aggregate communication in a data center network according to claim 5, characterized in that, The allocation result The method of obtaining the source port from the indicated set of ports includes at least one of the following: cyclic use, random use, sequential use, or use based on flow characteristics.

7. A path planning system for cross-host aggregate communication in a data center network, comprising one or more processors and a memory, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the path planning method for cross-host aggregate communication in a data center network as described in any one of claims 1-6.

Images