Multi-tenancy for rdma network in network aggregation transmission system and method

Abstract

The application discloses a multi-tenant network aggregation transmission system and method suitable for an RDMA network, wherein a host side (1) is provided with a worker node (3) and a parameter server node (4), the worker node (3) is provided with a speed limiter (5) and a double transmission path control module (6), the parameter server node (4) is provided with a memory request module (10), the memory request module (10) comprises a data / request aggregation module (11) and a feedback module (12), a switch side (3) is provided with a memory allocation module (16), an aggregation module (17) and a broadcast module (18), and a memory allocation algorithm based on a first-come-first-served strategy is adopted on the switch side to control the host side in a fast path mode, and the host side determines the host side to execute network aggregation of gradient data packets. The application improves the transmission efficiency of the multi-tenant transmission system as a whole, and realizes complete compatibility of the INA solution and the RDMA.

Description

Technical Field

[0001] This invention belongs to the field of data center networks, and particularly relates to a multi-tenant on-network aggregation transmission system and method for large-scale data centers. Background Technology

[0002] In recent years, large-scale data centers have increasingly adopted two technologies to accelerate the training of deep neural networks. The first is In-network Aggregation (INA), which offloads gradient aggregation from the parameter server (PS) to a network switch or FPGA middleware box, thereby freeing up network resources, accelerating training tasks, and improving training scalability. The second is Remote Direct Memory Access (RDMA), which offloads network functions to the hardware network interface card (NIC), bypassing the software protocol stack to achieve low latency and high throughput. However, In-network Aggregation breaks down the boundaries between computation and networking, causing data packets to be consumed within the network, leading to misjudgments of RDMA reliability. Therefore, using RDMA networks in INA solutions becomes a problem. This problem can be summarized into the following two objectives:

[0003] Objective 1: RDMA Compatibility: RDMA supports three types of connection transmission: Reliable Connection (RC), Unreliable Connection (UC), and Unreliable Datagram (UD). RC supports all RDMA transmission primitives and guarantees at most one ordered and lossless delivery. However, DNN training performance can degrade drastically with packet loss. Therefore, RC must be enabled for INA solutions to be fully compatible with RDMA.

[0004] Objective 2: INA Multi-tenancy: INA reduces network traffic by using switch memory (registers) to store and aggregate gradients. However, currently available commercial programmable switches have limited memory resources; for example, Tofino switches have only about 10MB of available memory, while DNN training clusters are typically shared by a large number of concurrent tasks. INA solutions must also support multi-tenant learning when integrated with RDMA to improve scalability.

[0005] To achieve compatibility between RDMA and INA in multi-tenant environments, a series of solutions have been proposed. These can be broadly categorized into two types: best-effort based solutions and static allocation based solutions. ① Best-effort based solutions, such as ATP, A2TP, and ESA, retain the parameter server (PS) and network switches used for gradient aggregation. In this solution, workers stream packets to the top-of-rack switch, which performs dynamic packet-level aggregation in a best-effort manner. If an idle aggregator exists on the switch, it performs in-network aggregation; otherwise, it pushes the packets to the parameter server for aggregation, a process called In-server Aggregation (ISA). This dynamic aggregation solution supports multi-tenant learning but is incompatible with RDMA RC. This is because packets with small sequence numbers may be dropped during in-network aggregation, and when packets with large sequence numbers are aggregated using the PS, the end-side transport protocol interprets this as packet loss, triggering unnecessary retransmissions of the in-network aggregated packets. ② Static allocation based solutions, such as SwitchML and NetReduce, eliminate the PS and completely offload gradients to the switches. This solution provides a way to establish RDMA RC connections between workers or between a worker and a switch. However, because static memory partition switching is performed during the training process of each task, the number of concurrent training tasks is limited. Therefore, this static memory switching solution cannot support multi-tenant training scenarios. Summary of the Invention

[0006] This invention aims to design a multi-tenant on-network aggregation transmission system and method suitable for RDMA networks, which realizes on-network aggregation transmission supporting multiple tenants by combining dual transmission paths with all-reduce level memory requests.

[0007] This invention is achieved using the following technical solution:

[0008] A multi-tenant on-network aggregation transmission system suitable for RDMA networks includes a host side 1 and a switch side 2. At least one worker node 3 and at least one parameter server node 4 are deployed on the host side 1. The worker node 3 is connected to the switch side 3 through a first RDMA network card 13, and the parameter server node 4 is connected to the switch side 3 through a second RDMA network card 14.

[0009] The Worker node 3 is equipped with a rate limiter 5 and a dual transmission path control module 6; the rate limiter 5 is used to limit the traffic sent by a single task to the network; the dual transmission path control module 6 provides dual transmission path control logic on the host side, and further includes a slow path mode 7, a fast path mode 8, and a marker 9; the slow path mode 7 is used for aggregation within the server, the fast path mode 8 is used for aggregation on the network; the marker 9 is used to insert a request flag into the data packets aggregated within the server.

[0010] The parameter server node 4 is equipped with a memory request module 10 for all-reduce level memory requests. The memory request module 10 further includes a data / request aggregation module 11, a feedback module 12, and a fast / slow path mode 18. The data / request aggregation module 11 aggregates gradient data and memory request results from all Worker nodes 3 to the parameter server PS node 4. The feedback module 12 returns the parameter data and memory request results sent from the PS node to the Worker node 3. The fast / slow path mode 18 forms a reliable connection transmission queue pair and a virtual reliable connection transmission queue pair with the slow path mode 7 and fast path mode 8 in the Worker node.

[0011] The switch side 3 is equipped with a memory allocation module 16, an aggregation module 17, and a broadcast module 18. The memory allocation module 16 adopts a memory allocation algorithm based on a first-come, first-served strategy to provide dual transmission path control for the training task on the host side 1. The aggregation module 17 is used to perform on-network aggregation on the switch side. The broadcast module 18 is used to distribute lost gradient data packets to restore the connection state and return parameter packets to the parameter server node 4.

[0012] Furthermore, the slow path mode and fast path mode of the Worker node 3 respectively include data transmission queue pairs for point-to-point reliable connections and data transmission queue pairs for many-to-one virtual reliable connections; the parameter server node 4 includes data transmission queue pairs for point-to-point reliable connections and data transmission queue pairs for one-to-many virtual reliable connections; wherein, the data transmission queue pairs for point-to-point reliable connections in the Worker node 3 are connected to the data transmission queue pairs for point-to-point reliable connections in the parameter server node 4; the data transmission queue pairs for one-to-many virtual reliable connections in the parameter server node 4 are connected to the data transmission queue pairs for many-to-one virtual reliable connections in the Worker node 3.

[0013] Furthermore, the slow path mode 7 establishes a reliable point-to-point connection between the Worker node 3 and the parameter server node 4.

[0014] Furthermore, the fast path mode 8 establishes a many-to-one virtual reliable connection for the Worker node and the parameter server node 4.

[0015] Furthermore, when a task enters the communication phase, the host side 1 transmits data packets by default through the slow path mode.

[0016] Furthermore, the memory allocation algorithm based on the first-come, first-served strategy includes:

[0017] When the switch receives a data packet, it parses the destination QPN field in the data packet header to obtain the task identification number and the requested memory pool sub-index IDX.

[0018] Track the allocation status of N memory pools. If the data packet carries a memory request flag and comes from a Worker node, the switch side calls memory allocation until all memory pool allocation requests are completed.

[0019] Ultimately, the switch forwards this data packet to the reference server node.

[0020] This invention provides a multi-tenant on-network aggregation transmission method applicable to RDMA networks, comprising:

[0021] Construct a multi-tenant on-network aggregation transmission system suitable for RDMA networks according to any one of claims 1-6, and realize the connection between the Worker node and the reference server node on the host side;

[0022] On the switch side, a memory allocation algorithm based on a first-come, first-served strategy is used to control the host side in fast path mode. The host side uses the execution isINA field carried by the Worker node gradient data packet to determine that the switch side has received the gradient data packet transmitted from the Worker node. The aggregation module on the switch side performs on-network aggregation of the gradient data packet. After the aggregation is completed, the gradient data packet header and payload are modified on the switch side, and the gradient data packet is forwarded to the transmission queue pair of the virtual reliable connection on the reference server.

[0023] To implement intra-network ACK broadcasting, when gradient data packets arrive at the transmission queue of the virtual reliable connection on the reference server node, intra-network ACK broadcasting is enabled to the transmission queue of the virtual reliable connection on all worker nodes in order to restore the connection state of all worker nodes.

[0024] Compared with the prior art, the present invention achieves the following beneficial technical effects:

[0025] 1) It demonstrates the advantage of host-side driving, which enables training tasks to dynamically utilize in-network aggregation (INA) according to their aggregation needs;

[0026] 2) Avoiding retransmissions improves the overall transmission efficiency of the multi-tenant transmission system;

[0027] 3) The INA solution is fully compatible with RDMA without requiring a reboot. Attached Figure Description

[0028] Figure 1 This is a system architecture diagram of the multi-tenant on-network aggregation transmission system applicable to RDMA networks according to the present invention;

[0029] Figure 2 This is an overall flowchart of the multi-tenant on-network aggregation transmission method applicable to RDMA networks according to the present invention.

[0030] Figure 3 This is a host-side deployment example diagram of an embodiment of the present invention;

[0031] Figure 4 This is a sample diagram of the memory pool allocation algorithm code implementation;

[0032] Figure label:

[0033] 1. Host side, 2. Switch side, 3. Worker node, 4. Reference server node, 5. Rate limiter, 6. Dual Transmission Path Control (DTP) module, 7. Slow path mode, 8. Fast path mode, 9. Marker, 10. Memory Request Module (AMR), 11. Data / Request Aggregation Module, 12. Feedback module, 13. First RDMA network card, 14. Second RDMA network card, 15. Memory allocation module, 16. Aggregation module, 17. Broadcast module, 18. Fast / Slow path mode.

[0034] 2. The end-side of this invention uses a commercial Mellanox ConnectX-5 dual-port 100Gbps network card, and the switch uses a commercial Tofino programmable switch that supports P4 language. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings.

[0036] First, the dynamic host on the endpoint can predetermine packet behavior (ISA or INA), thereby proactively deciding the necessity of retransmissions to support RDMA compatibility. Furthermore, the endpoint driver enables training tasks to dynamically utilize the advantages of INA based on their aggregation needs. In other words, one task can proactively free up memory for other tasks to support multi-tenant training.

[0037] Example:

[0038] like Figure 1As shown, the multi-tenant on-network aggregation transmission system for RDMA networks of the present invention includes a host-side 1 and a switch-side 2. Specifically, the host-side 1 includes a Worker node 3 and a Parameter Server node 4 (PS node). The Worker node 3 includes a rate limiter 5 and a Dual Transmission Paths (DTP) module 6. The rate limiter 5 is used to limit the traffic volume of a single task sent to the network to not exceed the delay-bandwidth product. The Dual Transmission Paths control module 6 further includes a slow path mode 7, a fast path mode 8, and a marker 9; the Dual Transmission Paths management module is used to provide Dual Transmission Paths (DTP) management logic on the host side. Based on the Dual Transmission Paths (DTP) management logic, slow path mode 7 is used for intra-server aggregation of ISAs, and fast path mode 8 is used for on-network aggregation of INAs. The marker 9 is used to insert a request flag into the ISA data packet to allow task-aware memory on intra-server aggregation. The parameter server node 4 further includes a memory request module 10 for all-reduce-level memory requests (AMR) in the parameter server node 4. The memory request module 10 further includes a data / request aggregation module 11, a feedback module 12, and a fast / slow path mode 18. The data / request aggregation module 11 aggregates gradient data and memory request results arriving at the parameter server (PS) node 4 from all Worker nodes 3. The feedback module 12 returns parameter data or memory request results sent from the PS node to all Worker nodes. The fast / slow path mode 18 forms a reliable connection transmission queue pair and a virtual reliable connection transmission queue pair with the slow path mode 7 and fast path mode 8 in the Worker nodes.

[0039] The Worker node 3 is connected to the switch side 3 via the first RDMA network card 13. The parameter server node 4 is connected to the switch side 3 via the second RDMA network card 14.

[0040] The switch side 3 further includes a memory allocation module 16, an aggregation module 17, and a broadcast module 18. The memory allocation module 16 employs a first-come, first-served (FFS) memory allocation algorithm to provide dual transmission path control for the host side 1 for the training task. The aggregation module 17 performs on-network aggregation on the switch side. The broadcast module 18 distributes lost gradient data packets to restore connection status and returns parameter packets to the parameter server node 4.

[0041] like Figure 2 As shown, the overall process of the multi-tenant on-network aggregation transmission method applicable to RDMA networks of the present invention specifically includes the following steps:

[0042] Construct a multi-tenant on-network aggregation transmission system suitable for RDMA networks to establish connections between Worker nodes and reference server nodes on the host side;

[0043] On the switch side, a memory allocation algorithm based on a first-come, first-served strategy is used to control the host side in fast path mode. The host side uses the isINA field carried by the Worker node gradient data packet to determine that the switch side has received the gradient data packet transmitted from the Worker node. The aggregation module on the switch side performs on-network aggregation of the gradient data packet. After the aggregation is completed, the gradient data packet header and payload are modified on the switch side, and the gradient data packet is forwarded to the transmission queue pair of the virtual reliable connection on the reference server.

[0044] To implement intra-network ACK broadcasting, when gradient data packets arrive at the transmission queue of the virtual reliable connection on the reference server node, intra-network ACK broadcasting is enabled to the transmission queue of the virtual reliable connection on all worker nodes in order to restore the connection state of all worker nodes.

[0045] like Figure 3 As shown, this embodiment of the invention provides a host-side deployment example. For instance, this embodiment demonstrates a process for establishing dual transmission paths for two Worker nodes and one reference server (PS) node. All Worker nodes and PS nodes follow standard procedures to establish a reliable connection (RC). Worker nodes perform gradient data transmission. PS nodes perform parameter transmission.

[0046] Worker node 1 has both slow and fast path modes. The slow path mode includes a local gradient data transmission queue pair (RC QP) with local QPNs (queue pair numbers) #1 and #3 forming a reliable connection. The fast path mode includes a remote transmission queue pair (virtual RC QP, or VQP) with remote QPNs #5 and #7 forming a virtual reliable connection. Worker node 2 also has slow and fast path modes. The slow path mode includes a local transmission queue pair (RC QP) with local QPNs #2 and #4 forming a reliable connection. The fast path mode includes a remote transmission queue pair (VQP) with remote QPNs #6 and #7 forming a virtual reliable connection. Each Worker node creates one vQP.

[0047] In this embodiment, the PS node has two slow path modes and one fast path mode. The slow path mode includes two reliable data transmission queue pairs (RC QPs) consisting of local QPNs #3 and #1, and local QPNs #4 and #2. The fast path mode includes a virtual reliable data transmission queue pair (VQP) consisting of remote QPNs #7 and #5. The PS node creates a vQP. The local QPN and remote QPN are used as the context information for the connection between the Worker node and the PS node. The specific deployment of the Worker node and PS node on the host side is described below:

[0048] 1. Establish connections between Worker nodes and PS nodes: Worker node #1's local QPN is connected to RC QP #3 on the PS node, and vice versa. Worker node #1's local QPN is connected to RC QP #1 on the PS node; Worker node #2's local QPN is connected to RC QP #4 on the PS node, and vice versa. In other words, in slow path mode, each Worker node establishes a point-to-point RC connection with the PS node according to the traditional PS node architecture.

[0049] The PS node assigns vQPs with remote QPNs #7 and #5 as destinations to all Worker nodes with remote QPNs #5 and #7, and designates the vQP of Worker 1 as the destination for the PS node's vQP. For example, Worker nodes 1 and 2 each have a vQP (with remote QPNs #5 and #7 respectively, and remote QPNs #6 and #7), and designate the vQP with remote QPN #7 as the destination. In other words, the fast path mode establishes a many-to-one virtual RC connection between all Worker nodes and the PS node.

[0050] 2. Implementing Aggregation on the Switch Side: Upon successful memory request, host-side 1, acting as the terminal, streams gradient data packets in fast path mode to determine the switch's reception of these packets. For example, the host-side determines the switch's behavior when receiving gradient data packets by allowing the gradient data packets to carry an isINA field indicating whether intra-network aggregation is performed. If the isINA field is set to 0, the switch forwards the packet to the PS node without modification. If isINA is set to 1, switch-side 2 performs intra-network aggregation in aggregation module 16. After aggregation is complete, the switch modifies the gradient data packet header and payload, forwarding the gradient data packet to the vQP on PS node 4 for further processing.

[0051] 3. Implement intra-network ACK broadcast: When gradient data packets arrive at the vQP of PS node 4, the RDMA network card is used to enable intra-network ACK broadcast to the vQP of all Worker nodes to restore the connection status of all Worker nodes. This intra-network ACK broadcast mechanism ensures that all Worker nodes can receive the necessary ACK data packets, thereby maintaining a consistent connection status and preventing unnecessary retransmissions.

[0052] Further description of the related technologies is as follows:

[0053] (1) All-reduce-level memory requests

[0054] Deep neural network training tasks iteratively involve communication and computation phases, known as the "on-and-off" training mode. During the communication phase, an all-reduce operation is typically required to synchronize parameters. Based on this, this invention employs an all-reduce level memory request strategy. The specific strategy is as follows:

[0055] When a task enters the communication phase, the host side transmits data packets via slow path mode by default. This default behavior allows PS nodes to provide the necessary guarantees for gradient aggregation when there is insufficient memory on the switch side. This invention allows each Worker node to periodically (every Round-Trip Time) set the memory request flag isRequest to request memory from the switch side.

[0056] (2) Memory allocation algorithm based on first-served strategy

[0057] This invention divides the switch memory into multiple memory pools, each containing a fixed number of aggregators (containing memory of the latency-bandwidth product size), which is sufficient for the switch to process aggregated network packets at line rate. Unlike previous work, this invention can dynamically allocate and deallocate memory pools for each task throughout its lifecycle. Figure 3 As shown, the specific process of the memory pool allocation algorithm is as follows:

[0058] When the switch receives a data packet, it parses the destination QPN field in the data packet header to obtain the task identification number JOBID and the requested memory pool sub-index IDX (JOBID%N).

[0059] The initial state of the memory pool is idle. The allocation status of N memory pools is tracked. If the data packet carries the memory request flag isRequest and comes from the Worker node, the switch side calls memory allocation until all memory pool allocation requests are completed.

[0060] Ultimately, the switch forwards this data packet to the reference server node.

[0061] The process for determining whether memory allocation has been completed in the memory pool is as follows: If memory allocation has not yet been completed, the JOBID is written into the memory pool s_pools on the switch side to indicate that the memory pool has been allocated to the current task; after memory allocation is completed, the switch returns the current content of s_pools. If the return value is equal to the current task ID, it means that memory allocation was successful, and the switch sets the request success flag isAllocate to 1.

[0062] like Figure 4 The diagram shown is an example of the code implementation for the memory pool allocation algorithm. The specific process is as follows:

[0063] Assuming there are N memory pools, this invention uses N registers named s_pools to track the allocation status of each memory pool. Initially, all s_pools registers are set to -1, indicating that all memory pools are idle. When the switch receives a data packet, this invention first uses the destination QPN field in the packet header to look up the table to obtain the task number JOBID and the requested memory pool sub-index IDX (JOBID%N).

[0064] When the switch receives a data packet, it parses the packet header (line 11). If the packet carries the memory request bit `isRequest` (line 12) and originates from a Worker node (line 13), the switch invokes the memory allocation procedure (line 14). This procedure determines whether the requested pool has already been allocated. If not (line 2), the switch writes the JOBID into `s_pools` (line 3) to indicate that the memory pool has been allocated to the current task (line 9). After memory allocation is complete, the switch returns the current contents of `s_pools` (line 5). If the return value equals the current task ID (line 15), it indicates successful memory allocation, and the switch sets the request success flag `isAllocate` to 1 (line 16). Finally, the switch forwards this data packet to the PS (line 22).

[0065] (3) Reference server node request aggregation and feedback

[0066] Considering that different Worker nodes may not be in the same rack, the memory request for the current task is considered successful only when the memory requests of all Worker nodes are successful. Therefore, the PS (Service Provider) is responsible for collecting the memory allocation results of all Worker nodes, aggregating the results, and returning them to each Worker. The specific process is as follows:

[0067] When the reference server node receives a data packet with the memory request flag (isRequest) from all worker nodes, it first verifies whether all data packets carry the memory request success flag (isAllocate). If so, the memory request for this task is successful, and the reference server node sets the task request success flag (isSuccess) in its broadcast parameter packet. Otherwise, the memory request for this task fails. When a worker node receives a data packet carrying the isSuccess flag, the worker node can choose to transmit the data packet on the fast path to enjoy the benefits of network aggregation.

[0068] (4) Memory release

[0069] To support multi-tenancy, switch memory should be dynamically released to make it available for other tasks. Specifically, there are two situations where the memory pool needs to be released:

[0070] The first scenario is memory allocation failure. This occurs when the switch receives a data packet from the PS node that carries the memory request flag "isRequest" but lacks the task request success flag "isSuccess".

[0071] In the second scenario, all-reduce has been completed. If the switch receives a data packet with the aggregation end flag "isEnd", it indicates that the communication phase has ended.

[0072] To address both of these scenarios, this invention allows the switch to invoke the memory release procedure. During this procedure, the switch sets s_pools to -1, indicating that no task is currently using this memory pool.

[0073] In summary, the multi-tenant on-network aggregation transmission system and method for RDMA networks of the present invention combines dual transmission paths with all-reduce level memory requests. Specifically, ① to support RDMA RC compatibility, the present invention employs dual transmission paths: the ISA uses a slow path, and the INA uses a fast path. For the slow path, the present invention reuses the traditional point-to-point connection of the PS. For the fast path, the present invention constructs a many-to-one virtual connection for each terminal host and jointly designs the switch and host logic to restore lost connection states. In this way, the present invention can support RDMA RC compatibility without modifying commercial RDMA network cards, while ensuring that the ISA transmission on the slow path is not affected. To support multi-tenant learning, the dual-path design ensures that when the switch memory is exhausted, the INA will revert to the default ISA. ② To improve the efficiency of memory switching, the present invention further utilizes an all-reduce level strategy to allocate memory among multiple tasks, which follows the well-known "on and off" training mode. That is, it allows each task to request memory when entering the communication phase and release memory during the computation phase. Therefore, this invention can enjoy the low latency and high throughput brought by RDMA, while also supporting multiple tasks to share scarce memory, providing scalability for on-network aggregation.

[0074] The key insight of this strategy lies in the host-side decision-making regarding intra-network aggregation. First, dynamic hosts on the host side can pre-determine packet behavior (ISA or INA), thus proactively deciding the necessity of retransmissions to support RDMA compatibility. Furthermore, host-side drivers enable training tasks to dynamically leverage the advantages of INA based on their aggregation needs. In other words, one task can proactively free up memory for other tasks to support multi-tenant training.

[0075] It should be noted that although the present invention has been shown and described with reference to specific exemplary embodiments thereof, those skilled in the art should understand that the present invention is not limited to the above embodiments.

[0076] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All modifications and improvements made without departing from the spirit of the invention should fall within the scope of protection of this application.

Claims

1. A multi-tenant on-network aggregation transmission system suitable for RDMA networks, characterized in that, It includes a host side (1) and a switch side (2). At least one worker node (3) and at least one parameter server node (4) are deployed on the host side (1). The worker node (3) is connected to the switch side (3) through a first RDMA network card (13), and the parameter server node (4) is connected to the switch side (3) through a second RDMA network card (14). The Worker node (3) is equipped with a rate limiter (5) and a dual transmission path control module (6); the rate limiter (5) is used to limit the traffic sent by a single task to the network; the dual transmission path control module (6) provides dual transmission path control logic on the host side, and further includes a slow path mode (7), a fast path mode (8) and a marker (9); the slow path mode (7) is used for aggregation within the server, the fast path mode (8) is used for aggregation on the network; the marker (9) is used to insert a request flag into the data packets aggregated within the server. The parameter server node (4) is equipped with a memory request module (10) for all-reduce level memory requests in the parameter server node (4); the memory request module (10) further includes a data / request aggregation module (11), a feedback module (12), and a fast / slow path mode (18); the data / request aggregation module (11) is used to aggregate gradient data and memory request results from all the Worker nodes (3) to the parameter server node (4); The feedback module 12 is used to return the parameter data and memory request results sent from the PS node to the Worker node (3). The fast / slow path mode (18) is used to form a reliable connection transmission queue pair and a virtual reliable connection transmission queue pair with the slow path mode (7) and fast path mode (8) in the Worker node. The switch side (3) is equipped with a memory allocation module (16), an aggregation module (17) and a broadcast module (18); the memory allocation module (16) adopts a memory allocation algorithm based on a first-come-first-served strategy to support the training task and provide dual transmission path control for the host side (1); the aggregation module (17) is used to perform on-network aggregation on the switch side, and the broadcast module (18) is used to distribute lost gradient data packets to restore the connection state and return parameter packets to the parameter server node (4).

2. The multi-tenant on-network aggregation transmission system suitable for RDMA networks according to claim 1, characterized in that, The slow path mode and fast path mode of the Worker node (3) respectively include data transmission queue pairs of point-to-point reliable connections and data transmission queue pairs of many-to-one virtual reliable connections; The parameter server node (4) includes a data transmission queue pair for point-to-point reliable connection and a data transmission queue pair for one-to-many virtual reliable connection; wherein, the data transmission queue pair for point-to-point reliable connection in the Worker node (3) is connected to the data transmission queue pair for point-to-point reliable connection in the parameter server node (4); the data transmission queue pair for one-to-many virtual reliable connection in the parameter server node (4) is connected to the data transmission queue pair for many-to-one virtual reliable connection in the Worker node (3).

3. A multi-tenant on-network aggregation transmission system suitable for RDMA networks according to claim 2, characterized in that, The slow path mode (7) establishes a reliable point-to-point connection between the Worker node (3) and the parameter server node (4).

4. A multi-tenant on-network aggregation transmission system suitable for RDMA networks according to claim 2, characterized in that, The fast path mode (8) establishes a many-to-one virtual reliable connection between the Worker node and the parameter server node (4).

5. A multi-tenant on-network aggregation transmission system suitable for RDMA networks according to claim 2, characterized in that, When a task enters the communication phase, the host side (1) transmits data packets by default through the slow path mode.

6. A multi-tenant on-network aggregation transmission system suitable for RDMA networks according to claim 1, characterized in that, The memory allocation algorithm based on the first-come, first-served strategy includes: When the switch receives a data packet, it parses the destination QPN field in the data packet header to obtain the task identification number and the requested memory pool sub-index IDX. Track the allocation status of N memory pools. If the data packet carries a memory request flag and comes from a Worker node, the switch side calls memory allocation until all memory pool allocation requests are completed. Ultimately, the switch forwards this data packet to the reference server node.

7. A multi-tenant on-network aggregation transmission method suitable for RDMA networks, characterized in that, include: Construct a multi-tenant on-network aggregation transmission system suitable for RDMA networks according to any one of claims 1-6, and realize the connection between the Worker node and the reference server node on the host side; On the switch side, a memory allocation algorithm based on a first-come, first-served strategy is used to control the host side in fast path mode. The host side uses the isINA field carried by the Worker node gradient data packet to determine that the switch side has received the gradient data packet transmitted from the Worker node. The aggregation module on the switch side performs on-network aggregation of the gradient data packet. After the aggregation is completed, the gradient data packet header and payload are modified on the switch side, and the gradient data packet is forwarded to the transmission queue pair of the virtual reliable connection on the reference server. To implement intra-network ACK broadcasting, when gradient data packets arrive at the transmission queue of the virtual reliable connection on the reference server node, intra-network ACK broadcasting is enabled to the transmission queue of the virtual reliable connection on all worker nodes in order to restore the connection state of all worker nodes.

Images