Distributed network data management gateway based on rdma multi-path and dynamic slicing

Abstract

This invention provides a distributed network data management gateway based on RDMA multipathing and dynamic slicing, comprising a sender-end processing module, an intelligent multipath scheduler, an in-transit slice state machine, a predictive congestion control module, a selective retransmission engine, and a receiver-end zero-copy reassembly module. It can also be configured with a cross-layer collaborative optimization module. The sender-end processing module generates differentiated data slices (compute-intensive, communication-intensive, etc.) based on GPU computing task load characteristics. The intelligent multipath scheduler distributes slices to matching RDMA transmission paths using an adaptive weighted scheduling algorithm. Combined with in-transit slice state management, predictive congestion control, differentiated slice retransmission, and receiver-end intelligent NIC hardware-level zero-copy reassembly, cross-layer collaboration between GPU computing and network transmission is achieved. This invention effectively utilizes multi-NIC resources, significantly improves network bandwidth utilization, reduces tail latency, eliminates receiver-end CPU overhead, improves GPU utilization, and significantly optimizes overall service processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of distributed network data communication technology, and is applicable to scenarios with extremely high requirements for network bandwidth and latency, such as high-performance computing, data center networks, and distributed storage. Specifically, it relates to a distributed network data management gateway based on RDMA multipath and dynamic slicing. Background Technology

[0002] With the development of cloud computing, big data, and artificial intelligence technologies, distributed systems are placing increasingly higher demands on network communication performance. Traditional network communication methods suffer from the following technical problems: (1) Single-path transmission bottleneck: Traditional RDMA communication mainly relies on a single path for data transmission, which cannot make full use of the bandwidth resources of multiple network cards in parallel processing, resulting in low network bandwidth utilization.

[0003] (2) Low efficiency of packet loss retransmission: In the existing technology, when network packet loss occurs, it is usually necessary to retransmit the entire data stream or a large data block, resulting in unnecessary bandwidth waste and increased latency.

[0004] (3) Insufficient congestion control precision: The existing RDMA congestion control mechanism is difficult to maintain high throughput in high bandwidth scenarios (such as 100Gbps and above), which can easily lead to network congestion and performance degradation.

[0005] (4) Low efficiency of GPU and network device collaboration: GPU computing resources and network communication resources fail to collaborate effectively, especially in the multi-NIC environment where there is a lack of efficient parallel scheduling mechanism.

[0006] (5) Tail Latency Problem: Existing distributed network gateways have significant tail latency during data transmission, which affects overall system performance and user experience.

[0007] Currently, common solutions in the industry include: using single-path RDMA for data transmission, but with low bandwidth utilization; using the traditional TCP / IP protocol stack, but with high protocol processing overhead; and implementing simple multi-path load balancing, but lacking fine-grained flow control and packet loss recovery mechanisms.

[0008] Therefore, there is an urgent need for a distributed network data management gateway technology solution that can effectively utilize multiple network interface card resources, improve bandwidth utilization, and reduce latency. Summary of the Invention

[0009] The purpose of this invention is to provide a distributed network data management gateway based on RDMA multipath and dynamic slicing, which integrates load characteristic-aware differentiated slicing, predictive congestion control, cross-layer collaborative optimization, and hardware-level zero-copy reassembly.

[0010] To achieve the above objectives, the present invention is implemented through the following technical solution: A distributed network data management gateway based on RDMA multipathing and dynamic slicing includes: The sending end processing module is configured on the GPU memory side and is used to register the memory regions of multiple network cards through the uDAPL interface. It dynamically generates differentiated data slices based on the load characteristics of GPU computing tasks. The differentiated data slices include compute-intensive slices, communication-intensive slices and hybrid load slices. Each type of slice has an adaptive slice size. The intelligent multipath scheduler, which is communicatively connected to the sending end processing module, is used to distribute differentiated data slices to different RDMA Queue Pairs based on the real-time performance indicators and slice type characteristics of each network path using an adaptive weighted scheduling algorithm. The adaptive weighted scheduling algorithm dynamically adjusts the weight coefficients according to the real-time bandwidth utilization, latency jitter, and queue depth of the path. Computation-intensive slices are preferentially allocated to high-bandwidth paths, and communication-intensive slices are preferentially allocated to low-latency paths. The in-transit slice state machine, set in the gateway, is used to maintain an independent in-transit slice counter and slice type distribution matrix for each network path. It tracks the number of data slices that have been sent but not acknowledged on each path and their type composition. When the total number of in-transit slices on a certain path reaches the first threshold or the number of slices of a specific type reaches the second threshold, the traffic shaping mechanism is triggered to suspend the sending of data slices of the corresponding type to that path. The predictive congestion control module integrates the DCQCN congestion control algorithm and a lightweight machine learning model. It is used to predict the congestion probability within a future time window based on historical congestion patterns, current network status trends, and slice type characteristics, and adjust the transmission rate in advance. When the predicted congestion probability exceeds a preset threshold, the transmission rate of communication-intensive slices is reduced first, while the transmission priority of computation-intensive slices is maintained. The selective retransmission engine is used to execute differentiated retransmission strategies based on slice type and priority when slice loss is detected. High-priority slices trigger immediate retransmission, while low-priority slices use aggregated retransmission or retransmission after timeout. The receiver-side zero-copy reassembly module is configured at the smart NIC hardware layer. It is used to directly sort and reassemble data slices in the NIC hardware based on the packet sequence number and slice type identifier, without CPU intervention. After reassembly, the complete data stream is directly written to the target GPU memory area through RDMA write operation.

[0011] Furthermore, the adaptive weighted scheduling algorithm includes: Real-time acquisition of bandwidth utilization B_i(t), latency D_i(t), and queue depth Q_i(t) for each path; calculation of the path comprehensive score S_i(t) = α·(1-B_i(t)) + β·(1 / D_i(t)) + γ·(1 / Q_i(t)), where α, β, and γ are dynamic weight coefficients; adjustment of weight coefficients according to slice type T_j: increase the value of α when T_j is computationally intensive; increase the value of β when T_j is communication intensive; selection of transmission path based on polling probability P_i(t) = S_i(t) / ΣS_k(t).

[0012] Furthermore, the in-transit slice state machine also includes: The slice type distribution matrix M, where M[i][j] represents the number of slices of type j on path i; the dynamic threshold adjustment unit is used to optimize the first threshold and the second threshold using a reinforcement learning algorithm based on network topology changes and historical performance data; the path isolation mechanism temporarily prohibits slices of type j from using the path when the packet loss rate of a certain type of slice on a specific path exceeds the threshold.

[0013] Furthermore, the predictive congestion control module includes: A temporal feature extractor is used to extract temporal feature vectors of network state; a lightweight LSTM prediction model is deployed on the gateway side to predict the probability of future congestion based on the state sequence of the past N time windows; a rate pre-regulator is used to reduce the transmission rate in advance within a time window T_pre before congestion occurs, wherein T_pre is dynamically calculated based on path delay; and a slice type-aware scheduler is used to reschedule low-priority slices to backup paths or delay transmission when the congestion prediction is true.

[0014] Furthermore, the zero-copy reassembly module at the receiving end includes: A hardware-level sorting buffer, integrated into the smart network card, is used to cache out-of-order data slices; a type-aware reassembly engine uses different reassembly strategies based on the slice type identifier: compute-intensive slices are reassembled in strict order, while communication-intensive slices are allowed limited out-of-order reassembly; and a direct memory write engine is used to write data directly to GPU memory via RDMA operations after reassembly, bypassing host memory and CPU processing.

[0015] Furthermore, it also includes a cross-layer collaborative optimization module, which is used to establish a joint scheduling interface for GPU computing load and network transmission. When a GPU computing bottleneck is detected, the sending rate of the corresponding data stream is reduced. When network congestion is detected, the GPU side is notified to adjust the granularity of computing tasks.

[0016] Furthermore, the adaptive size generation mechanism for the differentiated data slices includes: A slice size prediction model based on GPU kernel execution history; a network bandwidth detection unit for real-time detection of available bandwidth; and a dynamic slice generator for dynamically selecting the optimal slice size within the range of [4KB, 1MB] based on the prediction model output and available bandwidth, where large slices are used to reduce overhead in high-bandwidth scenarios and small slices are used to reduce retransmission costs in high-latency scenarios.

[0017] Furthermore, the intelligent multi-path scheduler supports heterogeneous path collaboration, including: The heterogeneous path identification unit is used to identify and classify network paths with different bandwidth levels and latency characteristics; the differentiated service mapping unit is used to optimally match slice types with path characteristics; and the path fault prediction unit is used to predict potential faults based on historical path fault patterns and migrate traffic in advance.

[0018] Compared with the prior art, the present invention has the following advantages: This invention is a distributed network data management gateway based on RDMA multipath and dynamic slicing. It integrates load characteristic-aware differentiated slicing, predictive congestion control, cross-layer collaborative optimization, and hardware-level zero-copy reassembly, achieving beneficial effects such as performance improvement, congestion prevention, resource optimization, and load adaptation.

[0019] Specifically: (1) Performance improvement: Compared with traditional single-path RDMA, the effective bandwidth is increased by 50-65%, and the tail latency (P99) is reduced from 90μs to 15-20μs, a reduction of about 78%; (2) Congestion prevention: Predictive control reduces congestion events by more than 60%, and the throughput of 100Gbps links is stabilized at 98-99%; (3) Resource optimization: Zero-copy reassembly eliminates the CPU overhead at the receiver, and the GPU utilization rate is increased by 8-12%; (4) Load adaptation: The overall training efficiency is improved by 25-40% in heterogeneous load scenarios. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the system architecture of the present invention. Detailed Implementation

[0021] The embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.

[0022] This invention relates to a distributed network data management gateway based on RDMA multipath and dynamic slicing, as shown in the appendix. Figure 1 .

[0023] The following presents specific embodiments of the present invention, combining three typical application scenarios: GPU distributed training in high-performance computing data centers, RDMA data interaction in distributed storage, and low-latency communication in high-frequency financial trading, to verify the actual performance and adaptability of the gateway. Example 1: GPU Distributed Training Scenario (Hybrid Computation + Communication Load)

[0024] Application scenario: An 8-GPU cluster (NVIDIA A100) in an artificial intelligence data center performs distributed training tasks for large models. During the training process, there are a lot of model parameter synchronization (computation-intensive) and gradient transmission (communication-intensive). The cluster is configured with 4 100Gbps RDMA smart network cards. Traditional single-path RDMA has problems such as low bandwidth utilization and large tail latency, which limits training efficiency.

[0025] Gateway deployment: The distributed network data management gateway of the present invention is deployed in the DPU of the GPU cluster. Each GPU card is registered with one RDMA memory region. Four network cards are configured with four RDMA transmission paths. Each module of the gateway is configured according to the above implementation method. The slice size range is set to [64KB, 1MB]. The time window N of the LSTM congestion prediction model is 20, and the congestion threshold is 70%.

[0026] Implementation process: The S101 transmitter processing module collects the load characteristics of the GPU training task and divides the model parameter synchronization data into computationally intensive slices (size 512KB-1MB) and gradient transmission data into communication-intensive slices (size 64KB-256KB). The S102 intelligent multipath scheduler collects the performance metrics of the four paths in real time, prioritizing the allocation of computationally intensive slices to high-bandwidth paths (bandwidth utilization <30%) and communication-intensive slices to low-latency paths (latency <2μs). The S103 in-transit slice state machine maintains independent counters for the four paths. When the in-transit compute-intensive slices for a certain path reach the threshold, the transmission of compute-intensive slices for that path is paused to avoid queue overflow. The S104 predictive congestion control module predicts the congestion probability in real time. When it detects that the congestion probability reaches 75% during the peak of gradient transmission, it reduces the transmission rate of communication-intensive slices by 20% 2×2μs=4μs in advance to maintain the transmission priority of model parameter synchronization slices. The S105 receiver's zero-copy reassembly module completes slice reassembly in the smart network card. Computationally intensive slices are reassembled in a strictly sequential manner, while gradient data is reassembled in a limited out-of-order manner. After reassembly, the data is directly written to the GPU memory, with no CPU overhead. When the S106 cross-layer collaborative optimization module detects that the computing utilization of a certain GPU card has reached 98%, it automatically reduces the gradient transfer rate of the card by 30%, and restores the rate after the computing utilization drops to 90%.

[0027] Implementation results: Compared with traditional single-path RDMA, the cluster's effective network bandwidth is increased by 62%, tail latency (P99) is reduced from 90μs to 18μs, a reduction of 78%, GPU utilization is increased by 11%, the overall efficiency of large model distributed training is improved by 38%, and the training time per round is shortened by 32%. Example 2: Distributed Storage RDMA Data Interaction Scenario (High Bandwidth, High Throughput Requirements)

[0028] Application scenario: A distributed storage cluster (Ceph) in a cloud computing data center uses RDMA for data interaction between storage nodes. The cluster is configured with 100Gbps RDMA links. Traditional multi-path load balancing solutions lack fine-grained scheduling, and the link throughput can only reach 70-80Gbps, resulting in frequent congestion and low efficiency in packet loss retransmission.

[0029] Gateway deployment: Deploy the gateway of this invention in the smart network card of the storage node, configure two 100Gbps RDMA paths, disable the mixed load slicing configuration, generate only compute-intensive slices (storage data read and write have high bandwidth requirements), set the slice size to 512KB-1MB, set the first threshold Th1 to 80% of the queue length, and adjust the rate reduction of the predictive congestion control module as follows: 30% reduction when the congestion probability is 70%-80%, and 60% reduction when the probability is above 80%.

[0030] Implementation process: The S201 sending end processing module divides the stored data into computationally intensive large slices (1MB), registers the memory regions of the storage nodes through the uDAPL interface, and enables direct access to the data; The S202 intelligent multipath scheduler adjusts the α weight to 0.7, prioritizing the allocation of slices to paths with lower bandwidth utilization, thus achieving load balancing between the two paths. The S203 predictive congestion control module, combined with the DCQCN algorithm, predicts link congestion in real time. When the throughput reaches 90Gbps, it predicts the congestion probability in advance by 72% and reduces the speed by 30% 5μs in advance to avoid link congestion. When a small number of slices are lost, the selective retransmission engine in S204 triggers an immediate retransmission to ensure the integrity of the stored data and reduce retransmission overhead by 65%. After the S205 receiver's zero-copy reassembly module completes the strict sequential reassembly of the large slice, it directly writes it into the memory of the storage node, eliminating the CPU's reassembly and copy overhead.

[0031] Implementation results: The throughput of the 100Gbps RDMA link stabilized at 98.5Gbps, the link utilization increased by 23%, congestion events decreased by 68%, packet loss retransmission overhead decreased by 65%, data read and write latency between storage nodes decreased by 45%, and the overall IOPS of the distributed storage cluster increased by 55%. Example 3: Low-latency communication scenario for high-frequency financial trading (ultra-low latency and high reliability requirements)

[0032] Application scenario: A financial institution's high-frequency trading system uses RDMA for data transmission of market data and issuance of trading orders between trading nodes. It has extremely high requirements for network latency (microsecond level). Traditional RDMA solutions have problems such as large tail latency fluctuations and sudden increases in latency during congestion, which affect the efficiency of issuing trading orders.

[0033] Gateway deployment: Deploy the gateway of this invention on the low-latency RDMA network card (latency <1μs) of the high-frequency transaction node, configure two 25Gbps low-latency RDMA paths, disable large slice configuration, set the slice size to 4KB-64KB, set communication-intensive slices to the highest priority, lighten the LSTM model of the predictive congestion control module, inference time <1μs, and set the timeout retransmission timer to 10μs.

[0034] Implementation process: The S301 transmitter processing module divides market data and trading instructions into communication-intensive small slices (4KB-32KB), carrying high-priority identifiers, and the slice generation time is <0.5μs; The S302 intelligent multipath scheduler adjusts the β weight to 0.8, prioritizing the allocation of slices to low-latency paths with a latency of <1μs, and the path switching time is <1μs; The S303 predictive congestion control module monitors path delay jitter in real time. When the delay jitter increases from 0.2μs to 0.5μs, the predicted congestion probability reaches 70%, reducing the transmission rate of non-transaction instruction slices by 2×1μs=2μs in advance to ensure ultra-low latency transmission of transaction instruction slices. The S304 in-transit slice state machine sets the second threshold Th2 to 40% of the queue length to strictly control the number of communication-intensive slices in transit on a single path and avoid queue delays. The S305 receiver's zero-copy reassembly module employs limited out-of-order reassembly, allowing up to three slices to be out of order. The reassembly time is less than 1μs. After reassembly, the transaction instructions are directly written into the transaction system's memory, achieving microsecond-level communication.

[0035] Implementation results: The end-to-end communication latency between trading nodes is stabilized at 3-5μs, the tail latency (P99) is reduced from 20μs to 4μs, latency jitter is reduced by 80%, congestion events are reduced by 72%, and the response time for trading instructions is shortened by 60%, meeting the ultra-low latency and high reliability requirements of high-frequency financial trading.

[0036] The gateway of the present invention underwent a 30-day continuous performance test in the scenarios of the above three embodiments. The test indicators included effective bandwidth, tail latency (P99), number of congestion events, GPU / CPU utilization, and overall service efficiency. The test results showed that the core performance indicators were improved compared with the traditional RDMA solution and the traditional multi-path load balancing solution.

[0037] Test results show that the distributed network data management gateway based on RDMA multipath and dynamic slicing of the present invention effectively solves the problems of single-path bottleneck, low congestion control accuracy, and low GPU and network collaboration efficiency of traditional RDMA solutions. It achieves a significant improvement in network bandwidth utilization and a significant reduction in latency, while eliminating CPU overhead at the receiving end and improving the utilization of computing resources. It is suitable for scenarios with extremely high requirements for network bandwidth and latency, such as high-performance computing, distributed storage, and high-frequency financial trading.

[0038] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the concept of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A distributed network data management gateway based on RDMA multi-path and dynamic slicing, characterized in that, include: The sending end processing module is configured on the GPU memory side and is used to register the memory regions of multiple network cards through the uDAPL interface. It dynamically generates differentiated data slices based on the load characteristics of GPU computing tasks. The differentiated data slices include compute-intensive slices, communication-intensive slices and hybrid load slices. Each type of slice has an adaptive slice size. The intelligent multipath scheduler, which is communicatively connected to the sending end processing module, is used to distribute differentiated data slices to different RDMAQueue Pairs based on the real-time performance indicators and slice type characteristics of each network path using an adaptive weighted scheduling algorithm. The adaptive weighted scheduling algorithm dynamically adjusts the weight coefficients according to the real-time bandwidth utilization, latency jitter, and queue depth of the path. Computation-intensive slices are preferentially allocated to high-bandwidth paths, and communication-intensive slices are preferentially allocated to low-latency paths. The in-transit slice state machine, set in the gateway, is used to maintain an independent in-transit slice counter and slice type distribution matrix for each network path. It tracks the number of data slices that have been sent but not acknowledged on each path and their type composition. When the total number of in-transit slices on a certain path reaches the first threshold or the number of slices of a specific type reaches the second threshold, the traffic shaping mechanism is triggered to suspend the sending of data slices of the corresponding type to that path. The predictive congestion control module integrates the DCQCN congestion control algorithm and a lightweight machine learning model. It is used to predict the congestion probability within a future time window based on historical congestion patterns, current network status trends, and slice type characteristics, and adjust the transmission rate in advance. When the predicted congestion probability exceeds a preset threshold, the transmission rate of communication-intensive slices is reduced first, while the transmission priority of computation-intensive slices is maintained. The selective retransmission engine is used to execute differentiated retransmission strategies based on slice type and priority when slice loss is detected. High-priority slices trigger immediate retransmission, while low-priority slices use aggregated retransmission or retransmission after timeout. The receiver-side zero-copy reassembly module is configured at the smart NIC hardware layer. It is used to directly sort and reassemble data slices in the NIC hardware based on the packet sequence number and slice type identifier, without CPU intervention. After reassembly, the complete data stream is directly written to the target GPU memory area through RDMA write operation.

2. The RDMA multi-path and dynamic slice based distributed network data management gateway of claim 1, wherein: The adaptive weighted scheduling algorithm includes: Real-time acquisition of bandwidth utilization B_i(t), latency D_i(t), and queue depth Q_i(t) for each path; calculation of the path comprehensive score S_i(t) = α·(1-B_i(t)) + β·(1 / D_i(t)) + γ·(1 / Q_i(t)), where α, β, and γ are dynamic weight coefficients; adjustment of weight coefficients according to slice type T_j: increase the value of α when T_j is computationally intensive; increase the value of β when T_j is communication intensive; selection of transmission path based on polling probability P_i(t) = S_i(t) / ΣS_k(t).

3. The distributed network data management gateway based on RDMA multipath and dynamic slicing according to claim 1, characterized in that: The in-transit slice state machine also includes: The slice type distribution matrix M, where M[i][j] represents the number of slices of type j on path i; the dynamic threshold adjustment unit is used to optimize the first threshold and the second threshold using a reinforcement learning algorithm based on network topology changes and historical performance data; the path isolation mechanism temporarily prohibits slices of type j from using the path when the packet loss rate of a certain type of slice on a specific path exceeds the threshold.

4. The distributed network data management gateway based on RDMA multipath and dynamic slicing according to claim 1, characterized in that: The predictive congestion control module includes: A temporal feature extractor is used to extract temporal feature vectors of network state; a lightweight LSTM prediction model is deployed on the gateway side to predict the probability of future congestion based on the state sequence of the past N time windows; a rate pre-regulator is used to reduce the transmission rate in advance within a time window T_pre before congestion occurs, wherein T_pre is dynamically calculated based on path delay; and a slice type-aware scheduler is used to reschedule low-priority slices to backup paths or delay transmission when the congestion prediction is true.

5. The distributed network data management gateway based on RDMA multipath and dynamic slicing according to claim 1, characterized in that: The receiver zero-copy reassembly module includes: A hardware-level sorting buffer, integrated into the smart network card, is used to cache out-of-order data slices; a type-aware reassembly engine uses different reassembly strategies based on the slice type identifier: compute-intensive slices are reassembled in strict order, while communication-intensive slices are allowed limited out-of-order reassembly; and a direct memory write engine is used to write data directly to GPU memory via RDMA operations after reassembly, bypassing host memory and CPU processing.

6. The distributed network data management gateway based on RDMA multipath and dynamic slicing according to claim 1, characterized in that: It also includes a cross-layer collaborative optimization module, which is used to establish a joint scheduling interface for GPU computing load and network transmission. When a GPU computing bottleneck is detected, the sending rate of the corresponding data stream is reduced. When network congestion is detected, the GPU side is notified to adjust the granularity of computing tasks.

7. The distributed network data management gateway based on RDMA multipath and dynamic slicing according to claim 1, characterized in that: The adaptive size generation mechanism for the differentiated data slices includes: A slice size prediction model based on GPU kernel execution history; a network bandwidth detection unit for real-time detection of available bandwidth; and a dynamic slice generator for dynamically selecting the optimal slice size within the range of [4KB, 1MB] based on the prediction model output and available bandwidth, where large slices are used to reduce overhead in high-bandwidth scenarios and small slices are used to reduce retransmission costs in high-latency scenarios.

8. The distributed network data management gateway based on RDMA multipath and dynamic slicing according to any one of claims 1 to 7, characterized in that: The intelligent multi-path scheduler supports heterogeneous path collaboration, including: The heterogeneous path identification unit is used to identify and classify network paths with different bandwidth levels and latency characteristics; the differentiated service mapping unit is used to optimally match slice types with path characteristics; and the path fault prediction unit is used to predict potential faults based on historical path fault patterns and migrate traffic in advance.

Images