Cross-platform distributed synchronous simulation method adopting RoCEv2

Abstract

The invention discloses a cross-platform distributed synchronous simulation method adopting RoCEv2, and relates to the technical field of communication simulation, and the method comprises the following steps: configuring parameter information of a switch and nodes, and constructing a heterogeneous distributed network of a RoCEv2 enhanced network protocol; deploying a master node global logic time reference, calibrating a slave node clock, starting a sliding window to continuously monitor offset, dynamically detecting clock offset to trigger adjustment, optimizing a time step length in combination with an ECN mark, and dynamically compensating the clock offset of the heterogeneous platform; sequencing events according to logic timestamps to generate DAG topology, and preloading high-frequency events; event data are transmitted through cross-node RDMA, and the main node is sorted and scheduled according to a logic timestamp, so that cross-platform distributed parallel simulation and consistency verification are realized. The method can improve the processing efficiency and fault-tolerant capability of the communication simulation system on large-scale simulation, and can be widely applied to the technical field of communication simulation.

Description

Technical Field

[0001] The present invention relates to the field of communication simulation technology, and in particular to a cross-platform distributed synchronous simulation method using RoCEv2. Background Art

[0002] With the prevalence of heterogeneous computing architectures, such as containerization (Docker / Kubernetes) and hybrid deployments of virtual machines and physical machines, cross-platform collaborative simulation faces the dual challenges of communication efficiency and time synchronization. First, industrial-grade simulations (such as digital twins and traffic simulations) require processing petabytes of discrete event data. Traditional TCP / IP protocol stacks, due to multiple memory copies, result in end-to-end latency of hundreds of microseconds and excessive CPU (central processing unit) resource utilization, making them difficult to meet real-time requirements. Second, physical clock skew across heterogeneous platforms (including lightweight container clocks, virtual machine clocks, and physical machine hardware clocks) causes inconsistent simulation logic time. Existing synchronization mechanisms, such as NTP / PTP, lack dynamic adaptability in cross-platform scenarios and cannot effectively suppress clock jitter at the virtualization layer. Furthermore, existing communication technologies have significant limitations. RoCEv1 only supports intra-subnet communication, and RoCEv2 (Remote Direct Memory Access over Ethernet), while routable, relies on lossless network guarantees. Overall, the system lacks in-depth optimization for heterogeneous protocol compatibility, further hindering the performance and reliability of large-scale simulation systems. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a cross-platform distributed synchronous simulation method using RoCEv2, which can improve the communication efficiency of cross-platform collaborative simulation, save CPU resources, improve the accuracy of time synchronization, and adapt to more complex communication simulation scenario requirements.

[0004] The present invention adopts a cross-platform distributed synchronous simulation method using RoCEv2, comprising the following steps: Configure switch and node parameters to build a heterogeneous distributed network based on the Enhanced RoCEv2 (E-RoCE) protocol. Deploy a global logical time base on the master node, calibrate the slave node clocks, enable a sliding window to continuously monitor offsets, dynamically detect clock offsets and trigger adjustments, optimize the time step size in conjunction with ECN marking, and dynamically compensate for clock offsets on heterogeneous platforms. Generate DAG topology by sorting events by logical timestamps and preload high-frequency events; Cross-node RDMA transmission of event data, the master node sorting and scheduling according to logical timestamps, to achieve cross-platform distributed parallel simulation and consistency verification.

[0005] Furthermore, the step of configuring switch and node parameter information to build a heterogeneous distributed network of the Enhanced RoCEv2 Network Protocol (E-RoCE) specifically includes: Configure PFC flow control priority on the switch, assign the highest bandwidth queue to the simulation data, and set the ECN explicit congestion marking threshold. Enable the RoCEv2 protocol stack on the node, bind UDP port 4791, and configure IP routing rules to support IPv4 / IPv6 multipath forwarding; Deploy the GPU Direct RDMA driver to establish physical address mapping between GPU memory and the network card DMA engine, achieving zero-copy data transmission; Configure the network card hardware to offload RDMA semantics (Send / Write operations), bypassing the kernel protocol stack to directly read and write user-mode memory; Configure a multipath load balancing policy based on the source port hashing algorithm. Furthermore, the steps of deploying a global logical time base on the master node, calibrating the slave node clocks, starting a sliding window to continuously monitor offsets, dynamically detecting clock offsets to trigger adjustments, optimizing the time step in combination with ECN marking, and dynamically compensating for clock offsets on heterogeneous platforms specifically include: The master node periodically broadcasts Sync messages, and the slave node records the local reception time T2 and feeds back Delay_Resp messages. The initial offset is calculated using the clock deviation formula, and the drift trend is predicted by fitting historical data within the sliding window using the least squares method. Dynamically integrate NTP coarse-grained calibration and PTP fine-grained compensation, and set the monitoring window size to filter instantaneous jitter; Real-time monitoring of the clock deviation rate within the sliding window. If the deviation exceeds the dynamic threshold, the local clock is triggered to dial back or accelerate. Parse the ECN markings of network packets and proportionally reduce the time advance step when the congestion level reaches a preset level.

[0006] Furthermore, the step of generating a DAG topology by sorting events by logical timestamps and preloading high-frequency events specifically includes: Construct a logical timestamp directed acyclic graph (DAG) based on event causality, and generate event execution sequence through topological sorting algorithm; Analyze historical simulation data, identify high-frequency event types, and preload them into the node's local cache; Furthermore, the cross-node RDMA transmission event data, the master node sorting and scheduling according to the logical timestamp, realizes the cross-platform distributed parallel simulation and consistency verification step, which specifically includes: The slave node writes local event data to the master node's memory pool through RDMA Write operations, along with a logical timestamp and dependency tag. The master node performs multi-way sorting of the event queue based on timestamps to generate a globally ordered execution sequence; When a causal conflict is detected, roll back to the last conflict-free checkpoint and re-execute; At the end of the simulation cycle, the hash summary of the result data of each node is calculated. If there is any inconsistency, the backup queue replay verification is triggered. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a step flow chart of a cross-platform distributed synchronous simulation method using RoCEv2 of the present invention.

[0008] Figure 2 This is a schematic diagram of the framework of a cross-platform distributed synchronous simulation method using RoCEv2 in the present invention.

[0009] Figure 3 This is a logic diagram of a dynamic adaptive time synchronization protocol using a cross-platform distributed synchronization simulation method of RoCEv2 in the present invention. DETAILED DESCRIPTION

[0010] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0011] Reference Figure 1 The present invention provides a cross-platform distributed synchronous simulation method using RoCEv2, the method comprising the following steps: S100, configure switch and node parameter information to build a heterogeneous distributed network based on the Enhanced RoCEv2 (E-RoCE) protocol; Specifically, PFC flow control priority is configured on the switch side, the highest bandwidth queue is allocated to the simulation data, and the ECN explicit congestion marking threshold is set; the RoCEv2 protocol stack is enabled on the node side, UDP port 4791 is bound, and IP routing rules are configured to support IPv4 / IPv6 multipath forwarding; the GPU Direct RDMA driver is deployed to establish a physical address mapping between the GPU memory and the network card DMA engine to achieve zero-copy data transmission; the network card hardware is configured to offload RDMA semantics (Send / Write operations), bypassing the kernel protocol stack to directly read and write user-mode memory; and a multipath load balancing policy is configured based on the source port hashing algorithm.

[0012] In this embodiment, a RoCEv2 Enhanced Network Protocol (E-RoCE) heterogeneous distributed network is constructed to implement zero-copy data transmission.

[0013] S200: Deploy the global logical time base of the master node, calibrate the slave node clocks, start a sliding window to continuously monitor the offset, dynamically detect clock offset and trigger adjustments, optimize the time step in combination with ECN marking, and dynamically compensate for the clock offset of heterogeneous platforms; Specifically, the master node broadcasts the Sync message, the slave node records the local reception time, and feeds back the Delay_Resp message; the initial offset is calculated through the clock deviation formula, and the least squares method is used to fit the historical data in the sliding window to predict the drift trend; the NTP coarse-grained calibration and PTP fine-grained compensation are dynamically integrated; the clock offset rate in the sliding window is monitored in real time, and if the deviation exceeds the dynamic threshold, the local clock is triggered to dial back or accelerate; the ECN mark of the network data packet is parsed, and when the congestion level reaches the preset level, the time advancement step is proportionally reduced to achieve dynamic adaptive time synchronization. Figure 3 As shown in Figure 2, the logic diagram of the dynamic adaptive time synchronization protocol is divided into the initialization phase and the runtime phase.

[0014] S210, dynamic adaptive time synchronization initialization stage; Specifically, the master node broadcasts a Sync message. The slave node monitors data packets in the data link, parses the Sync message, and sends back a Delay_Resp message to calculate the initial offset. The Sync message carries the logical timestamp T1 and is forwarded with the highest priority in the switch. The slave node records the event stamp T2 of receiving the Sync message and generates a Delay_Resp message containing the message processing completion time T3 and the response message sending time T4. Based on the bidirectional time exchange protocol, the initial offset is calculated as: Δ = ((T2 - T1) - (T4 - T3)) / 2.

[0015] S220, dynamic adaptive time synchronization runtime stage; Specifically, the most recent 10 Δ values are saved in the sliding window, and the weighted least squares method is used to fit the linear model: ,in is the frequency drift rate, is the initial offset, is the random error term. The least squares method determines the parameters by minimizing the residual sum of squares. The calculation formula is as follows: , , in, , , represents the number of Δ, Represents time, for and The covariance of for The system monitors the drift rate within the sliding window in real time. If the drift rate exceeds a dynamic threshold, it triggers a local clock rollback or acceleration. It also periodically (every 10 seconds) aligns the system clock using the NTP protocol to compensate for large drifts (>1ms). Within each synchronization period, the PTP hardware clock adjustment interface fine-tunes the clock frequency.

[0016] The receiving network card parses the ECN field (2 bits) in the IPv4 / IPv6 header to identify congestion indications. If the ECN field is 11, it means that the switch buffer exceeds the threshold. The percentage of packets with an ECN field of 11 per unit time needs to be counted. , and mapped into 4 levels , the calculation formula is as follows: , Dynamically scale the time advance step size based on the congestion level , the calculation formula is as follows: .

[0017] S300, sorting events by logical timestamp to generate DAG topology and preload high-frequency events; Specifically, a logical timestamp directed acyclic graph (DAG) is constructed based on event causality, and an event execution sequence is generated through a topological sorting algorithm. Historical simulation data is analyzed to identify high-frequency event types and preload them into the node local cache. S400, cross-node RDMA transmission event data, the master node sorts and schedules according to logical timestamps, and realizes cross-platform distributed parallel simulation and consistency verification.

[0018] Specifically, the slave node writes local event data into the master node's memory pool through RDMA Write operations, along with a logical timestamp and dependency label. The master node performs multi-way sorting on the event queue based on the timestamp to generate a globally ordered execution sequence. When a causal conflict is detected, it rolls back to the last conflict-free checkpoint and re-executes. At the end of the simulation cycle, a hash digest is calculated for the result data of each node, and any inconsistencies in the comparison trigger replay verification of the backup queue.

[0019] In summary, the present embodiment first constructs a heterogeneous distributed network based on the Enhanced Network Architecture Protocol (RoCEv2) (E-RoCE). The RoCEv2 protocol stack is enabled on the nodes, UDP port 4791 is bound, and IP routing rules are configured to support IPv4 / IPv6 multipath forwarding. Parameters are then configured, PFC flow control priority is configured on the switch, the highest bandwidth queue is allocated for simulation data, the ECN explicit congestion marking threshold is set, the GPU Direct RDMA driver is deployed, and the physical address mapping between the GPU memory and the network card DMA engine is established. Dynamic adaptive time synchronization is then performed. A global logical time base is deployed on the master node. The master node broadcasts Sync messages, and slave nodes monitor data packets in the data link. Initial offsets are calculated to calibrate the slave node clocks. A sliding window is enabled to continuously monitor offsets, dynamically detecting clock offsets and triggering adjustments. If the offset rate exceeds a dynamic threshold, local clocks are rewound or accelerated. The time step is optimized based on ECN marking, and the time advance step is dynamically scaled based on the congestion level to dynamically compensate for clock offsets on heterogeneous platforms. Finally, the simulation results from each distributed node are transmitted via RDMA, and the master node schedules them according to logical timestamps and verifies their consistency. That is Figure 2 As shown, the system uses the Enhanced RoCEv2 (E-RoCE) network protocol, which supports IPv4 / IPv6 routing and is compatible with standard Ethernet switches. GPU Direct RDMA technology enables direct DMA connection between GPU memory and the network card, allowing the network card to directly handle RDMA semantics (Send / Write), bypassing the kernel protocol stack. This approach combines RDMA's advantages of reducing CPU utilization and conserving resources with a dynamic, adaptive time synchronization protocol to ensure synchronization of distributed parallel simulations, improving the system's responsiveness and processing capabilities for large-scale simulations.

[0020] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A cross-platform distributed synchronous simulation method using RoCEv2, characterized in that: The following steps are involved: Configure switch and node parameters to build a heterogeneous distributed network based on the Enhanced RoCEv2 (E-RoCE) protocol. Deploy a global logical time base on the master node, calibrate the slave node clocks, enable a sliding window to continuously monitor offsets, dynamically detect clock offsets and trigger adjustments, optimize the time step size using explicit congestion notification (ECN) marking, and dynamically compensate for clock offsets on heterogeneous platforms. Sort events by logical timestamp to generate a directed acyclic graph (DAG) topology and preload high-frequency events; Cross-node remote direct memory access (RDMA) transmits event data, and the master node sorts and schedules according to logical timestamps to achieve cross-platform distributed parallel simulation and consistency verification.

2. The cross-platform distributed synchronous simulation method using RoCEv2 according to claim 1, characterized in that: The step of configuring switch and node parameter information to build a heterogeneous distributed network of the RoCEv2 enhanced network protocol E-RoCE specifically includes: Configure priority-based flow control (PFC) on the switch, assign the highest bandwidth queue to the simulation data, and set the ECN explicit congestion marking threshold. Enable the RoCEv2 protocol stack on the node, bind the User Datagram Protocol UDP port 4791, and configure IP routing rules to support IPv4 / IPv6 multipath forwarding; Deploy the GPU Direct RDMA driver to establish physical address mapping between the graphics processing unit (GPU) memory and the network card's direct memory access (DMA) engine, enabling zero-copy data transfer. Configure the network card hardware to offload RDMA technology, which transfers tasks from the computer's processor to the network card, bypassing the kernel protocol stack and directly reading and writing user-mode memory; Configure a multipath load balancing policy based on the source port hashing algorithm.

3. The cross-platform distributed synchronous simulation method using RoCEv2 according to claim 2, characterized in that: The steps of deploying a global logical time base on the master node, calibrating the slave node clocks, starting a sliding window to continuously monitor offsets, dynamically detecting clock offsets to trigger adjustments, optimizing the time step in combination with ECN marking, and dynamically compensating for clock offsets on heterogeneous platforms specifically include: The master node periodically broadcasts the synchronization Sync message, and the slave node records the local reception time T2 and feeds back the delayed response Delay_Resp message; The initial offset is calculated using the clock deviation formula, and the drift trend is predicted by fitting historical data within the sliding window using the least squares method. Dynamically integrates Network Time Protocol (NTP) coarse-grained calibration and high-precision time synchronization (PTP) fine-grained compensation, and sets the monitoring window size to filter instantaneous jitter. Real-time monitoring of the clock deviation rate within the sliding window. If the deviation exceeds the dynamic threshold, the local clock is triggered to dial back or accelerate. Parse the ECN markings of network packets and proportionally reduce the time advance step when the congestion level reaches a preset level.

4. The cross-platform distributed synchronous simulation method using RoCEv2 according to claim 3, characterized in that: The step of generating a DAG topology by sorting events by logical timestamps and preloading high-frequency events specifically includes: Construct a logical timestamp directed acyclic graph (DAG) based on event causality, and generate event execution sequences using a topological sorting algorithm; Analyze historical simulation data, identify high-frequency event types, and preload them into the node local cache.

5. The cross-platform distributed synchronous simulation method using RoCEv2 according to claim 4, characterized in that: The cross-node RDMA transmission event data, the master node sorting and scheduling according to the logical timestamp, and implementing the cross-platform distributed parallel simulation and consistency verification step specifically include: The slave node writes local event data to the master node's memory pool through RDMA Write operations, along with a logical timestamp and dependency tag. The master node performs multi-way sorting of the event queue based on timestamps to generate a globally ordered execution sequence; When a causal conflict is detected, the process rolls back to the last conflict-free checkpoint and re-executes. At the end of the simulation cycle, a hash summary is calculated for the result data of each node. If there is any inconsistency, the backup queue replay verification is triggered.

Images