A soft-roce-based virtual network topology simulation method

Abstract

The application provides a kind of based on Soft-RoCE virtual network topology simulation method, belong to network topology simulation technical field, including the following steps: S1 creates isolated virtual node step;S2 configures virtual network link attribute step, this step uses tcnetem tool to dynamically configure link parameters on veth device, parameters include delay, packet loss rate and bandwidth limit, parameter configuration supports real-time dynamic adjustment, to meet the simulation needs of different topology scenarios, in view of the problems that the topology flexibility of traditional method is low and multi-tenant isolation has defects, the application realizes the isolation of GID, IP and protocol stack by flexibly configuring link attributes and based on network namespace, realizes the effect that traditional method cannot realize.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network topology simulation, and particularly relates to a virtual network topology simulation method based on Soft-RoCE. BACKGROUND

[0002] (1) RDMA technology development background

[0003] Remote Direct Memory Access (RDMA) technology was first applied to InfiniBand network architecture (InfiniBand Architecture Specification, version 1.4), which achieved microsecond-level delay and extremely low CPU overhead by bypassing the operating system kernel. With the growing demand for high-performance networks in data centers, RDMA over Converged Ethernet (RoCE) has become the mainstream solution:

[0004] RoCEv1 (RFC5041): based on Ethernet link layer (L2), requires lossless network support.

[0005] RoCEv2 (RFC7170): based on UDP / IP (L3), introduces traffic classification (DCSP / ECN).

[0006] iWARP (RFC5040-5044): based on TCP / IP, better compatibility but poorer performance.

[0007] (2) Technical positioning of Soft-RoCE

[0008] Soft-RoCE (also known as RXE) is a RoCEv2 software protocol stack implemented by the Linux kernel, and its core value lies in:

[0009] Hardware independence: implement RDMA functions on standard Ethernet cards (in contrast to hardware RoCE, which requires support for NICs).

[0010] Development and testing scenarios: provide a low-cost verification environment for RDMA applications.

[0011] The existing virtual network topology method has the following disadvantages:

[0012] RDMA virtualization deficiency: traditional RoCE relies on hardware switches and cannot simulate multi-node RDMA network topology at the software level.

[0013] Lack of topology flexibility: existing Soft-RoCE implementations only support a single physical network and lack dynamic configuration capabilities for virtual link properties (delay / bandwidth).

[0014] Multi-tenant isolation defect: the standard Soft-RoCE does not realize resource isolation at the network namespace level, resulting in GID conflicts in multi-tenant scenarios. SUMMARY

[0015] The purpose of the present application is to realize the isolation of GID, IP and protocol stack based on network namespace, solve the resource conflict problem in multi-tenant scenarios, and overcome the problems in the above background art.

[0016] Based on the above technical ideas, the technical scheme adopted by the present application is:

[0017] A virtual network topology simulation method based on Soft-RoCE, comprising the following steps:

[0018] S1, a step of creating an isolated virtual node, which uses Linux network namespace to create an independent isolated environment for each virtual node; an independent Soft-RoCE instance (based on an improved rxe kernel module) is deployed in each namespace, which supports namespace awareness capability; interconnection between nodes is realized through a virtual Ethernet device (vethpair);

[0019] S2, a step of configuring virtual network link attributes, which uses the tcnetem tool to dynamically configure link parameters on the veth device, including delay, packet loss rate and bandwidth limitation, and the parameter configuration supports real-time dynamic adjustment to meet the simulation needs of different topology scenarios;

[0020] S3, a step of dynamically generating and managing virtual topology, which defines the following contents through a YAML / JSON format topology configuration file:

[0021] The connection relationship between nodes;

[0022] The QoS parameters of the link;

[0023] The GID allocation strategy of each node;

[0024] S4, a step of improving the rxe kernel module to support multiple namespaces, which replaces the global data structure rxe of the original rxe module with a pernet_operations structure body based on namespace, realizes resource isolation, and realizes independent GID to IP address mapping under multiple namespaces through the net_generic method, avoiding GID conflicts in multi-tenant scenarios;

[0025] S5, a step of system components and their functions, which includes a topology management engine link, a namespace manager link, a performance optimization and precautions link, and a Soft-RoCE enhancement module link.

[0026] In furtherance of the above technical solution, the S1 step of creating isolated virtual nodes further comprises a step of deploying virtual network devices, a step of configuring IP addresses and activating devices, a step of configuring routing rules, and a step of initializing Soft-RoCE instances. The step of deploying virtual network devices comprises establishing logical links between namespaces to simulate the connection of a physical network. The step of configuring IP addresses and activating devices comprises assigning IP addresses to virtual nodes and activating network interfaces to support communication. The step of configuring routing rules comprises configuring routing tables if a virtual node needs to access external networks or other subnets. The step of initializing Soft-RoCE instances comprises starting independent RDMA protocol stacks in each namespace.

[0027] In furtherance of the above technical solution, the S2 step of configuring virtual network link attributes further comprises a step of locating target virtual devices, a step of configuring link delays, a step of configuring packet loss rates, a step of configuring bandwidth limits, and a step of configuring traffic shaping. The step of locating target virtual devices comprises determining virtual Ethernet devices that need to have link attributes configured. The step of configuring link delays comprises simulating transmission delays of data packets on the link. The step of configuring packet loss rates comprises simulating data packet loss in the link. The step of configuring bandwidth limits comprises limiting the maximum transmission rate of the link. The step of configuring traffic shaping comprises simulating complex network behaviors such as data packet reordering, duplication, or corruption.

[0028] In furtherance of the above technical solution, the S3 step of dynamically generating and managing virtual topology comprises a step of defining topology configuration files, a step of parsing configuration files and generating topology, a step of initializing RDMA protocol stacks, a step of dynamically adjusting topology, a step of topology management and monitoring, and a step of verifying and testing use cases. The step of defining topology configuration files comprises describing the logical topology and link parameters of the virtual RDMA network through structured files to enable rapid deployment and dynamic adjustment. The step of parsing configuration files and generating topology comprises converting configuration files into deployment instructions for actual virtual nodes and links. The step of initializing RDMA protocol stacks comprises starting Soft-RoCE instances that support multiple namespaces in each virtual node.

[0029] In furtherance of the above technical solution, the step of dynamically adjusting topology comprises modifying the topology structure or link parameters during the operation of the virtual network. The step of topology management and monitoring comprises maintaining the state of the virtual network and providing visual monitoring. The step of verifying and testing use cases comprises ensuring that the virtual topology works as expected and supports target scenarios.

[0030] Further to the above technical solution, the improved rxe kernel module supports a multi-namespace step, which includes a global limitation analysis step of the original module, a network namespace reconstruction step of the module, a cross-namespace communication capability enhancement step, and a verification and test step, wherein the global limitation analysis step of the original module includes using a global data structure (such as rxe_net) to manage RDMA resources, which results in the inability to isolate resources of different network namespaces; the network namespace reconstruction step of the module includes transforming global resources (such as a device list and a GID table) into independent resource pools based on network namespaces; and the cross-namespace communication capability enhancement step includes modifying CM event processing logic to implement cross-namespace routing queries.

[0031] Further to the above technical solution, the verification and test step includes testing of multi-namespace device isolation, testing of GID isolation, and testing of cross-namespace RDMA communication; and the performance optimization and matters needing attention step includes concurrency control, use of read-write locks (rwlock) to protect resource access within a namespace, memory management to ensure that all associated resources are released when a namespace is destroyed, and kernel compatibility to adapt to pernet_operations APIs of different kernel versions.

[0032] Further to the above technical solution, the S5 system component and its function step include a topology management engine step that is responsible for parsing a configuration file, generating node connection, link parameter, and GID allocation instructions, and supporting dynamic adjustment of topology; a namespace manager step that includes creating and maintaining a mapping relationship table of virtual nodes and network namespaces, and calling a Linux kernel interface to implement an isolated environment of a namespace; and a Soft-RoCE enhancement module step that includes extending support for multiple namespaces on a standard basis and providing cross-namespace communication capabilities.

[0033] Further to the above technical solution, the step also includes a running and verification step, which includes a virtual RDMA network startup step and a dynamic adjustment example step, the virtual RDMA network startup step includes executing a topology management engine to generate virtual nodes and links according to a configuration file, and the dynamic adjustment example step includes adjusting a bandwidth from 10 Gbps to 5 Gbps from node 1 to node 2 in running.

[0034] Further to the above technical solution, the step also includes a core process, which includes 1. isolated environment construction, 2. virtual link configuration, 3. topology definition and deployment, 4. RDMA protocol stack enhancement, and 5. dynamic operation and maintenance management.

[0035] Compared with the prior art, the present application has the following advantages:

[0036] 1. Simulate complex topology on a single machine, simulate large-scale RDMA network (support >256 nodes) in a single physical / virtual machine without relying on hardware switches.

[0037] 2. Flexible configuration of link attributes, dynamically set the delay, bandwidth and other parameters of virtual links through tcnetem, support multiple network scenario simulation.

[0038] 3. Multi-tenant resource isolation, GID, IP and protocol stack isolation based on network namespace, solve the resource conflict problem in multi-tenant scenario.

[0039] 4. Low-cost development and testing, no need for special hardware (such as RoCE card or InfiniBand switch), reduce the cost of RDMA application verification. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0041] Figure 1 The flowchart of the overall architecture of the virtual network topology simulation method based on Soft-RoCE of the present application;

[0042] Figure 2 The flowchart of the RDMA of the virtual network topology simulation method based on Soft-RoCE of the present application;

[0043] Figure 3 The step diagram of the overall architecture of the virtual network topology simulation method based on Soft-RoCE of the present application;

[0044] Figure 4 The flowchart of creating topology of the virtual network topology simulation method based on Soft-RoCE of the present application;

[0045] Figure 5 The flowchart of the transceiving packet process of the virtual network topology simulation method based on Soft-RoCE of the present application;

[0046] Figure 6 The step diagram of the transceiving packet process of the virtual network topology simulation method based on Soft-RoCE of the present application;

[0047] Figure 7A flowchart for adjusting a rate in a Soft-RoCE-based virtual network topology simulation method of the present application. DETAILED DESCRIPTION

[0048] The application will be further described below with reference to the accompanying drawings. Figures 1-7 The application will be further described below with reference to the accompanying drawings.

[0049] Embodiment 1: This embodiment provides a Soft-RoCE-based virtual network topology simulation method, as shown in the following steps: Figures 1-7

[0050] S1: Create an isolated virtual node step, which uses a Linux network namespace to create an independent isolated environment for each virtual node; deploy an independent Soft-RoCE instance (based on an improved rxe kernel module) in each namespace, which supports namespace awareness capability; and realize interconnection between nodes through a virtual Ethernet device (veth pair);

[0051] S2: Configure virtual network link attributes step, which uses a tcnetem tool to dynamically configure link parameters on the veth device, including delay, packet loss rate and bandwidth limit, and the parameter configuration supports real-time dynamic adjustment to meet the simulation requirements of different topology scenarios;

[0052] S3: Dynamically generate and manage virtual topology step, which defines the following contents through a YAML / JSON format topology configuration file:

[0053] Connection relationship between nodes;

[0054] QoS parameters of the link;

[0055] GID allocation strategy of each node;

[0056] S4: Improve rxe kernel module to support multiple namespaces step, which replaces the global data structure rxe of the original rxe module with a pernet_operations structure body based on namespaces, realizes resource isolation, and realizes independent GID to IP address mapping under multiple namespaces through the net_generic() method, avoiding GID conflicts in multi-tenant scenarios;

[0057] S5: System components and their functions step, which includes a topology management engine link, a namespace manager link, a performance optimization and precautions link, and a Soft-RoCE enhancement module link.

[0058] ​The S1 creates isolated virtual nodes, which further includes deploying virtual network devices, configuring IP addresses and activating devices, configuring routing rules, and initializing Soft-RoCE instances. Deploying virtual network devices includes establishing logical links between namespaces to simulate the connections of a physical network. Configuring IP addresses and activating devices includes assigning IP addresses to virtual nodes and activating network interfaces to support communication. Configuring routing rules includes configuring routing tables if virtual nodes need to access external networks or other subnets. Initializing Soft-RoCE instances includes starting independent RDMA protocol stacks in each namespace.

[0059] The S2 configures virtual network link attributes, which further includes locating target virtual devices, configuring link delays, configuring packet loss rates, configuring bandwidth limits, and configuring traffic shaping. Locating target virtual devices includes determining virtual Ethernet devices that need to have link attributes configured. Configuring link delays includes simulating transmission delays of data packets on links. Configuring packet loss rates includes simulating data packet loss in links. Configuring bandwidth limits includes limiting the maximum transmission rate of links. Configuring traffic shaping includes simulating complex network behaviors such as data packet reordering, duplication, or corruption.

[0060] The S3 dynamically generates and manages virtual topologies, which includes defining topology configuration files, parsing configuration files and generating topologies, initializing RDMA protocol stacks, dynamically adjusting topologies, topology management and monitoring, and verifying and testing use cases. Defining topology configuration files includes describing the logical topology and link parameters of a virtual RDMA network through structured files to enable fast deployment and dynamic adjustment. Parsing configuration files and generating topologies includes converting configuration files into deployment instructions for actual virtual nodes and links. Initializing RDMA protocol stacks includes starting Soft-RoCE instances that support multiple namespaces in each virtual node.

[0061] The dynamically adjusting topologies includes modifying topology structures or link parameters during the operation of virtual networks. The topology management and monitoring includes maintaining the state of virtual networks and providing visual monitoring. The verifying and testing use cases includes ensuring that virtual topologies work as expected and support target scenarios.

[0062] The improved rxe kernel module supports a multi-namespace step, which includes a step of analyzing global limitations of original modules, a step of reconstructing module support for network namespaces, a step of enhancing cross-namespace communication capabilities, and a step of verification and testing, wherein the step of analyzing global limitations of original modules includes using a global data structure (such as rxe_net) to manage RDMA resources, which results in inability to isolate resources of different network namespaces; the step of reconstructing module support for network namespaces includes transforming global resources (such as a device list and a GID table) into independent resource pools based on network namespaces; and the step of enhancing cross-namespace communication capabilities includes modifying CM event processing logic to implement cross-namespace routing queries.

[0063] The step of verification and testing includes testing of multi-namespace device isolation, testing of GID isolation, and testing of cross-namespace RDMA communication; and the step of performance optimization and considerations includes concurrency control, use of read-write locks (rwlock) to protect resource access within namespaces, memory management, ensuring release of all associated resources when a namespace is destroyed, and kernel compatibility, adaptation of pernet_operations APIs of different kernel versions.

[0064] The S5 system component and a function step thereof, wherein the topology management engine includes a step of being responsible for parsing a configuration file, generating node connection, link parameter, and GID allocation instructions, and supporting dynamic adjustment of topology; the namespace manager includes a step of creating and maintaining a mapping relationship table of virtual nodes and network namespaces, and calling a Linux kernel interface to implement an isolated environment of a namespace; and the Soft-RoCE enhancement module includes a step of extending support for multiple namespaces on a standard basis, and providing communication capabilities across namespaces.

[0065] The step of running and verification further includes a step of starting a virtual RDMA network and a dynamic adjustment example, wherein the step of starting a virtual RDMA network includes executing a topology management engine to generate virtual nodes and links according to a configuration file, and the dynamic adjustment example includes adjusting a bandwidth from 10 Gbps to 5 Gbps from node 1 to node 2 in running.

[0066] The core flow further includes 1. Isolated environment construction → 2. Virtual link configuration → 3. Topology definition and deployment → 4. RDMA protocol stack enhancement → 5. Dynamic operation and maintenance management.

[0067] Embodiment 2: The embodiment provides a virtual network topology simulation method based on Soft-RoCE, as shown in Figures 1-7 The method further includes the following steps:

[0068] 1. Isolated virtual nodes are created through Linux network namespace, and each node deploys an independent Soft-RoCE instance, wherein:

[0069] Each instance is implemented by a modified rxe kernel module to realize namespace awareness.

[0070] Virtual Ethernet devices (veth pairs) are used for inter-node communication.

[0071] 2. Network characteristic parameters, including latency, packet loss rate, and bandwidth limit, are configured on the veth device using the tcnetem tool.

[0072] 3. A topology configuration file (in YAML / JSON format) is dynamically parsed to automatically generate a virtual RDMA network topology, wherein:

[0073] The configuration file defines the connection relationship between nodes and the link QoS parameters.

[0074] The topology management engine updates the rxe_net structure in real time to realize network reconstruction.

[0075] Embodiment 3: The embodiment provides a virtual network topology simulation method based on Soft-RoCE, as shown in Figures 1-7 The improved scheme of the rxe kernel module is also included:

[0076] The global data structure rxe_net is replaced by the pernet_operations structure based on network namespace.

[0077] GID to IP address mapping is realized under multiple namespaces through net_generic().

[0078] Embodiment 4: The embodiment provides a virtual network topology simulation method based on Soft-RoCE, as shown in Figures 1-7 The embodiment also includes:

[0079] Topology management engine: used to parse the topology configuration file in YAML / JSON format and generate an instruction set containing the following elements:

[0080] The veth pair connection relationship between nodes;

[0081] The tcnetem parameter configuration of each link;

[0082] The GID allocation strategy of the Soft-RoCE instance;

[0083] Namespace manager: creates an isolated environment through system calls of the topology engine, including:

[0084] assigning independent network namespace to each virtual node;

[0085] maintaining a mapping table of namespace and virtual node;

[0086] Soft-RoCE enhancement module: on the basis of standard rxe.ko, increase:

[0087] namespace-aware QP (queue pair) creation interface;

[0088] RDMACM event routing mechanism across namespaces.

[0089] The above is a further detailed description of the present application in combination with specific preferred embodiments, which facilitates the understanding and application of the present application by those skilled in the art, and cannot be regarded as a limitation of the specific implementation of the present application.

Claims

1. A virtual network topology simulation method based on Soft-RoCE, characterized in that, Includes the following steps: The S1 step creates isolated virtual nodes, which utilizes Linux network namespaces to create an independent isolated environment for each virtual node; deploys an independent Soft-RoCE instance in each namespace to enable namespace awareness; and achieves interconnection between nodes through virtual Ethernet devices. The S2 step involves configuring virtual network link attributes. This step uses the tcnetem tool to dynamically configure link parameters on the veth device. The parameters include latency, packet loss rate, and bandwidth limit. The parameter configuration supports real-time dynamic adjustment to meet the simulation requirements of different topology scenarios. S3 dynamically generates and manages virtual topology steps, which define the following content through a topology configuration file in YAML / JSON format: The connection relationships between nodes; QoS parameters of the link; GID allocation strategy for each node; S4 improves the rxe kernel module to support multi-namespace steps. This step replaces the original rxe module's global data structure rxe with a namespace-based pernet_operations structure to achieve resource isolation. It uses the net_generic method to implement independent GID to IP address mapping under multiple namespaces, avoiding GID conflicts in multi-tenant scenarios. S5 system components and their functional steps, which include the topology management engine step, the namespace manager step, the performance optimization and precautions step, and the Soft-RoCE enhancement module step; The S1 step of creating isolated virtual nodes further includes deploying virtual network devices, configuring IP addresses and activating devices, configuring routing rules, and initializing Soft-RoCE instances. Deploying virtual network devices includes establishing logical links between namespaces to simulate physical network connections; configuring IP addresses and activating devices includes assigning IP addresses to virtual nodes and activating network interfaces to support communication; configuring routing rules includes configuring routing tables if virtual nodes need to access external networks or other subnets; and initializing Soft-RoCE instances includes starting an independent RDMA protocol stack in each namespace.

2. The virtual network topology simulation method based on Soft-RoCE according to claim 1, characterized in that, The S2 step of configuring virtual network link attributes further includes locating the target virtual device, configuring link latency, configuring packet loss rate, configuring bandwidth limitation, and configuring traffic shaping. Locating the target virtual device involves identifying the virtual Ethernet device for which link attributes need to be configured; configuring link latency involves simulating data packet transmission delay on the link; configuring packet loss rate involves simulating data packet loss on the link; configuring bandwidth limitation involves limiting the maximum transmission rate of the link; and configuring traffic shaping involves simulating complex network behaviors such as data packet reordering, duplication, or corruption.

3. The virtual network topology simulation method based on Soft-RoCE according to claim 2, characterized in that, The S3 step of dynamically generating and managing virtual topology includes defining a topology configuration file, parsing the configuration file and generating the topology, initializing the RDMA protocol stack, dynamically adjusting the topology, managing and monitoring the topology, and verifying and testing cases. Specifically, defining the topology configuration file involves describing the logical topology and link parameters of the virtual RDMA network using a structured file, enabling rapid deployment and dynamic adjustment; parsing the configuration file and generating the topology involves converting the configuration file into deployment instructions for actual virtual nodes and links; and initializing the RDMA protocol stack involves starting a Soft-RoCE instance supporting multiple namespaces in each virtual node.

4. The virtual network topology simulation method based on Soft-RoCE according to claim 3, characterized in that, The dynamic topology adjustment step includes modifying the topology structure or link parameters during the operation of the virtual network; the topology management and monitoring step includes maintaining the virtual network status and providing visual monitoring; the verification and test case step includes ensuring that the virtual topology works as expected and supports the target scenario.

5. The virtual network topology simulation method based on Soft-RoCE according to claim 4, characterized in that, The improved Rxe kernel module supports a multi-namespace approach, which includes analyzing the global limitations of the original module, reconstructing the module to support network namespaces, enhancing cross-namespace communication capabilities, and verification and testing. The analysis of the global limitations of the original module includes using global data structures to manage RDMA resources, which prevents the isolation of resources in different network namespaces. The reconstruction of the module to support network namespaces includes transforming global resources into independent resource pools based on network namespaces. The enhancement of cross-namespace communication capabilities includes modifying the CM event handling logic to enable cross-namespace routing queries.

6. The virtual network topology simulation method based on Soft-RoCE according to claim 5, characterized in that, The verification and testing phase includes testing the isolation of devices in multiple namespaces, testing GID isolation, and testing cross-namespace RDMA communication; the performance optimization and precautions phase includes concurrency control, using read-write locks to protect resource access within a namespace, memory management to ensure that all associated resources are released when a namespace is destroyed, kernel compatibility, and adapting to pernet_operationsAPI for different kernel versions.

7. A virtual network topology simulation method based on Soft-RoCE according to claim 6, characterized in that, The S5 system components and their functional steps include the following steps: the topology management engine is responsible for parsing configuration files, generating node connections, link parameters, and GID allocation instructions, and supporting dynamic topology adjustments; the namespace manager is responsible for creating and maintaining a mapping table between virtual nodes and network namespaces, and calling the Linux kernel interface to implement a namespace isolation environment; the Soft-RoCE enhancement module is responsible for extending support for multiple namespaces on the basis of the standard, and providing cross-namespace communication capabilities.

8. The virtual network topology simulation method based on Soft-RoCE according to claim 7, characterized in that, It also includes operation and verification steps, which include starting the virtual RDMA network segment and dynamically adjusting the example segment. Starting the virtual RDMA network segment includes executing the topology management engine to generate virtual nodes and links according to the configuration file. The dynamically adjusting example segment includes adjusting the bandwidth from 10Gbps to 5Gbps from node 1 to node 2 during operation.

9. A virtual network topology simulation method based on Soft-RoCE according to claim 8, characterized in that, It also includes core processes, which include:

1. Isolation environment construction → 2. Virtual link configuration → 3. Topology definition and deployment → 4. RDMA protocol stack enhancement → 5. Dynamic operation and maintenance management.

Images