Soft-RoCE-based virtual network topology simulation method

Abstract

The invention provides a soft-RoCE-based virtual network topology simulation method, which belongs to the technical field of network topology simulation, and comprises the following steps: S1, creating isolated virtual nodes; s2, a virtual network link attribute configuration step: link parameters are dynamically configured on a veth device by using a tcnetem tool, the parameters comprise time delay, packet loss rate and bandwidth limitation, parameter configuration supports real-time dynamic adjustment so as to meet simulation requirements of different topology scenes, and aiming at the problems that a traditional method is low in topology flexibility and has defects in multi-tenant isolation, the virtual network link attribute configuration method is provided for solving the problems that the traditional method is low in topology flexibility and has defects in multi-tenant isolation. According to the method, the GID, the IP and the protocol stack are isolated by flexibly configuring the link attributes and based on the network namespace, and the effect which cannot be achieved by a traditional method is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of network topology simulation, and in particular to a virtual network topology simulation method based on Soft-RoCE. Background Art

[0002] (1) RDMA technology development background Remote Direct Memory Access (RDMA) technology was first used in the InfiniBand network architecture (InfiniBand Architecture Specification, version 1.4). By bypassing the operating system kernel, it achieves microsecond latency and extremely low CPU overhead. As data centers grow in demand for high-performance networks, RDMA over Converged Ethernet (RoCE) has become a mainstream solution. RoCEv1 (RFC5041): Based on the Ethernet link layer (L2), requiring lossless network support.

[0003] RoCEv2 (RFC7170): Based on UDP / IP (L3), it introduces traffic classification (DCSP / ECN).

[0004] iWARP (RFC5040-5044): Based on TCP / IP, it has better compatibility but poorer performance.

[0005] (2) Technical positioning of Soft-RoCE Soft-RoCE (also known as RXE) is the RoCEv2 software protocol stack implemented in the Linux kernel. Its core value lies in: Hardware independence: RDMA functionality is implemented on standard Ethernet cards (compared to hardware RoCE which requires a supported NIC).

[0006] Development and testing scenarios: Provides a low-cost verification environment for RDMA applications.

[0007] Existing virtual network topology methods have the following shortcomings: Insufficient RDMA virtualization: Traditional RoCE relies on hardware switches and cannot simulate multi-node RDMA network topology at the software level.

[0008] Lack of topology flexibility: Existing Soft-RoCE implementations only support a single physical network and lack the ability to dynamically configure virtual link properties (latency / bandwidth).

[0009] Multi-tenant isolation flaw: Standard Soft-RoCE does not implement resource isolation at the network namespace level, resulting in GID conflicts in multi-tenant scenarios. Summary of the Invention

[0010] The purpose of the present invention is to achieve isolation of GID, IP and protocol stack based on network namespace, solve the resource conflict problem in multi-tenant scenario, and thus overcome the problems in the above-mentioned background technology.

[0011] Based on the above technical ideas, the technical solution adopted by the present invention is: A virtual network topology simulation method based on Soft-RoCE includes the following steps: S1 creates isolated virtual nodes. This step uses Linux network namespaces to create an independent, isolated environment for each virtual node. A separate Soft-RoCE instance (based on an improved rxe kernel module) is deployed in each namespace to enable namespace awareness. Inter-node interconnection is achieved through virtual Ethernet devices (vethpairs). S2 configures the virtual network link properties. This step uses the tcnetem tool to dynamically configure link parameters on the veth device. Parameters include latency, packet loss rate, and bandwidth limit. Parameter configuration supports real-time dynamic adjustment to meet the simulation requirements of different topology scenarios. S3 dynamically generates and manages virtual topology steps, which define the following through a topology configuration file in YAML / JSON format: The connection relationship between nodes; QoS parameters of the link; GID allocation strategy for each node; S4 improves the rxe kernel module to support multiple namespaces. This step replaces the original rxe module's global data structure rxe with a namespace-based pernet_operations structure to achieve resource isolation. It uses the net_generic method to implement independent GID-to-IP address mapping in multiple namespaces, avoiding GID conflicts in multi-tenant scenarios. S5 system components and their functional steps, which include the topology management engine, namespace manager, performance optimization and precautions, and Soft-RoCE enhancement module.

[0012] Further limiting the above technical solution, the step of creating an isolated virtual node in S1 also includes deploying a virtual network device, configuring an IP address and activating a device, configuring routing rules, and initializing a Soft-RoCE instance. The step of deploying a virtual network device includes establishing a logical link between namespaces to simulate a physical network connection; configuring an IP address and activating a device includes assigning an IP address to the virtual node and activating a network interface to support communication; configuring routing rules includes configuring a routing table if the virtual node needs to access an external network or other subnet; and initializing a Soft-RoCE instance includes starting an independent RDMA protocol stack in each namespace.

[0013] A further limitation of the above technical solution is that the S2 step of configuring the virtual network link attributes also includes a target virtual device positioning link, a link delay configuration link, a packet loss rate configuration link, a bandwidth limitation configuration link, and a traffic shaping configuration link, wherein the target virtual device positioning link includes determining the virtual Ethernet device for which the link attributes need to be configured; the link delay configuration link includes simulating the transmission delay of the data packet on the link; the packet loss rate configuration link includes simulating the data packet loss in the link; the bandwidth limitation configuration link includes limiting the maximum transmission rate of the link; and the traffic shaping configuration link includes simulating complex network behaviors, such as data packet reordering, duplication, or damage.

[0014] Further limiting the above technical solution, the S3 step of dynamically generating and managing the virtual topology includes defining a topology configuration file, parsing the configuration file and generating the topology, initializing the RDMA protocol stack, dynamically adjusting the topology, managing and monitoring the topology, and verifying and testing cases. The topology configuration file definition step includes describing the logical topology and link parameters of the virtual RDMA network through a structured file to achieve rapid deployment and dynamic adjustment; the configuration file parsing and generating the topology step includes converting the configuration file into deployment instructions for actual virtual nodes and links; and the RDMA protocol stack initialization step includes starting a Soft-RoCE instance that supports multiple namespaces in each virtual node.

[0015] To further limit the above technical solution, the dynamic topology adjustment link includes modifying the topology structure or link parameters during the operation of the virtual network; the topology management and monitoring link includes maintaining the virtual network status and providing visual monitoring; the verification and test case link includes ensuring that the virtual topology works as expected and supports the target scenario.

[0016] Further limiting the above technical solution, the step of improving the rxe kernel module to support multiple namespaces includes analyzing the global restrictions of the original module, reconstructing the module to support network namespaces, enhancing cross-namespace communication capabilities, and verification and testing. Among them, the analysis of the global restrictions of the original module includes using global data structures (such as rxe_net) to manage RDMA resources, resulting in the inability to isolate resources in different network namespaces; reconstructing the module to support network namespaces includes transforming global resources (such as device lists and GID tables) into independent resource pools based on network namespaces; and enhancing cross-namespace communication capabilities includes modifying the CM event processing logic to achieve cross-namespace routing queries.

[0017] To further limit the above technical solution, the verification and testing phase includes testing multi-namespace device isolation, testing GID isolation, and testing cross-namespace RDMA communication; the performance optimization and precautions phase includes concurrency control, using read-write locks (rwlocks) to protect resource access within the namespace, memory management to ensure that all associated resources are released when the namespace is destroyed, kernel compatibility, and adapting the pernet_operations API to different kernel versions.

[0018] To further define the above technical solution, the S5 system components and their functional steps, the topology management engine link in this step includes being responsible for parsing the configuration file, generating node connections, link parameters and GID allocation instructions, and supporting dynamic adjustment of the topology; the namespace manager link includes creating and maintaining a mapping relationship table between virtual nodes and network namespaces, calling the Linux kernel interface to implement an isolated environment for the namespace; the Soft-RoCE enhancement module link includes expanding support for multiple namespaces on the basis of the standard and providing cross-namespace communication capabilities.

[0019] A further limitation of the above technical solution also includes an operation and verification step, which includes starting a virtual RDMA network link and a dynamic adjustment example link. Starting the virtual RDMA network link includes executing a topology management engine and generating virtual nodes and links according to a configuration file. The dynamic adjustment example link includes adjusting the bandwidth from node 1 to node 2 from 10Gbps to 5Gbps during operation.

[0020] Further limitations on the above technical solution also include core processes, which include 1. Isolation environment construction → 2. Virtual link configuration → 3. Topology definition and deployment → 4. RDMA protocol stack enhancement → 5. Dynamic operation and maintenance management.

[0021] Compared with the prior art, the present invention has the following beneficial effects: 1. Single-machine simulation of complex topologies, simulating large-scale RDMA networks (supporting >256 nodes) in a single physical machine / virtual machine without relying on hardware switches.

[0022] 2. Flexible configuration of link attributes. Dynamically set virtual link parameters such as latency and bandwidth through tcnetem to support simulation of various network scenarios.

[0023] 3. Multi-tenant resource isolation: Isolation of GID, IP, and protocol stack based on network namespace to resolve resource conflicts in multi-tenant scenarios.

[0024] 4. Low-cost development and testing, without the need for dedicated hardware (such as RoCE network cards or InfiniBand switches), reducing the cost of RDMA application verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of the overall architecture of a virtual network topology simulation method based on Soft-RoCE in the present invention; Figure 2 A schematic diagram of the RDMA process in a virtual network topology simulation method based on Soft-RoCE of the present invention; Figure 3 A schematic diagram of the overall architecture steps in a virtual network topology simulation method based on Soft-RoCE of the present invention; Figure 4 A schematic diagram of a process for creating a topology in a virtual network topology simulation method based on Soft-RoCE according to the present invention; Figure 5 A schematic diagram of the packet sending and receiving process in a virtual network topology simulation method based on Soft-RoCE of the present invention; Figure 6 A schematic diagram of the steps of the packet sending and receiving process in a virtual network topology simulation method based on Soft-RoCE of the present invention; Figure 7 The present invention is a flow chart of rate adjustment in a virtual network topology simulation method based on Soft-RoCE. DETAILED DESCRIPTION

[0027] The following is combined with Figure 1-Figure 7The present invention is described in further detail.

[0028] Example 1: This example provides a virtual network topology simulation method based on Soft-RoCE. Figure 1-Figure 7 As shown, the following steps are included: S1 creates isolated virtual nodes. This step uses Linux network namespaces to create an independent, isolated environment for each virtual node. A separate Soft-RoCE instance (based on an improved rxe kernel module) is deployed in each namespace to enable namespace awareness. Inter-node interconnection is achieved through virtual Ethernet devices (vethpairs). S2 configures the virtual network link properties. This step uses the tcnetem tool to dynamically configure link parameters on the veth device. Parameters include latency, packet loss rate, and bandwidth limit. Parameter configuration supports real-time dynamic adjustment to meet the simulation requirements of different topology scenarios. S3 dynamically generates and manages virtual topology steps, which define the following through a topology configuration file in YAML / JSON format: The connection relationship between nodes; QoS parameters of the link; GID allocation strategy for each node; S4 improves the rxe kernel module to support multiple namespaces. This step replaces the original rxe module's global data structure rxe with a namespace-based pernet_operations structure to achieve resource isolation. The net_generic() method is used to implement independent GID-to-IP address mapping in multiple namespaces to avoid GID conflicts in multi-tenant scenarios. S5 system components and their functional steps, which include the topology management engine, namespace manager, performance optimization and precautions, and Soft-RoCE enhancement module.

[0029] The S1 step of creating an isolated virtual node also includes deploying a virtual network device, configuring an IP address and activating the device, configuring routing rules, and initializing a Soft-RoCE instance. The deploying virtual network device step includes establishing a logical link between namespaces to simulate a physical network connection; configuring an IP address and activating the device step includes assigning an IP address to the virtual node and activating the network interface to support communication; configuring routing rules includes configuring a routing table if the virtual node needs to access an external network or other subnet; and initializing the Soft-RoCE instance step includes starting an independent RDMA protocol stack in each namespace.

[0030] The S2 step of configuring the virtual network link attributes also includes a target virtual device positioning link, a link delay configuration link, a packet loss rate configuration link, a bandwidth limitation configuration link, and a traffic shaping configuration link. The target virtual device positioning link includes determining the virtual Ethernet device for which link attributes need to be configured; the link delay configuration link includes simulating the transmission delay of data packets on the link; the packet loss rate configuration link includes simulating data packet loss in the link; the bandwidth limitation configuration link includes limiting the maximum transmission rate of the link; and the traffic shaping configuration link includes simulating complex network behaviors, such as data packet reordering, duplication, or damage.

[0031] The S3 dynamically generates and manages the virtual topology, which includes defining a topology configuration file, parsing the configuration file and generating the topology, initializing the RDMA protocol stack, dynamically adjusting the topology, managing and monitoring the topology, and verifying and testing cases. The topology configuration file definition step includes describing the logical topology and link parameters of the virtual RDMA network through a structured file to achieve rapid deployment and dynamic adjustment; parsing the configuration file and generating the topology step includes converting the configuration file into deployment instructions for actual virtual nodes and links; and initializing the RDMA protocol stack step includes starting a Soft-RoCE instance that supports multiple namespaces in each virtual node.

[0032] The dynamic topology adjustment link includes modifying the topology structure or link parameters during the operation of the virtual network; the topology management and monitoring link includes maintaining the virtual network status and providing visual monitoring; the verification and test case link includes ensuring that the virtual topology works as expected and supports the target scenario.

[0033] The steps for improving the rxe kernel module to support multiple namespaces include analyzing the global limitations of the original module, reconstructing the module to support network namespaces, enhancing cross-namespace communication capabilities, and verification and testing. The analysis of the global limitations of the original module includes using global data structures (such as rxe_net) to manage RDMA resources, resulting in the inability to isolate resources in different network namespaces. Reconstructing the module to support network namespaces includes transforming global resources (such as device lists and GID tables) into independent resource pools based on network namespaces. Enhancing cross-namespace communication capabilities includes modifying the CM event processing logic to implement cross-namespace routing queries.

[0034] The verification and testing phase includes testing multi-namespace device isolation, testing GID isolation, and testing cross-namespace RDMA communication. The performance optimization and precautions phase includes concurrency control, using read-write locks (rwlocks) to protect resource access within the namespace, memory management, ensuring that all associated resources are released when the namespace is destroyed, kernel compatibility, and adapting the pernet_operations API to different kernel versions.

[0035] The S5 system components and their functional steps include the topology management engine link in this step, which is responsible for parsing the configuration file, generating node connections, link parameters and GID allocation instructions, and supporting dynamic topology adjustment; the namespace manager link includes creating and maintaining a mapping relationship table between virtual nodes and network namespaces, and calling the Linux kernel interface to implement an isolated namespace environment; the Soft-RoCE enhancement module link includes expanding support for multiple namespaces on the basis of the standard and providing cross-namespace communication capabilities.

[0036] It also includes an operation and verification step, which includes starting a virtual RDMA network link and a dynamic adjustment example link. Starting the virtual RDMA network link includes executing a topology management engine and generating virtual nodes and links according to a configuration file. The dynamic adjustment example link includes adjusting the bandwidth from node 1 to node 2 from 10Gbps to 5Gbps during operation.

[0037] It also includes core processes, including 1. Isolation environment construction → 2. Virtual link configuration → 3. Topology definition and deployment → 4. RDMA protocol stack enhancement → 5. Dynamic operation and maintenance management.

[0038] Example 2: This example provides a virtual network topology simulation method based on Soft-RoCE. Figure 1-Figure 7 As shown, the following steps are also included: 1. Create isolated virtual nodes through the Linux network namespace, and deploy an independent Soft-RoCE instance on each node. -Each instance achieves namespace awareness through a modified rxe kernel module; -Inter-node communication uses virtual Ethernet devices (vethpair) for interconnection; 2. Use the tcnetem tool to configure network characteristic parameters on the veth device, including latency, packet loss rate, and bandwidth limit; 3. Dynamically parse the topology configuration file (YAML / JSON format) and automatically generate the virtual RDMA network topology, including: -The configuration file defines the connection relationship between nodes and link QoS parameters; -The topology management engine updates the rxe_net structure in real time to achieve network reconstruction.

[0039] Example 3: This embodiment provides a virtual network topology simulation method based on Soft-RoCE, such as Figure 1-Figure 7 As shown, the improvement scheme of the rxe kernel module is also included: Replace the global data structure rxe_net with the pernet_operations structure based on the network namespace; Use net_generic() to achieve independent mapping of GID to IP address in multiple namespaces.

[0040] Example 4: This embodiment provides a virtual network topology simulation method based on Soft-RoCE, such as Figure 1-Figure 7 As shown, it also includes: Topology management engine: used to parse topology configuration files in YAML / JSON format and generate instruction sets containing the following elements: Vethpair connection relationship between nodes; tcnetem parameter configuration for each link; GID allocation policy for Soft-RoCE instances; Namespace Manager: Creates an isolated environment through topology engine system calls, including: Allocate an independent network namespace to each virtual node; Maintain the mapping relationship table between namespace and virtual node; Soft-RoCE enhancement module: Adds the following based on the standard rxe.ko: Namespace-aware QP (queue pair) creation interface; RDMACM event routing mechanism across namespaces.

[0041] The above contents are further detailed descriptions of the present invention in conjunction with specific preferred embodiments, so as to facilitate those skilled in the art to understand and apply the present invention. It should not be considered that the specific implementation of the present invention is limited to these descriptions.

Claims

1. A virtual network topology simulation method based on Soft-RoCE, characterized in that: The following steps are involved: S1 creates isolated virtual nodes. This step uses Linux network namespaces to create an independent, isolated environment for each virtual node. A separate Soft-RoCE instance is deployed in each namespace to enable namespace awareness. Inter-node interconnection is achieved through virtual Ethernet devices. S2 configures the virtual network link properties. This step uses the tcnetem tool to dynamically configure link parameters on the veth device. Parameters include latency, packet loss rate, and bandwidth limit. Parameter configuration supports real-time dynamic adjustment to meet the simulation requirements of different topology scenarios. S3 dynamically generates and manages virtual topology steps, which define the following through a topology configuration file in YAML / JSON format: The connection relationship between nodes; QoS parameters of the link; GID allocation strategy for each node; S4 improves the rxe kernel module to support multiple namespaces. This step replaces the original rxe module's global data structure rxe with a namespace-based pernet_operations structure to achieve resource isolation. It uses the net_generic method to implement independent GID-to-IP address mapping in multiple namespaces, avoiding GID conflicts in multi-tenant scenarios. S5 system components and their functional steps, which include the topology management engine, namespace manager, performance optimization and precautions, and Soft-RoCE enhancement module.

2. A virtual network topology simulation method based on Soft-RoCE according to claim 1, characterized in that: The S1 step of creating an isolated virtual node also includes deploying a virtual network device, configuring an IP address and activating the device, configuring routing rules, and initializing a Soft-RoCE instance. The deploying virtual network device step includes establishing a logical link between namespaces to simulate a physical network connection; configuring an IP address and activating the device step includes assigning an IP address to the virtual node and activating the network interface to support communication; configuring routing rules includes configuring a routing table if the virtual node needs to access an external network or other subnet; and initializing the Soft-RoCE instance step includes starting an independent RDMA protocol stack in each namespace.

3. The method for simulating a virtual network topology based on Soft-RoCE according to claim 2, wherein: The S2 step of configuring the virtual network link attributes also includes a target virtual device positioning link, a link delay configuration link, a packet loss rate configuration link, a bandwidth limitation configuration link, and a traffic shaping configuration link. The target virtual device positioning link includes determining the virtual Ethernet device for which link attributes need to be configured; the link delay configuration link includes simulating the transmission delay of data packets on the link; the packet loss rate configuration link includes simulating data packet loss in the link; the bandwidth limitation configuration link includes limiting the maximum transmission rate of the link; and the traffic shaping configuration link includes simulating complex network behaviors, such as data packet reordering, duplication, or damage.

4. The method for simulating a virtual network topology based on Soft-RoCE according to claim 3, wherein: The S3 dynamically generates and manages the virtual topology, which includes defining a topology configuration file, parsing the configuration file and generating the topology, initializing the RDMA protocol stack, dynamically adjusting the topology, managing and monitoring the topology, and verifying and testing cases. The topology configuration file definition step includes describing the logical topology and link parameters of the virtual RDMA network through a structured file to achieve rapid deployment and dynamic adjustment; parsing the configuration file and generating the topology step includes converting the configuration file into deployment instructions for actual virtual nodes and links; and initializing the RDMA protocol stack step includes starting a Soft-RoCE instance that supports multiple namespaces in each virtual node.

5. The method for simulating a virtual network topology based on Soft-RoCE according to claim 4, wherein: The dynamic topology adjustment link includes modifying the topology structure or link parameters during the operation of the virtual network; the topology management and monitoring link includes maintaining the virtual network status and providing visual monitoring; the verification and test case link includes ensuring that the virtual topology works as expected and supports the target scenario.

6. The method for simulating a virtual network topology based on Soft-RoCE according to claim 5, wherein: The improved RXE kernel module supports multiple namespaces. This step includes analyzing the global limitations of the original module, reconstructing the module to support network namespaces, enhancing cross-namespace communication capabilities, and verification and testing. The analysis of the global limitations of the original module includes using a global data structure to manage RDMA resources, resulting in an inability to isolate resources in different network namespaces. The reconstruction of the module to support network namespaces includes transforming global resources into independent resource pools based on network namespaces. The enhancement of cross-namespace communication capabilities includes modifying the CM event processing logic to implement cross-namespace routing queries.

7. The method for simulating a virtual network topology based on Soft-RoCE according to claim 6, wherein: The verification and testing phase includes testing multi-namespace device isolation, testing GID isolation, and testing cross-namespace RDMA communication; the performance optimization and precautions phase includes concurrency control, using read-write locks to protect resource access within the namespace, memory management, ensuring that all associated resources are released when the namespace is destroyed, kernel compatibility, and adapting the pernet_operations API to different kernel versions.

8. The method for simulating a virtual network topology based on Soft-RoCE according to claim 7, wherein: The S5 system components and their functional steps include the topology management engine link in this step, which is responsible for parsing the configuration file, generating node connections, link parameters and GID allocation instructions, and supporting dynamic topology adjustment; the namespace manager link includes creating and maintaining a mapping relationship table between virtual nodes and network namespaces, and calling the Linux kernel interface to implement an isolated namespace environment; the Soft-RoCE enhancement module link includes expanding support for multiple namespaces on the basis of the standard and providing cross-namespace communication capabilities.

9. The method for simulating a virtual network topology based on Soft-RoCE according to claim 8, wherein: It also includes an operation and verification step, which includes starting a virtual RDMA network link and a dynamic adjustment example link. Starting the virtual RDMA network link includes executing a topology management engine and generating virtual nodes and links according to a configuration file. The dynamic adjustment example link includes adjusting the bandwidth from node 1 to node 2 from 10Gbps to 5Gbps during operation.

10. The method for simulating a virtual network topology based on Soft-RoCE according to claim 9, characterized in that: It also includes core processes, including 1. Isolation environment construction → 2. Virtual link configuration → 3. Topology definition and deployment → 4. RDMA protocol stack enhancement → 5. Dynamic operation and maintenance management.

Images