A high-precision flow control protocol simulation system and method for large-scale networks

Abstract

The application discloses a high-precision flow control protocol simulation system and method for large-scale networks, and relates to the field of network operation simulation services.In the system, a user flow control protocol integration module automatically converts a congestion control protocol algorithm or a flow control protocol provided by a user into a BPF program; a congestion control protocol simulation module compiles the BPF program into bytecode capable of being run by a Linux kernel and mounts the bytecode at a hook function.A flow generator generates flow by using the mounted congestion control algorithm or flow control protocol; a data collection and processing module collects and processes the flow of the flow control protocol simulation experiment by using a database; and an experimental network topology management tool generates a network topology and performs a simulation experiment based on the flow generated by the flow generator.The application can reduce simulation overhead, completely and truly reflect complex conditions in an actual network, and thus does not limit the applicability and authenticity of simulation results.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of network operation simulation services, in particular to a high-precision traffic control protocol simulation system and method for large-scale networks. BACKGROUND

[0002] New generation networks are gradually becoming complex and heterogeneous, and the cost of building networks is also gradually increasing. In order to simulate the actual network operation, thereby helping researchers, engineers and decision makers to better understand the behavior of network systems, save costs, optimize network performance and testing, network simulation can be used to evaluate network design, protocol performance, topology and other related factors. If the delay, bandwidth and other indicators of the network are simulated and made to fit the real value, the traffic needs to be controlled. The traffic control protocol is a network communication protocol used to manage the speed of data transmission to ensure that the receiving end will not be congested or data lost due to the speed of receiving data being too fast. Therefore, the traffic control protocol needs to be simulated to save costs and evaluate the performance of the protocol when designing the traffic control protocol.

[0003] It is known that there are mainly two kinds of tools for simulating traffic control protocols: one is a discrete event-based simulation tool, and the other is an existing network simulation tool based on virtualization technology.

[0004] Among them, the discrete event-based simulation tool uses discrete events for computing the data required for communication between nodes. When the number of nodes is too large, the time required to calculate the result is long, and the amount of resources consumed is large, and the simulation event-driven model needs to process a large number of discrete events. In a large-scale network, the number of events can grow exponentially, resulting in a significant increase in computing overhead, which in turn affects the simulation speed. It does not support native distributed simulation functions. Limited multi-core support is provided to improve simulation efficiency through parallel processing. However, this parallelization usually requires manual task allocation and modification of the simulation script, so the complexity is high when used. Therefore, the following disadvantages exist when using a discrete event-based simulation tool to simulate a traffic control protocol:

[0005] 1) If a new traffic control protocol needs to be simulated, a new traffic control layer needs to be implemented, which is complex and requires a large amount of work.

[0006] 2) When the number of nodes is large, memory consumption is one of the main bottlenecks of discrete event-based simulation. The simulation event-driven model needs to process a large number of discrete events. In a large-scale network, the number of events can grow exponentially, resulting in a significant increase in computing overhead, which in turn affects the simulation speed.

[0007] 3) It does not support native distributed simulation functions.

[0008] The principle of existing network simulation tools based on virtualization technology, such as Mininet, to simulate a flow control protocol is to create a virtual network topology on a single Linux kernel. This network topology includes switches, routers and hosts and the like. Then a user mode TCP (Transmission Control Protocol) protocol stack or a flow control protocol algorithm in the system kernel is implemented to simulate the network as a flow control protocol. Switching when implementing a new flow control protocol is complex and introduces a lot of overhead, and when simulating a large-scale network, the performance bottleneck becomes more significant. Since all virtual nodes share the resources of the same host machine, when the simulation scale expands, system resources become a bottleneck, resulting in slow simulation or simulation failure. Based on this, the main shortcomings of simulating a flow control protocol using existing network simulation tools based on virtualization technology are as follows:

[0009] 1) A user-defined flow control protocol needs to be implemented in the Linux kernel first, and complex kernel protocols need to be implemented when simulating different protocols, which is large in simulation overhead.

[0010] 2) Although network simulation tools based on virtualization technology can simulate network conditions such as delay, bandwidth and packet loss rate, these simulations are often based on simple models and cannot fully and truly reflect the complex situation in the actual network, which may limit the applicability and authenticity of the simulation results for the study of flow control in the network. SUMMARY

[0011] The purpose of the present application is to provide a high-precision flow control protocol simulation system and method for large-scale networks, which can reduce simulation overhead and fully and truly reflect the complex situation in the actual network, thereby not limiting the applicability and authenticity of the simulation results.

[0012] To achieve the above purpose, the present application provides the following solutions:

[0013] In a first aspect, the present application provides a high-precision flow control protocol simulation system for large-scale networks, comprising:

[0014] A user flow control protocol integration module for automatically converting a congestion control protocol algorithm or a flow control protocol provided by a user into a BPF program;

[0015] A congestion control protocol simulation module connected to the user flow control protocol integration module, for compiling a BPF program corresponding to a congestion control algorithm or a flow control protocol into bytecode executable by a Linux kernel, and mounting the bytecode to a hook function of a Linux kernel network stack to implement loading and application of the congestion control algorithm or the flow control protocol.

[0016] a flow generator, connected with the congestion control protocol emulation module, for generating flow of the flow control protocol emulation experiment using a congestion control algorithm or a flow control protocol mounted at a hook function of a Linux kernel network stack;

[0017] a data collection processing module, connected with the flow generator, for collecting and processing data of the flow control protocol emulation experiment using a database; the data of the flow control protocol emulation experiment includes flow and data packets generated in the flow control protocol emulation experiment;

[0018] an experimental network topology management tool, for generating a network topology required for the flow control protocol emulation experiment, deploying and managing virtual routers, switches and emulation nodes according to the network topology using a function of Kubernetes, and performing the emulation experiment based on the flow generated by the flow generator.

[0019] Optionally, the flow generator comprises a flow sending end and a flow receiving end.

[0020] The flow sending end sends a random network packet; the network packet uses a congestion control protocol for flow control when entering a transmission layer of a Linux kernel network stack, and is then sent out to reach the flow receiving end through a series of network transmission protocols.

[0021] Optionally, the data collection processing module comprises:

[0022] a network flow acquisition submodule, for acquiring flow and flow timestamps of the flow control protocol emulation experiment from the Linux kernel using a database, and performing statistics on packet sizes;

[0023] a data processing submodule, connected with the network flow acquisition submodule, for determining flow parameters in a set time period in the network based on the flow, the flow timestamps and the statistical packet sizes acquired by the network flow acquisition submodule; the flow parameters include flow throughput, bandwidth and latency.

[0024] Optionally, the network flow acquisition submodule acquires and stores the flow of the flow control protocol emulation experiment using a BPF map.

[0025] Optionally, a large language model is implanted in the user flow control protocol integration module; the large language model is used to automatically convert C language code or an algorithm description file of a congestion control protocol algorithm or a flow control protocol input by a user into a BPF program.

[0026] Optionally, the experimental network topology management tool deploys the simulation nodes to different physical machines using a network topology division algorithm when generating the network topology, to obtain physical nodes.

[0027] In a second aspect, the application provides a high-precision flow control protocol simulation method for large-scale networks, which is applied to the high-precision flow control protocol simulation system for large-scale networks provided above; the high-precision flow control protocol simulation method for large-scale networks comprises:

[0028] automatically converting a congestion control protocol algorithm or a flow control protocol provided by a user into a BPF program;

[0029] compiling the BPF program corresponding to the congestion control algorithm or the flow control protocol into bytecode executable by a Linux kernel, and mounting the bytecode to a hook function of a Linux kernel network stack of a simulation node;

[0030] when a function of the TCP protocol is running, using the hook function to mount the flow control protocol required by the user to a corresponding mounting point and run the flow control protocol;

[0031] simulating the simulation node using a container of the flow control protocol that has been mounted, to obtain a network topology; the simulation node and a corresponding physical machine use the same Linux kernel network stack; a network topology division algorithm is used to deploy the simulation node to different physical machines when generating the required network topology;

[0032] generating flow required for a flow control protocol simulation experiment in the network topology;

[0033] collecting the generated flow using a BPF map and storing the flow into a database;

[0034] performing data processing and visualization operations based on the collected flow.

[0035] According to the specific embodiments provided by the application, the application has the following technical effects:

[0036] The application provides a high-precision flow control protocol simulation system and method for large-scale networks, which automatically converts a congestion control protocol algorithm or a flow control protocol provided by a user into a BPF program by using a user flow control protocol integration module, so that the user can perform flow control protocol simulation by only inputting C language code or an algorithm description file of a kernel version, without re-implementing a corresponding flow control protocol by using a system-provided API (Application Program Interface), reducing the complexity of code modification and implementation, ensuring that the flow control protocol is consistent with the original flow control protocol function of the kernel, and improving the fidelity of simulation. The network topology required for flow control protocol simulation experiments is generated by using an experimental network topology management tool, which can fully and truly reflect the complex situation in the actual network, thereby not limiting the applicability and authenticity of the simulation results. The data collection and processing module collects and processes the data of the flow control protocol simulation experiment using a database, the generated flow data does not need to be copied from the kernel space to the user space, can be used for real-time analysis of network flow, and can respond based on kernel events, reducing simulation overhead. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 A structural schematic diagram of a high-precision flow control protocol simulation system for large-scale networks provided by an embodiment of the present application is shown in the figure.

[0039] Figure 2 An implementation flowchart of a high-precision flow control protocol simulation system for large-scale networks provided by an embodiment of the present application is shown in the figure.

[0040] Figure 3 A flowchart of a high-precision flow control protocol simulation method for large-scale networks provided by another embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0042] The high-precision flow control protocol simulation system and method for large-scale networks provided by the present application adopts a network simulation method based on virtualization technology, can reduce simulation overhead, completely and truly reflect the complex situation in the actual network, and thus will not limit the applicability and authenticity of the simulation results.

[0043] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0044] In one exemplary embodiment, a high-precision flow control protocol simulation system for large-scale networks is provided, which uses the Kubernetes deployment minimum unit, container group (Pod), as the simulation network node (i.e., simulation node), and each node contains a Linux kernel network stack for simulating the flow control protocol and various modules for simulation as shown in Figure 1 , wherein:

[0045] 1) User flow control protocol integration module: used to automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program, and then the BPF program of the congestion control protocol algorithm or flow control protocol will be used in the congestion control protocol simulation module.

[0046] 2) Congestion control protocol simulation module: mainly responsible for compiling the BPF program corresponding to the congestion control algorithm or flow control protocol required by the user into bytecode that can be run by the Linux kernel, and mounting it to a specific hook function (i.e., Hook function) of the simulation node Linux kernel network stack, realizing the loading and application of the user-defined congestion control algorithm.

[0047] 3) Flow generator: contains a flow sending end and a flow receiving end in each simulation node. The sending and receiving of the flow uses the congestion control protocol algorithm or flow control protocol loaded into the Linux kernel network stack by the congestion control protocol simulation module. For example, the flow sending end sends a random network packet, which uses the user-defined congestion control protocol for flow control when entering the transport layer of the network stack, and then is sent out and reaches the flow receiving end through a series of network transmission protocols.

[0048] 4) Data collection processing module: including network traffic acquisition submodule and data processing submodule, mainly used for collecting and visualizing data of traffic control protocol simulation experiment using database. The network traffic acquisition submodule acquires the traffic generated by the traffic generator from the Linux kernel and stores it in the database, and then acquires the traffic timestamp and counts the packet size. The data processing submodule calculates the throughput, bandwidth, and delay of the traffic in the network within a period of time based on the collected data.

[0049] 5) Experimental network topology management tool: used to generate the network topology required by the user for traffic control protocol simulation. Virtual routers, switches, and virtual simulation nodes are deployed according to the network topology using the functions of Kubernetes. Then the congestion control protocol simulation module in the simulation node loads the traffic control protocol, and the traffic generator generates traffic for simulation.

[0050] As another embodiment of the present application, in order to reduce the CPU and memory overhead of single physical machine simulation and prevent affecting the efficiency and performance of simulation, the experimental network topology management tool uses a network topology division algorithm to deploy simulation nodes on different physical machines when generating the required network topology, obtaining a physical node as shown in Figure 1

[0051] As another embodiment of the present application, in order to make the generated traffic data not need to be copied from the kernel space to the user space, the network traffic acquisition submodule provided by the present application uses BPF map to acquire and store the traffic of the traffic control protocol simulation experiment. Using BPF map can directly access and analyze the data structure and events in the kernel, which can be used for real-time analysis of network traffic and response based on kernel events.

[0052] As another embodiment of the present application, in order to be able to perform traffic control protocol simulation without using the system-provided API to re-implement the corresponding traffic control protocol, reduce the complexity of code modification and implementation, ensure that the traffic control protocol is consistent with the original traffic control protocol function of the kernel, and improve the fidelity of the simulation, the user traffic control protocol integration module provided by the present application is implanted with a large language model. The large language model is used to automatically convert the congestion control protocol algorithm or C language code or algorithm description file input by the user into a BPF program.

[0053] Based on the structure given above, the working process of the high-precision traffic control protocol simulation system for large-scale networks provided by the present application is as shown in Figure 2

[0054] ​​(1) The user flow control protocol algorithm description file or the implementation code of the kernel version input by the user is converted into compilable BPF code by the user flow control protocol integration module.

[0055] (2) The BPF code binary file that has been compiled is mounted into the Linux kernel of the simulation node by the congestion control protocol simulation module, and the user required flow control protocol is mounted to the corresponding mounting point and run using a hook function at different function runtime of the TCP protocol (Transmission Control Protocol).

[0056] (3) The experimental network topology management tool uses the container loaded with the required flow control protocol to simulate the simulation node, which uses the same Linux network stack as the corresponding physical machine. The experimental network topology management tool uses a network topology division algorithm to deploy the simulation node to different physical machines when generating the required network topology.

[0057] (4) The required traffic for the experiment is generated in the network topology by the traffic generator.

[0058] (5) After the BPF map is used to collect the traffic generated in the experiment and stored in the database by the data collection processing module, a series of data processing and visualization operations are performed.

[0059] Further, the network stack used by the present application is the Linux system kernel network stack. The flow control protocol implemented in the user state needs to be mounted in the system kernel network stack, and the network packet is sent and received during the flow control protocol simulation to implement the simulation of different flow control protocols required by the user.

[0060] In addition, if the Linux kernel is replaced by a traditional kernel module, the flow control protocol is developed using a traditional kernel module, and the kernel mounted with the flow control kernel module required by the user is used for simulation. However, this scheme needs to modify and recompile the kernel, which may bring security risks and maintenance costs. Different kernel modules may use different kernel versions, which is less efficient.

[0061] Based on the same inventive concept, the embodiments of the present application also provide a large-scale network-oriented high-precision flow control protocol simulation method applied to the large-scale network-oriented high-precision flow control protocol simulation system described above. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme described in the above method, so the specific limitations in one or more large-scale network-oriented high-precision flow control protocol simulation method embodiments provided below can refer to the limitations of the large-scale network-oriented high-precision flow control protocol simulation system in the foregoing, which will not be repeated here.

[0062] In one exemplary embodiment, as shown in Figure 3 A high-precision traffic control protocol simulation method for large-scale networks is provided, comprising:

[0063] Step 100: Automatically convert the congestion control protocol algorithm or traffic control protocol provided by the user into a BPF program.

[0064] Step 101: Compile the BPF program corresponding to the congestion control algorithm or traffic control protocol into bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack of the simulation node.

[0065] Step 102: When the different functions of the TCP protocol are running, use the hook function to mount the traffic control protocol required by the user to the corresponding mounting point and run it.

[0066] Step 103: Simulate the simulation node using the container of the traffic control protocol that has been mounted, and obtain the network topology. The simulation node and the corresponding physical machine use the same Linux kernel network stack. When generating the required network topology, use a network topology division algorithm to deploy the simulation node to different physical machines.

[0067] Step 104: Generate the traffic required for the traffic control protocol simulation experiment in the network topology.

[0068] Step 105: Use BPF map to collect the generated traffic and store it in the database.

[0069] Step 106: Perform data processing and visualization operations based on the collected traffic.

[0070] Based on the above description, compared with existing discrete event-based network simulation platforms, the present application also has the following advantages:

[0071] 1. The present application supports distributed traffic control protocol simulation and does not need to calculate discrete events, which can improve simulation efficiency and reduce simulation time.

[0072] 2. The present application does not need to implement different congestion control protocol codes according to different simulation systems, but only needs C language code that can be run by the kernel, which can ensure consistent program functions and reduce the complexity of implementing simulation functions.

[0073] 3. The present application uses BPF to directly switch traffic control protocols and BPF map to obtain data, which improves simulation efficiency and reduces simulation overhead.

[0074] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0075] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. In the embodiments provided in the present application, any reference to memory, database or other medium can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0076] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0077] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0078] The principles and implementations of the present application are described in detail with specific examples in this paper, and the above examples are only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A high-precision flow control protocol simulation system for large-scale networks, characterized by: The high-precision flow control protocol simulation system for large-scale networks includes: A user flow control protocol integration module, configured to automatically convert a user-provided congestion control protocol algorithm or flow control protocol into a BPF program; wherein the user flow control protocol integration module is embedded with a large language model; the large language model is configured to automatically convert the user-input C language code or algorithm description file of the congestion control protocol algorithm or flow control protocol into a BPF program; A congestion control protocol simulation module, connected to the user flow control protocol integration module, is used to compile the BPF program corresponding to the congestion control algorithm or flow control protocol into bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack to realize the loading and application of the congestion control algorithm or flow control protocol; A traffic generator, connected to the congestion control protocol simulation module, for generating traffic for a flow control protocol simulation experiment using a congestion control algorithm or flow control protocol mounted at a hook function of a Linux kernel network stack; A data collection and processing module, connected to the traffic generator, is used to collect and process data of the flow control protocol simulation experiment using a database; the data of the flow control protocol simulation experiment includes the flow and data packets generated during the flow control protocol simulation experiment; Experimental network topology management tool, used to generate the network topology required for traffic control protocol simulation experiments, use Kubernetes functions to deploy and manage virtual routers, switches, and simulation nodes according to the network topology, and conduct simulation experiments based on the traffic generated by the traffic generator.

2. The high-precision flow control protocol simulation system for large-scale networks according to claim 1, characterized in that: The traffic generator includes: a traffic sending end and a traffic receiving end; The traffic sending end sends a random network packet; when the network packet enters the transport layer of the Linux kernel network stack, it uses the congestion control protocol to perform flow control, and then is sent out and reaches the traffic receiving end through a series of network transmission protocols.

3. The high-precision flow control protocol simulation system for large-scale networks according to claim 1, characterized in that: The data collection and processing module includes: The network traffic collection submodule is used to obtain the traffic and traffic timestamps of the flow control protocol simulation experiment from the Linux kernel using a database, and to perform statistics on the size of data packets; The data processing submodule is connected to the network traffic collection submodule and is used to determine the traffic parameters in a set time period in the network based on the traffic, traffic timestamp and statistical data packet size obtained by the network traffic collection submodule; the traffic parameters include traffic throughput, bandwidth and latency.

4. The high-precision flow control protocol simulation system for large-scale networks according to claim 3, characterized in that: The network traffic collection submodule uses BPF map to collect and store the traffic of the flow control protocol simulation experiment.

5. The high-precision flow control protocol simulation system for large-scale networks according to claim 1, characterized in that: When generating the network topology, the experimental network topology management tool uses a network topology partitioning algorithm to deploy simulation nodes to different physical machines to obtain physical nodes.

6. A high-precision flow control protocol simulation method for large-scale networks, characterized in that: The high-precision flow control protocol simulation method for large-scale networks is applied to the high-precision flow control protocol simulation system for large-scale networks according to any one of claims 1 to 5; the high-precision flow control protocol simulation method for large-scale networks comprises: Automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program; Compile the BPF program corresponding to the congestion control algorithm or flow control protocol into bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack of the simulation node; When different functions of the TCP protocol are running, the hook function is used to mount the flow control protocol required by the user to the corresponding mount point and run it; Using a container with a mounted flow control protocol to simulate a simulation node to obtain a network topology; the simulation node and the corresponding physical machine use the same Linux kernel network stack; when generating the required network topology, the simulation node is deployed on different physical machines using a network topology partitioning algorithm; Generate the traffic required for flow control protocol simulation experiments in the network topology; Use BPF map to collect generated traffic and store it in the database; Perform data processing and visualization based on the collected traffic.

Images