Large-scale network-oriented high-precision flow control protocol simulation system and method

Abstract

The invention discloses a high-precision flow control protocol simulation system and method for a large-scale network, and relates to the field of network operation simulation services. In the system, a user flow control protocol integration module automatically converts a congestion control protocol algorithm or a flow control protocol provided by a user into a BPF program; and the congestion control protocol simulation module compiles the BPF program into a byte code which can be operated by the Linux kernel, and mounts the byte code to the hook function. The traffic generator generates traffic by using a mounted congestion control algorithm or a traffic control protocol; the data collecting and processing module collects and processes the flow of the flow control protocol simulation experiment by using a database; and the experimental network topology management tool generates a network topology and performs a simulation experiment based on the flow generated by the flow generator. According to the method and the device, the simulation overhead can be reduced, the complex condition in an actual network can be completely and truly reflected, and the applicability and authenticity of a simulation result cannot be limited.

Description

Technical Field

[0001] The present application relates to the field of network operation simulation services, and in particular to a high-precision flow control protocol simulation system and method for large-scale networks. Background Art

[0002] The new generation of networks is becoming increasingly complex and heterogeneous, and the cost of building networks is also gradually increasing. In order to simulate the actual network operation, so as to help researchers, engineers and decision makers better understand the behavior of network systems, save costs, optimize network performance and testing, network simulation can be used to evaluate network design, protocol performance, topology and other related factors. If the network's latency, bandwidth and other indicators are simulated and these indicators are made to fit the real values, the flow needs to be controlled. The flow control protocol is a network communication protocol used to manage the speed of data transmission to ensure that the receiving end does not experience congestion or data loss due to the speed of receiving data too fast. Therefore, it is necessary to simulate the flow control protocol to save costs and evaluate the performance of the protocol when designing the flow control protocol.

[0003] There are two main known tools for simulating flow control protocols: one is a simulation tool based on discrete events, and the other is an existing network simulation tool based on virtualization technology.

[0004] Among them, discrete event-based simulation tools use discrete events to calculate the data required for communication between nodes. When the number of nodes is too large, it takes a long time to calculate the results and consumes a lot of resources. The simulation event-driven model needs to process a large number of discrete events. In large-scale networks, the number of events may grow exponentially, resulting in a significant increase in computing overhead, which in turn affects the simulation speed. Native distributed simulation functions are not supported. Limited multi-core support is provided to improve simulation efficiency through parallel processing. However, this parallelization usually requires manual task allocation and modification of simulation scripts, so it is more complex when used. Based on this, the use of discrete event-based simulation tools for simulating flow control protocols has the following main disadvantages:

[0005] 1) If a new flow control protocol needs to be simulated, a new flow control layer needs to be implemented, which is relatively complex and labor-intensive.

[0006] 2) When the number of nodes is large, memory consumption is one of the main bottlenecks of discrete event simulation. The simulation event-driven model needs to process a large number of discrete events. In a large-scale network, the number of events may grow exponentially, resulting in a significant increase in computational overhead, which in turn affects the simulation speed.

[0007] 3) Does not support native distributed simulation functions.

[0008] Existing network simulation tools based on virtualization technology, such as Mininet, simulate flow control protocols based on the principle of creating a virtual network topology on a single Linux kernel. This network topology includes switches, routers, and hosts. Subsequently, the user-mode TCP (Transmission Control Protocol) protocol stack is implemented or the flow control protocol algorithm in the system kernel is implemented as a flow control protocol for network simulation. Switching is more complicated when implementing a new flow control protocol, which will introduce a lot of overhead, and when simulating large-scale networks, performance bottlenecks will become more significant. Since all virtual nodes share the resources of the same host machine, when the simulation scale is expanded, system resources will become a bottleneck, resulting in slower simulation speed or simulation failure. Based on this, the use of existing network simulation tools based on virtualization technology to simulate flow control protocols mainly has the following disadvantages:

[0009] 1) The user-defined traffic control protocol needs to be implemented in the Linux kernel first. When simulating different protocols, it is necessary to implement complex kernel protocols and switch between different protocols, which results in high simulation overhead.

[0010] 2) Although network simulation tools based on virtualization technology can simulate network conditions such as delay, bandwidth and packet loss rate, these simulations are often based on simple models and cannot fully and truly reflect the complex situations in the actual network. Research on traffic control in the network may limit the applicability and authenticity of the simulation results. Summary of the invention

[0011] The purpose of this application is to provide a high-precision flow control protocol simulation system and method for large-scale networks, which can reduce simulation overhead and fully and truly reflect the complex situations in the actual network without limiting the applicability and authenticity of the simulation results.

[0012] To achieve the above objectives, this application provides the following solutions:

[0013] In a first aspect, the present application provides a high-precision flow control protocol simulation system for large-scale networks, comprising:

[0014] User flow control protocol integration module, used to automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program;

[0015] A congestion control protocol simulation module is connected to the user flow control protocol integration module, and is used to compile the BPF program corresponding to the congestion control algorithm or the flow control protocol into a bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack to realize the loading and application of the congestion control algorithm or the flow control protocol;

[0016] A traffic generator, connected to the congestion control protocol simulation module, used to generate traffic for a flow control protocol simulation experiment using a congestion control algorithm or flow control protocol mounted at a hook function of a Linux kernel network stack;

[0017] A data collection and processing module, connected to the traffic generator, is used to collect and process the data of the flow control protocol simulation experiment using a database; the data of the flow control protocol simulation experiment includes the flow and data packets generated during the flow control protocol simulation experiment;

[0018] The experimental network topology management tool is used to generate the network topology required for the flow control protocol simulation experiment. It uses the functions of Kubernetes to deploy and manage virtual routers, switches, and simulation nodes according to the network topology, and conducts simulation experiments based on the traffic generated by the traffic generator.

[0019] Optionally, the traffic generator includes: a traffic sending end and a traffic receiving end;

[0020] The traffic sending end sends a random network packet; when the network packet enters the transport layer of the Linux kernel network stack, a congestion control protocol is used to perform flow control, and then the packet is sent out and reaches the traffic receiving end through a series of network transmission protocols.

[0021] Optionally, the data collection and processing module includes:

[0022] The network traffic collection submodule is used to obtain the traffic and traffic timestamp of the traffic control protocol simulation experiment from the Linux kernel using the database, and to perform statistics on the size of data packets;

[0023] The data processing submodule is connected to the network traffic collection submodule and is used to determine the traffic parameters in a set time period in the network based on the traffic, traffic timestamp and statistical data packet size obtained by the network traffic collection submodule; the traffic parameters include traffic throughput, bandwidth and latency.

[0024] Optionally, the network traffic collection submodule uses BPF map to collect and store the traffic of the traffic control protocol simulation experiment.

[0025] Optionally, a large language model is implanted in the user traffic control protocol integration module; the large language model is used to automatically convert the C language code or algorithm description file of the congestion control protocol algorithm or traffic control protocol input by the user into a BPF program.

[0026] Optionally, when generating the network topology, the experimental network topology management tool uses a network topology partitioning algorithm to deploy simulation nodes to different physical machines to obtain physical nodes.

[0027] In a second aspect, the present application provides a high-precision flow control protocol simulation method for large-scale networks, and the high-precision flow control protocol simulation method for large-scale networks is applied to the high-precision flow control protocol simulation system for large-scale networks provided above; the high-precision flow control protocol simulation method for large-scale networks includes:

[0028] Automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program;

[0029] Compile the BPF program corresponding to the congestion control algorithm or flow control protocol into bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack of the simulation node;

[0030] When different functions of the TCP protocol are running, the hook function is used to mount the flow control protocol required by the user to the corresponding mount point and run it;

[0031] Using a container of a mounted flow control protocol to simulate a simulation node, a network topology is obtained; the simulation node and the corresponding physical machine use the same Linux kernel network stack; when generating the required network topology, a network topology partitioning algorithm is used to deploy the simulation node to different physical machines;

[0032] Generate the traffic required for the flow control protocol simulation experiment in the network topology;

[0033] Use BPF map to collect the generated traffic and store it in the database;

[0034] Perform data processing and visualization based on the collected traffic.

[0035] According to the specific embodiments provided in this application, this application has the following technical effects:

[0036] The present application provides a high-precision flow control protocol simulation system and method for large-scale networks. By using a user flow control protocol integration module to automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program, the user only needs to enter the C language code or algorithm description file of the kernel version to perform flow control protocol simulation, without using the API (Application Program Interface) provided by the system to re-implement the corresponding flow control protocol, reduce the complexity of code changes and implementation, ensure that the flow control protocol is consistent with the original flow control protocol function of the kernel, and improve the fidelity of the simulation. By using an experimental network topology management tool to generate the network topology required for the flow control protocol simulation experiment, it can fully and truly reflect the complex situation in the actual network, and thus will not limit the applicability and authenticity of the simulation results. By using a data collection and processing module to use a database to collect and process the data of the flow control protocol simulation experiment, the generated flow data does not need to be copied from the kernel space to the user space, and can be used to analyze network traffic in real time, and respond based on kernel events to reduce simulation overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A schematic diagram of the structure of a high-precision flow control protocol simulation system for large-scale networks provided in one embodiment of the present application;

[0039] Figure 2 A schematic diagram of an implementation process of a high-precision flow control protocol simulation system for large-scale networks provided in one embodiment of the present application;

[0040] Figure 3 A flowchart of a high-precision flow control protocol simulation method for large-scale networks provided in another embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] The high-precision flow control protocol simulation system and method for large-scale networks provided in the present application adopt a network simulation method based on virtualization technology, which can reduce simulation overhead and fully and realistically reflect the complex situations in the actual network, thereby not limiting the applicability and authenticity of the simulation results.

[0043] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0044] In an exemplary embodiment, a high-precision flow control protocol simulation system for large-scale networks is provided. The system uses the smallest deployment unit of Kubernetes, the container group (Pod), as a node of the simulation network (i.e., a simulation node). Each node includes a Linux kernel network stack for simulating flow control protocols and Figure 1 The modules used for simulation are shown in Figure 1, where:

[0045] 1) User flow control protocol integration module: used to automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program, and then the BPF program of this congestion control protocol algorithm or flow control protocol will be used in the congestion control protocol simulation module.

[0046] 2) Congestion control protocol simulation module: It is mainly responsible for compiling the BPF program corresponding to the congestion control algorithm or flow control protocol that the user needs to use into bytecode that can be run by the Linux kernel, and mounting it to the specific hook function (i.e. Hook function) of the Linux kernel network stack of the simulation node to realize the loading and application of the user-defined congestion control algorithm.

[0047] 3) Traffic generator: Each simulation node contains a traffic sender and a traffic receiver. Traffic is sent and received using the congestion control protocol algorithm or flow control protocol loaded into the Linux kernel network stack by the congestion control protocol simulation module. For example, the traffic sender sends a random network packet, and when the network packet enters the transport layer of the network stack, it uses the user-defined congestion control protocol for flow control, and then it is sent out and reaches the traffic receiver through a series of network transmission protocols.

[0048] 4) Data collection and processing module: It includes a network traffic collection submodule and a data processing submodule, which are mainly used to collect and visualize the data of the flow control protocol simulation experiment using a database. The network traffic collection submodule obtains the traffic generated by the traffic generator from the Linux kernel and stores it in the database, and then obtains the traffic timestamp and counts the size of the data packet. The data processing submodule calculates the throughput, bandwidth, delay and other parameters of the traffic in the network over a period of time based on the collected data.

[0049] 5) Experimental network topology management tool: used to generate the network topology required by users for flow control protocol simulation. Use Kubernetes functions to deploy and manage virtual routers and switches as well as virtual simulation nodes according to the network topology. Then the congestion control protocol simulation module in the simulation node loads the flow control protocol, and the traffic generator generates traffic for simulation.

[0050] As another embodiment of the present application, in order to reduce the CPU and memory overhead of a single physical machine simulation and prevent the efficiency and performance of the simulation from being affected, the experimental network topology management tool uses a network topology partitioning algorithm to deploy simulation nodes to different physical machines when generating the required network topology, and obtains the following: Figure 1 The physical nodes shown.

[0051] As another embodiment of the present application, in order to avoid copying the generated traffic data from the kernel space to the user space, the network traffic collection submodule provided by the present application uses BPF map to collect and store the traffic of the traffic control protocol simulation experiment. The use of BPF map can directly access and analyze the data structure and events in the kernel, which can be used to analyze network traffic in real time and respond based on kernel events.

[0052] As another embodiment of the present application, in order to perform flow control protocol simulation, there is no need to re-implement the corresponding flow control protocol using the API provided by the system, reduce code changes and implementation complexity, ensure that the flow control protocol is consistent with the original flow control protocol function of the kernel, and improve the fidelity of the simulation, a large language model is implanted in the user flow control protocol integration module provided by the present application. The large language model is used to automatically convert the C language code or algorithm description file of the congestion control protocol algorithm or flow control protocol input by the user into a BPF program.

[0053] Based on the above structure, the workflow of the high-precision flow control protocol simulation system for large-scale networks provided by this application is as follows: Figure 2 The entire workflow can be described as:

[0054] (1) The user traffic control protocol integration module converts the traffic control protocol algorithm description file or kernel version implementation code input by the user into a compilable BPF code.

[0055] (2) The congestion control protocol simulation module mounts the compiled BPF code binary file into the Linux kernel of the simulation node. When different functions of the TCP protocol (Transmission Control Protocol) are running, the hook function is used to mount the flow control protocol required by the user to the corresponding mount point and run it.

[0056] (3) The experimental network topology management tool uses a container that has already loaded the required traffic control protocol to simulate the simulation node. The simulation node uses the same Linux network stack as the corresponding physical machine. When generating the required network topology, the experimental network topology management tool uses the network topology partitioning algorithm to deploy the simulation nodes to different physical machines.

[0057] (4) The traffic generator generates the traffic required for the experiment in the network topology.

[0058] (5) Through the data collection and processing module, the traffic generated in the experiment is collected using the BPF map and stored in the database, and then a series of data processing and visualization operations are performed.

[0059] Furthermore, the network stack used in this application is the Linux system kernel network stack. The flow control protocol implemented in user mode needs to be mounted into the system kernel network stack, and network packets are sent and received when simulating the flow control protocol to realize the simulation of different flow control protocols required by users.

[0060] In addition, if the Linux kernel is replaced with a traditional kernel module, the flow control protocol is developed using the traditional kernel module, and the kernel with the flow control kernel module required by the user is used for simulation. However, this solution requires the kernel to be modified and recompiled, which may bring security risks and maintenance costs. Using different kernel modules may require using different kernel versions, which is inefficient.

[0061] Based on the same inventive concept, the embodiment of the present application also provides a high-precision flow control protocol simulation method for large-scale networks applied to the high-precision flow control protocol simulation system for large-scale networks involved above. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the high-precision flow control protocol simulation method for large-scale networks provided below can refer to the limitations of the high-precision flow control protocol simulation system for large-scale networks above, and will not be repeated here.

[0062] In an exemplary embodiment, Figure 3 As shown, a high-precision flow control protocol simulation method for large-scale networks is provided, including:

[0063] Step 100: Automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program.

[0064] Step 101: compile the BPF program corresponding to the congestion control algorithm or the flow control protocol into bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack of the simulation node.

[0065] Step 102: When different functions of the TCP protocol are running, a hook function is used to mount the flow control protocol required by the user to a corresponding mount point and run it.

[0066] Step 103: Use the container of the mounted traffic control protocol to simulate the simulation node and obtain the network topology. The simulation node and the corresponding physical machine use the same Linux kernel network stack. When generating the required network topology, the network topology partitioning algorithm is used to deploy the simulation node to different physical machines.

[0067] Step 104: Generate the traffic required for the traffic control protocol simulation experiment in the network topology.

[0068] Step 105: Use BPF map to collect the generated traffic and store it in the database.

[0069] Step 106: Perform data processing and visualization operations based on the collected traffic.

[0070] Based on the above description, compared with the existing discrete event-based network simulation platform, this application also has the following advantages:

[0071] 1. This application supports distributed flow control protocol simulation, which does not require the calculation of discrete events, can improve simulation efficiency and reduce the time required for simulation.

[0072] 2. This application does not need to implement different congestion control protocol codes according to different simulation systems. It only requires C language code that can be run by the kernel, which can ensure the consistency of program functions and reduce the complexity of implementing simulation functions.

[0073] 3. This application uses BPF to directly switch the flow control protocol, as well as BPF map and other data acquisition methods, which improves simulation efficiency and reduces simulation overhead.

[0074] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0075] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0076] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0077] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A high-precision flow control protocol simulation system for large-scale networks, characterized in that: The high-precision flow control protocol simulation system for large-scale networks includes: User flow control protocol integration module, used to automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program; A congestion control protocol simulation module is connected to the user flow control protocol integration module, and is used to compile the BPF program corresponding to the congestion control algorithm or the flow control protocol into a bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack to realize the loading and application of the congestion control algorithm or the flow control protocol; A traffic generator, connected to the congestion control protocol simulation module, used to generate traffic for a flow control protocol simulation experiment using a congestion control algorithm or flow control protocol mounted at a hook function of a Linux kernel network stack; A data collection and processing module, connected to the traffic generator, is used to collect and process the data of the flow control protocol simulation experiment using a database; the data of the flow control protocol simulation experiment includes the flow and data packets generated during the flow control protocol simulation experiment; The experimental network topology management tool is used to generate the network topology required for the flow control protocol simulation experiment. It uses the functions of Kubernetes to deploy and manage virtual routers, switches, and simulation nodes according to the network topology, and conducts simulation experiments based on the traffic generated by the traffic generator.

2. The high-precision flow control protocol simulation system for large-scale networks according to claim 1 is characterized in that: The traffic generator includes: a traffic sending end and a traffic receiving end; The traffic sending end sends a random network packet; when the network packet enters the transport layer of the Linux kernel network stack, a congestion control protocol is used to perform flow control, and then the packet is sent out and reaches the traffic receiving end through a series of network transmission protocols.

3. The high-precision flow control protocol simulation system for large-scale networks according to claim 1 is characterized in that: The data collection and processing module includes: The network traffic collection submodule is used to obtain the traffic and traffic timestamp of the traffic control protocol simulation experiment from the Linux kernel using the database, and to perform statistics on the size of data packets; The data processing submodule is connected to the network traffic collection submodule and is used to determine the traffic parameters in a set time period in the network based on the traffic, traffic timestamp and statistical data packet size obtained by the network traffic collection submodule; the traffic parameters include traffic throughput, bandwidth and latency.

4. The high-precision flow control protocol simulation system for large-scale networks according to claim 3 is characterized in that: The network traffic collection submodule uses BPF map to collect and store the traffic of the traffic control protocol simulation experiment.

5. The high-precision flow control protocol simulation system for large-scale networks according to claim 1 is characterized in that: A large language model is implanted in the user flow control protocol integration module; the large language model is used to automatically convert the C language code or algorithm description file of the congestion control protocol algorithm or flow control protocol input by the user into a BPF program.

6. The high-precision flow control protocol simulation system for large-scale networks according to claim 1 is characterized in that: When generating the network topology, the experimental network topology management tool uses a network topology partitioning algorithm to deploy simulation nodes to different physical machines to obtain physical nodes.

7. A high-precision flow control protocol simulation method for large-scale networks, characterized in that: The high-precision flow control protocol simulation method for large-scale networks is applied to the high-precision flow control protocol simulation system for large-scale networks as claimed in any one of claims 1 to 6; the high-precision flow control protocol simulation method for large-scale networks comprises: Automatically convert the congestion control protocol algorithm or flow control protocol provided by the user into a BPF program; Compile the BPF program corresponding to the congestion control algorithm or flow control protocol into bytecode that can be run by the Linux kernel, and mount it to the hook function of the Linux kernel network stack of the simulation node; When different functions of the TCP protocol are running, the hook function is used to mount the flow control protocol required by the user to the corresponding mount point and run it; Using a container of a mounted flow control protocol to simulate a simulation node, a network topology is obtained; the simulation node and the corresponding physical machine use the same Linux kernel network stack; when generating the required network topology, a network topology partitioning algorithm is used to deploy the simulation node to different physical machines; Generate the traffic required for the flow control protocol simulation experiment in the network topology; Use BPF map to collect the generated traffic and store it in the database; Perform data processing and visualization based on the collected traffic.

Images