Network target range traffic real-time adjustment monitoring system and method

Abstract

This invention discloses a real-time network range traffic adjustment and monitoring system and method. The system includes a task management module and a traffic execution module. The traffic execution module includes a traffic parsing unit, used to parse the traffic packet file associated with the traffic task, obtain the protocols contained in each packet, supplement the protocols that need to be included or filter out the protocols that need to be excluded, and obtain a packet template. The traffic execution unit is used to modify the source address and destination address of the packet based on the packet template, add a preset tag to the packet and send the traffic, and adjust the traffic bandwidth according to the traffic sending status monitored in real time by the traffic monitoring unit. The traffic monitoring unit is used to monitor the traffic generation of each traffic packet at a preset network interface and provide feedback to the traffic execution unit on whether the traffic bandwidth needs to be adjusted and by how much. This invention can adjust the traffic bandwidth in real time as needed, realize traffic packet-level monitoring, and improve the controllability and flexibility of traffic generation.

Description

Technical Field

[0001] This invention relates to a real-time network range traffic adjustment and monitoring system and method, belonging to the fields of network security and computer software. Background Technology

[0002] Currently, the main methods for generating traffic in cyber ranges are script-based traffic generation and pre-configured pcap playback. While these methods can realistically simulate real-life traffic scenarios, they lack flexibility for specific situations, such as generating large amounts of traffic over a long period, generating traffic at a fixed bandwidth, or generating traffic containing only certain protocols. Furthermore, current traffic generation monitoring primarily relies on monitoring traffic passing through network interfaces, making it impossible to differentiate the individual traffic patterns of multiple pcap packets on the same interface. Additionally, generating traffic through pcap playback cannot filter traffic from specific protocols or supplement traffic from protocols not present in the pcap packets, making it difficult to send targeted traffic. Summary of the Invention

[0003] Purpose of the invention: To address the shortcomings of the above-mentioned traffic generation methods, the present invention aims to provide a network range traffic real-time adjustment and monitoring system and method. This system can adjust the bandwidth of traffic in real time as needed during the traffic generation process, enabling separate monitoring of the traffic generation status of each traffic packet, thereby improving the controllability and flexibility of traffic generation.

[0004] Technical Solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution: a real-time network range traffic adjustment and monitoring system, comprising:

[0005] The task management module is used to manage traffic tasks and traffic packet files; the configuration information of the traffic task includes the target node, one or more traffic packet files used, traffic execution time, traffic bandwidth limit, and the transport layer or application layer protocol types that need to be included or excluded in the traffic.

[0006] In addition, there is a traffic execution module, which is used to parse traffic packet files, generate traffic, and monitor and adjust traffic bandwidth; including a traffic parsing unit, a traffic execution unit, and a traffic monitoring unit.

[0007] The traffic parsing unit is used to parse the traffic packet file associated with the traffic task, obtain the protocol contained in each message, supplement the protocols that need to be included or filter the protocols that need to be excluded, and obtain the message template.

[0008] The traffic execution unit is used to modify the source and destination addresses of the message based on the message template, add a preset tag to the message and send the traffic, and adjust the traffic bandwidth according to the traffic sending status monitored in real time by the traffic monitoring unit; the tag is used to associate the message with the traffic packet configured in the task.

[0009] The traffic monitoring unit is used to monitor the traffic generation of each traffic packet at a preset network port, and to report back to the traffic execution unit whether the traffic bandwidth needs to be adjusted and by how much.

[0010] Furthermore, the task management module allocates the bandwidth occupied by each traffic packet to a traffic task that uses multiple traffic packet files by configuring a bandwidth weight ratio.

[0011] Furthermore, when the protocol type required by the task configuration is not present in the traffic packet, the traffic parsing unit may randomly generate one or more packets containing the specified protocol type as needed.

[0012] Furthermore, the task management module configures bandwidth requirements and traffic execution time using a bandwidth curve. The traffic monitoring unit generates hash values ​​of time and bandwidth based on the bandwidth curve, and provides the packet sending bandwidth adjustment range to the traffic execution unit based on the actual traffic bandwidth at a specific time point and the target traffic bandwidth. The traffic execution unit adjusts the number of packets sent per unit time based on the bandwidth adjustment range.

[0013] Furthermore, the bandwidth adjustment range is determined according to the following rules: the difference between the target traffic bandwidth and the actual traffic bandwidth is divided according to a preset interval to determine the adjustment base, and the adjustment base is multiplied by a preset adjustment ratio to obtain the adjustment range.

[0014] Furthermore, when determining the bandwidth adjustment range, the traffic monitoring unit pre-sets a time base for the specific time point of the target and then compares the corresponding target traffic bandwidth with the actual traffic bandwidth at the current monitoring time point.

[0015] Furthermore, the traffic execution unit and the traffic monitoring unit are implemented using an observer pattern based on an event bus. The traffic execution unit acts as an event subscriber, and the traffic monitoring unit acts as an event publisher, notifying the traffic execution unit to adjust the traffic bandwidth. Different traffic tasks generate traffic, and different tasks are distinguished by identifiers in the event bus.

[0016] Furthermore, the traffic monitoring unit compares the maximum achievable bandwidth of the network port with the traffic bandwidth configured for the task. If the traffic bandwidth configured for the task is greater than the maximum achievable bandwidth of the network port, the task bandwidth is adjusted to not exceed the maximum achievable bandwidth of the network port.

[0017] Furthermore, the task management module records the maximum bandwidth configuration of each traffic task configured on the same network interface. When a traffic task is started, if the remaining bandwidth of the network interface associated with the task does not meet the bandwidth required by the task, a prompt will be issued. The remaining bandwidth is the maximum bandwidth that the network interface can reach minus the maximum bandwidth configuration of each traffic task that has been started.

[0018] A method for real-time adjustment and monitoring of network range traffic based on the above system includes the following steps:

[0019] Users create and configure traffic task information through the task management module;

[0020] After the user starts the traffic task, the task management module sends the task configuration information to the traffic execution module;

[0021] After receiving the task information, the traffic execution module parses the traffic packet file associated with the traffic task through the traffic parsing unit, obtains the protocol contained in each packet, supplements the protocols that need to be included or filters out the protocols that need to be excluded, and obtains the packet template.

[0022] The traffic execution module's traffic execution unit modifies the source and destination addresses of the packets based on the packet template, adds a preset tag to the packets, and then sends the traffic.

[0023] While generating traffic, the traffic execution unit monitors the traffic generation of each traffic packet on a preset network port and provides feedback to the traffic execution unit on whether the traffic bandwidth needs to be adjusted and by how much.

[0024] The traffic execution unit adjusts the traffic bandwidth based on the traffic transmission status monitored in real time by the traffic monitoring unit.

[0025] Beneficial Effects: Compared with existing technologies, this invention has the following advantages: 1. This invention monitors the traffic generation of each traffic packet through a traffic monitoring unit and provides real-time feedback to the traffic execution unit. During traffic generation, the bandwidth can be adjusted in real-time as needed. 2. This invention distinguishes different traffic packets by tagging the generated packets, enabling separate monitoring of the traffic generation of multiple traffic packets within a single task. 3. During traffic generation, this invention parses the protocol of each packet within a traffic packet, allowing for filtering of required protocol traffic or supplementation of missing protocol packets. 4. When a traffic generation task contains multiple traffic packets, this invention allocates bandwidth to each traffic packet by setting different bandwidth weight ratios, enabling different traffic packets to be generated at different rates. 5. This invention generates traffic based on a packet template using a packet copying method, allowing for real-time adjustment of the traffic transmission rate of each traffic packet. The traffic generation method is flexible and controllable, applicable to various traffic demand scenarios. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a real-time traffic adjustment and monitoring system according to an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of the flow execution process in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0029] like Figure 1As shown in the figure, an embodiment of the present invention discloses a real-time network range traffic adjustment and monitoring system, which involves a task management module, a traffic execution module, and a target node for receiving traffic data. The task management module is used to manage traffic tasks and traffic packet (pcap packet) files. The traffic task is used to configure the target node, one or more traffic packet files used when sending traffic, traffic execution time, traffic bandwidth limit, and the types of transport layer or application layer protocols that need to be included or excluded in the traffic, etc. The traffic execution module is used to parse traffic packet files, generate traffic, monitor traffic, and adjust traffic bandwidth in real time. It mainly includes a traffic parsing unit, a traffic execution unit, and a traffic monitoring unit. The traffic parsing unit parses the traffic packet files associated with the traffic task, obtains the protocols contained in each packet, supplements the protocols that need to be included or filters the protocols that need to be excluded, and obtains the packet template. The traffic execution unit modifies the source and destination addresses of the packets based on the packet template, adds preset tags to the packets (used to associate the packets with the traffic packets configured in the task), and sends the traffic. It also adjusts the traffic bandwidth according to the traffic sending status monitored in real time by the traffic monitoring unit. The traffic monitoring unit monitors the traffic generation status of each traffic packet on a preset network interface and provides feedback to the traffic execution unit on whether the traffic bandwidth needs to be adjusted and by how much.

[0030] Specifically, the traffic monitoring unit can obtain the actual maximum bandwidth available to the current network interface in real time, and compare the difference between the maximum bandwidth available to the network interface and the bandwidth configuration according to the bandwidth configuration file of the traffic task, thereby adjusting the bandwidth. This avoids the situation where the actual bandwidth sent does not meet the requirements due to network interface occupation or congestion when controlling bandwidth through the traffic execution unit.

[0031] The traffic task allows configuration of the bandwidth curve for generated traffic, enabling automatic bandwidth adjustment during traffic generation. It also allows configuration of the bandwidth weight for each pcap, allocating bandwidth among multiple pcaps within a task. Furthermore, the traffic task allows configuration of protocol content information to be filtered, enabling protocol filtering during traffic generation.

[0032] The test range platform's task management module configures the maximum bandwidth achievable by the network interface card in the current network environment as the remaining bandwidth. Simultaneously, the module records the maximum bandwidth configuration for each traffic task. When a traffic task is started, the test range platform subtracts the task's maximum bandwidth limit from the current environment's remaining bandwidth, using the difference as the latest remaining bandwidth. If the remaining bandwidth is insufficient for the traffic task's requirements, a prompt is displayed, allowing other traffic tasks to be stopped to restore the remaining bandwidth.

[0033] After starting a traffic task, the task management module can send bandwidth adjustment commands to the traffic execution module at any time during the traffic generation process, enabling real-time adjustment of the traffic task bandwidth. Each bandwidth adjustment will also update the remaining bandwidth in real time.

[0034] The traffic execution module is used to send corresponding traffic information to the target node according to the instructions sent by the test range platform. After the traffic task is started, the traffic parsing unit analyzes the pcap packets associated with the task and records the protocol information contained in each packet, thereby filtering out unnecessary packet data according to the protocol during the actual traffic transmission process. In addition, the traffic parsing unit can also randomly generate packets containing a certain protocol. When a certain protocol configured in the traffic task does not exist in the pcap packets, one or more traffic packets containing that protocol can be randomly generated as needed and added to the packets to be sent.

[0035] When the traffic execution unit performs traffic tasks, it assigns different tags to each packet of different PCAP packets, based on their characteristics. This allows the traffic monitoring unit to distinguish the traffic generation status of different PCAP packets. The traffic monitoring unit can monitor real-time data on traffic generation in the network interface, including traffic rate, number of packets sent, total traffic size, and number of error packets. The traffic monitoring unit can distinguish which PCAP packet a packet belongs to based on its tag, thus enabling separate monitoring of the traffic status of different PCAP packets when a task contains multiple PCAP packets.

[0036] The following details the method for real-time adjustment and monitoring of network range traffic based on the above system.

[0037] Users log in to the test range platform, create traffic tasks through the task management module, and configure relevant information for the traffic tasks, including the pcap required to generate traffic, traffic execution time, traffic bandwidth limit, target node, bandwidth curve of traffic, bandwidth weight of each pcap, and protocols to be filtered.

[0038] After the traffic task is configured, the user can start the traffic task, and the task management module will send the task configuration information to the traffic execution module.

[0039] like Figure 2 As shown, after receiving the task, the traffic execution module parses the traffic packet file, generates traffic, and monitors and adjusts the traffic bandwidth. Specifically, after receiving the task information, the traffic execution module calls the traffic analysis unit to parse the information of each packet in the task-related pcap packet and analyze the protocol content contained in each packet.

[0040] As shown below, the content displayed after analyzing a certain message is presented. Here, "|" is used to separate different protocols. "<Ether" represents the Ethernet protocol, "<IP " represents the IP protocol, and "<TCP" represents the TCP protocol. The following message contains the Ethernet protocol, IP protocol, and TCP protocol simultaneously. The traffic analysis unit will record this protocol information according to the message sequence number.

[0041] <Ether dst=00:1e:08:15:11:f3 src=cc:64:a6:5a:39:0f type=IPv4 |<IP version=4 ihl=5 tos=0x0 len=60 id=0 flags=DF frag=0 ttl=64 proto=tcp chksum=0x85f9 src=192.168.21.1 dst=192.168.30.113 |<TCP sport=5902 dport=41200 seq=2458843568 ack=4090706077 dataofs=10 reserved=0 flags=SA window=28960 chksum=0xb4f1 urgptr=0 options=[('MSS', 1460), ('SAckOK', b''), ('Timestamp',(161663185, 1312751905)), ('NOP', None), ('WScale', 7)] |>>>

[0042] The traffic execution module will determine whether a bandwidth curve is configured in the traffic task. If it is configured, it will generate corresponding time points based on the function formula of the bandwidth curve and the task running duration. For example, if the task execution time is 1 minute, a time point will be generated every 2 seconds, corresponding to the abscissa of the function; the maximum value of the function ordinate is the bandwidth limit, thereby generating the hash value of time and bandwidth. When the task execution time reaches a certain time point, the current theoretical bandwidth size will be obtained according to the function, and the current total bandwidth will be adjusted to achieve the automatic adjustment of bandwidth during the traffic generation process.

[0043] When the traffic execution module controls the bandwidth, it will also judge the pcap bandwidth weight ratio. If the total bandwidth is set to 100MB / s, the current traffic task contains 2 pcap packets, and the bandwidth weight ratio is 2:3, then the bandwidths of the 2 traffic packets are 40MB / s and 60MB / s respectively. <000.org /

[0044] In this embodiment, bandwidth control is implemented based on the EventBus observer pattern of Google Guava. After the task starts, the traffic execution unit registers with EventBus, becoming a subscriber to the task's events; while the traffic monitoring unit registers as a publisher of the task's events. Simultaneously, the traffic execution unit obtains each packet parsed by the traffic parsing unit, modifies the source IP and destination IP of the packets, and uses these packets as packet templates for sending traffic. When traffic transmission begins, the traffic execution module will, by default, repeatedly send all traffic packet templates at the maximum rate. The traffic monitoring unit simultaneously parses the bandwidth configuration at various points in time for the traffic task and monitors the actual bandwidth of the network interface in real time. When the current bandwidth exceeds the bandwidth configuration, the traffic monitoring module sends a bandwidth reduction event. Upon receiving this event, EventBus sends it to the subscribers who have subscribed to the task's events. Upon receiving this event, the subscribers continuously reduce the current packet transmission rate. When the traffic monitoring module detects that the current bandwidth matches the bandwidth configuration, it sends a stop bandwidth adjustment event, and the subscribers stop reducing the packet transmission rate upon receiving this event. Similarly, when the bandwidth is lower than the configured bandwidth, the traffic monitoring module will send a bandwidth increase event until the bandwidth matches, at which point it will send a stop bandwidth adjustment event. For different traffic tasks, EventBus provides an identifier parameter to distinguish between different types of EventBus. Here, the taskId of the traffic task can be used as the identifier to differentiate the event handling for different tasks.

[0045] Specifically, when the traffic monitoring unit sends a bandwidth adjustment event, it carries the bandwidth adjustment magnitude L. This magnitude is calculated based on the current bandwidth and the target bandwidth to be adjusted: the difference between the target bandwidth and the actual bandwidth is divided according to a preset interval to determine the adjustment base. The adjustment base is then multiplied by a preset adjustment ratio to obtain the adjustment magnitude. In this example, the difference between the target bandwidth and the current bandwidth is used as an interval of 100MB / s to obtain the adjustment base B. 0-100MB / s (excluding 100MB / s) is set as base 1, 100-200MB / s (excluding 200MB / s) is set as base 2, and so on, to obtain bases 3, 4, 5, and so on. The default bandwidth adjustment percentage P is 10% each time, and the adjustment interval is 0.02s. When the bandwidth needs to be adjusted from 10MB / s to 200MB / s, the bandwidth difference is 190MB / s. The adjustment base B is 2, and the final adjustment range L = P * B, which is equal to 10% * 2 = 20%. The number of packets sent for each adjustment is then calculated based on the adjustment range L. If the current number of packets sent per second is V, then the final number of packets sent for each adjustment N = V * L.

[0046] Furthermore, since bandwidth adjustment is initiated only after an event is detected, even if the bandwidth adjustment event is sent in real time, it may still cause a certain delay in the bandwidth adjustment process. Therefore, we can appropriately advance the time base, for example, by setting the advance time base to 1 second. That is, the target traffic bandwidth 1 second later is used as the target value for the current actual traffic bandwidth adjustment.

[0047] During the execution of a task, users can modify the bandwidth limit value of the task at any time through the task management module on the test range platform. After modification, the test range platform will send the latest bandwidth limit to the traffic monitoring unit. After receiving the new bandwidth limit, the traffic monitoring unit will recalculate the bandwidth allocated to each pcap based on the value and send a traffic bandwidth adjustment event to adjust the bandwidth.

[0048] If a set of message protocols to be sent is configured in the traffic task, the traffic execution unit will simultaneously obtain the protocol content contained in each message of the pcap packet recorded by the traffic analysis module when it obtains the task-related pcap packet. Before sending the traffic, messages that do not meet the requirements will be filtered out in advance, and only the remaining messages will be used as message templates. In addition, if a certain protocol does not exist in the current pcap packet, the traffic execution module can randomly generate one or more messages containing that protocol and supplement them to the message template.

[0049] When the traffic execution unit loads the packet template, it can set a tag for each packet, i.e., tag_id in the code below. By setting different tag_ids for packets from different pcap packets, and setting the same tag_id for all packets from the same pcap packet, it is possible to monitor the traffic occurrence of different pcap packets in the same task separately.

[0050] stream = STLStream(packet=STLPktBuilder(pkt=pkt, vm=vm),

[0051] mode=STLTXCont(),

[0052] isg=isg,

[0053] flow_stats=STLFlowStats(pg_id=tag_id) )

[0055] When the traffic execution unit starts generating traffic, it notifies the traffic monitoring unit, which then monitors the traffic generation on the network interface used by the task. If different pcap packets are tagged, the real-time traffic status of different packets is obtained based on their tag_ids, as shown below.

[0056] stats = stl_client.get_stats(ports)['flow_stats'].get(tag_id)

[0057] In summary, the solution in this embodiment has the following advantages: 1. During traffic generation, the bandwidth can be adjusted in real time as needed. Furthermore, if a single traffic generation task contains multiple pcaps, different bandwidth weights can be set for each pcap packet, allowing different pcaps to be generated at different rates. 2. By tagging packets from different pcaps, the generation of traffic from multiple pcaps within a single task can be monitored separately. 3. During traffic generation, the protocol of each packet in the pcap packet can be parsed, allowing for filtering of required protocol traffic or supplementation of protocol packets not present in the pcap packet.

Claims

1. A network range traffic real-time adjustment and monitoring system, characterized in that, include: The task management module is used to manage traffic tasks and traffic packet files. The configuration information of the traffic task includes the target node, one or more traffic packet files used, traffic execution time, traffic bandwidth limit, and the transport layer or application layer protocol types that need to be included or excluded in the traffic. For traffic tasks that use multiple traffic packet files, the bandwidth occupied by each traffic packet is allocated by configuring the bandwidth weight ratio. In addition, there is a traffic execution module, which is used to parse traffic packet files, generate traffic, and monitor and adjust traffic bandwidth; It includes a traffic parsing unit, a traffic execution unit, and a traffic monitoring unit; The traffic parsing unit is used to parse the traffic packet file associated with the traffic task, obtain the protocol contained in each packet, supplement the protocols that need to be included or filter the protocols that need to be excluded, and obtain the packet template; when the protocol type that needs to be included in the task configuration does not exist in the traffic packet, one or more packets containing the specified protocol type are randomly generated as needed. The traffic execution unit is used to modify the source and destination addresses of the message based on the message template, add a preset tag to the message and send the traffic, and adjust the traffic bandwidth according to the traffic sending status monitored in real time by the traffic monitoring unit; the tag is used to associate the message with the traffic packet configured in the task. The traffic monitoring unit is used to monitor the traffic generation of each traffic packet at a preset network port, and to report back to the traffic execution unit whether the traffic bandwidth needs to be adjusted and by how much; the traffic monitoring unit can tag packets to distinguish the traffic packets to which the packets belong, thereby enabling separate monitoring of the traffic status of different traffic packets when a task contains multiple traffic packets. The task management module configures bandwidth requirements and traffic execution time using a bandwidth curve. The traffic monitoring unit generates hash values ​​of time and bandwidth based on the bandwidth curve, and provides the packet sending bandwidth adjustment range to the traffic execution unit based on the actual traffic bandwidth at a specific time point and the target traffic bandwidth. The traffic execution unit adjusts the number of packets sent per unit time based on the bandwidth adjustment range.

2. The network range traffic real-time adjustment and monitoring system according to claim 1, characterized in that, The bandwidth adjustment range is determined according to the following rules: the difference between the target traffic bandwidth and the actual traffic bandwidth is divided according to a preset interval to determine the adjustment base, and the adjustment base is multiplied by a preset adjustment ratio to obtain the adjustment range.

3. The network range traffic real-time adjustment and monitoring system according to claim 1, characterized in that, When determining the bandwidth adjustment range, the traffic monitoring unit pre-sets a time base for the specific time point of the target and compares the corresponding target traffic bandwidth with the actual traffic bandwidth at the current monitoring time point.

4. The network range traffic real-time adjustment and monitoring system according to claim 1, characterized in that, The traffic execution unit and the traffic monitoring unit are implemented using an observer pattern based on an event bus. The traffic execution unit acts as an event subscriber, and the traffic monitoring unit acts as an event publisher, notifying the traffic execution unit to adjust the traffic bandwidth. Different traffic tasks generate traffic, and different tasks are distinguished by identifiers in the event bus.

5. The network range traffic real-time adjustment and monitoring system according to claim 1, characterized in that, The traffic monitoring unit compares the maximum achievable bandwidth of the network port with the traffic bandwidth configured for the task. If the traffic bandwidth configured for the task is greater than the maximum achievable bandwidth of the network port, the task bandwidth is adjusted to not exceed the maximum achievable bandwidth of the network port.

6. The network range traffic real-time adjustment and monitoring system according to claim 1, characterized in that, The task management module records the maximum bandwidth configuration of each traffic task configured on the same network interface. When a traffic task is started, if the remaining bandwidth of the network interface associated with the task does not meet the bandwidth required by the task, a prompt will be issued. The remaining bandwidth is the maximum bandwidth that the network interface can reach minus the maximum bandwidth configuration of each traffic task that has been started.

7. A method for real-time adjustment and monitoring of network range traffic based on the system according to any one of claims 1-6, characterized in that, Includes the following steps: Users create and configure traffic task information through the task management module; After the user starts the traffic task, the task management module sends the task configuration information to the traffic execution module; After receiving the task information, the traffic execution module parses the traffic packet file associated with the traffic task through the traffic parsing unit, obtains the protocol contained in each packet, supplements the protocols that need to be included or filters out the protocols that need to be excluded, and obtains the packet template. The traffic execution module's traffic execution unit modifies the source and destination addresses of the packets based on the packet template, adds a preset tag to the packets, and then sends the traffic. While generating traffic, the traffic execution unit monitors the traffic generation of each traffic packet on a preset network port and provides feedback to the traffic execution unit on whether the traffic bandwidth needs to be adjusted and by how much. The traffic execution unit adjusts the traffic bandwidth based on the traffic transmission status monitored in real time by the traffic monitoring unit.

Images