Linux-based network flow analysis and optimization method and system, and electronic equipment
By loading eBPF programs in Linux system to intercept and process network packets at XDP layer, the problem of traditional methods increasing latency and system overhead is solved, and the performance and security improvement of network traffic management is achieved.
Patent Information
- Application Number
- CN202410717081.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-04
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional methods increase latency and system overhead when processing packets in user space, which is not conducive to improving the performance and security of network traffic management.
Loading eBPF programs in Linux system, intercepting, classifying, counting, tagging and modifying network packets at XDP layer through eBPF programs, and forwarding them to different goals for in-depth analysis and optimization, including using eBPF loader, Tools such as bpf_helpers.h header file, libbpf library, etc. define and apply rules and policies, transform data structures, and display data packet information.
Improve the performance and security of network traffic management, and achieve efficient management and optimization of network traffic by reducing latency and system overhead.
Smart Images

Figure CN118972326A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and in particular to a method, system and electronic equipment for analyzing and optimizing network traffic based on Linux. Background Art
[0002] With the rapid growth of network traffic, it is increasingly important to effectively manage and optimize network traffic. With the rapid development of technologies such as cloud computing, big data, and the Internet of Things, the amount and complexity of data carried by the network are also increasing. Therefore, efficient network traffic management and optimization are essential to ensure network security, improve network performance, optimize user experience, and reduce operating costs. Effective network traffic management can help network administrators identify and solve problems such as network congestion, latency, and packet loss, and improve network throughput and stability. At the same time, implementing intelligent scheduling and priority processing for different types of traffic can better meet business needs and improve network resource utilization. In addition, by monitoring and analyzing network traffic, abnormal traffic and security threats can be discovered in a timely manner to ensure network security.
[0003] However, traditional methods increase latency and system overhead when processing data packets in user space, which is not conducive to improving the performance and security of network traffic management. Summary of the invention
[0004] In order to overcome the deficiencies of the prior art, the present invention provides a method for network traffic analysis and optimization based on Linux, which can improve the performance and security of network traffic management.
[0005] The technical solution adopted by the present invention to solve the technical problem is: a method for network traffic analysis and optimization based on Linux, comprising the following steps:
[0006] Step S10: Load the eBPF program in the Linux system, and use the program to operate and intercept the network data packets of the XDP layer;
[0007] Step S20: classify, count, mark and modify the intercepted network data packets according to preset rules and policies;
[0008] Step S30: forwarding the network data packet to different targets according to the type and purpose of the network data packet;
[0009] Step S40: Collect and display relevant information of network data packets in the specified target to achieve in-depth analysis and optimization of network traffic.
[0010] In the above method, in step S10, the eBPF program is loaded into the Linux system through the eBPF program loader.
[0011] In the above method, the rules and policies preset in step S20 include data packet type rules, IP address range rules, port range rules, protocol rules, bandwidth limitation rules and compound rules.
[0012] In the above method, in step S20, the network data packet first needs to be converted into a data structure that can be recognized and processed by the eBPF / XDP program, and then the auxiliary function provided by the bpf_helpers.h header file is used to operate the converted data structure.
[0013] In the above method, the different targets in step S30 include user space, network card and network stack.
[0014] In the above method, the network data packet may also be discarded in step S30.
[0015] In the above method, before displaying the relevant information of the network data packet in step S40, the network data packet needs to be converted into a data structure that can be recognized and processed by a GUI or CLI tool.
[0016] In the above method, the relevant information of the network data packet displayed in step S40 includes the source address, destination address, protocol, port, length, content, label and statistical data.
[0017] A network traffic analysis and optimization system based on Linux, comprising:
[0018] Linux system, used to load eBPF programs on the Linux system;
[0019] eBPF program, used to intercept and manipulate network packets at the XDP layer;
[0020] A network data packet forwarding device, used to forward network data packets to different targets;
[0021] A network data packet collection device, used to collect relevant information of network data packets in a specified target;
[0022] The network data packet display device is used to display relevant information of the network data packet in the user space.
[0023] An electronic device for analyzing and optimizing network traffic based on Linux, comprising: at least one processor and at least one memory, wherein:
[0024] The memory has computer-readable instructions stored thereon;
[0025] The computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the Linux-based network traffic analysis and optimization method as described in any one of the above items.
[0026] The beneficial effect of the present invention is that the method can improve the performance and security of network traffic management. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 This is a step diagram of a method for network traffic analysis and optimization based on Linux of the present invention.
[0028] Figure 2 The present invention is a schematic diagram of the structure of a Linux-based network traffic analysis and optimization system.
[0029] Figure 3 The present invention is a schematic structural diagram of an electronic device for network traffic analysis and optimization based on Linux. DETAILED DESCRIPTION
[0030] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0031] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interchangeably without conflicting with each other.
[0032] Reference Figures 1 to 3 As shown, the present invention discloses a method for analyzing and optimizing network traffic based on Linux. Specifically, the method comprises the following steps:
[0033] Step S10: Load the eBPF program in the Linux system, and use the program to operate and intercept the network data packets of the XDP layer;
[0034] Specifically, first load the eBPF program on the Linux system: use an eBPF program loader, such as bpf-loader, bpftrace, bpftool, libbpf, etc. After the eBPF program is loaded into the Linux system, the network data packets are intercepted and operated at the XDP layer through the eBPF program.
[0035] Use the eBPF loader (bpf-loader, bpftrace, bpftool, or libbpf, etc.) to load the eBPF program into the Linux kernel, select the appropriate loading mode (Native XDP, Offloaded XDP, or Generic XDP), and use the iproute2 tool to bind the eBPF program to the specified network interface.
[0036] More specifically, the eBPF program is written in C language, and uses the auxiliary functions provided by the bpf_helpers.h header file to read, modify and return network data packets, and uses the libbpf library to load the eBPF program into the Linux system. In order to better load the eBPF / XDP program into the network interface of the Linux system, we finally select the appropriate loading mode according to the kernel version of the Linux system, the driver and device of the network interface, and the performance requirements of the eBPF / XDP program, such as Native XDP, Offloaded XDP or Generic XDP. Then use the iproute2 tool to bind the eBPF / XDP program to the specified network interface, and use the iplink command to view and modify the status and properties of the network interface. At the same time, in order to monitor and debug the running status of the eBPF / XDP program, you can use the perf tool to collect and display the performance data of the eBPF / XDP program, such as processing speed, packet loss rate, error rate, etc. You can also use the trace-cmd tool to record and analyze the execution process of the eBPF / XDP program, such as function calls, event triggering, conditional judgment, etc. You can also use the gdb tool to single-step debug the eBPF / XDP program, view and modify variable values, set breakpoints, etc.
[0037] Step S20: Classify, count, mark, and modify the intercepted network data packets according to preset rules and policies.
[0038] Specifically, in this method, one or more sets of rules and policies need to be predefined to guide the eBPF program to classify, count, mark and modify network packets. These rules and policies can be defined in YAML or JSON format, including but not limited to: packet type (HTTP, HTTPS, DNS, ICMP, etc.), source / destination IP address range, source / destination port range, protocol type, priority, bandwidth limit, action (forward to user space, network stack, other network card or discard), etc.
[0039] The preset rules and policies can be referred to as follows, including:
[0040] Packet type rules: allow all HTTP traffic, block all telnet traffic, mark HTTPS traffic as high priority, and limit DNS query packets to less than 1024 bytes;
[0041] IP address range rules: allow all traffic from 192.168.1.0 / 24, drop ICMP traffic from 10.0.0.0 / 8, and forward SSH traffic from 172.16.10.0 / 24 to the VPN server;
[0042] Port range rules: block all traffic with destination port 25 (SMTP), mark traffic with source port 80 (HTTP) as normal-web, and limit the number of concurrent connections with destination port 22 (SSH) to 100;
[0043] Protocol rules: forward TCPSYNC packets to the IPS / IDS system, discard all GRE protocol packets, and lower the priority of IGMP protocol packets;
[0044] Bandwidth restriction rules: limit the total bandwidth of all P2P traffic to 10Mbps, reserve 3Mbps of bandwidth for VoIP traffic, and give video streaming a maximum of 75% of bandwidth resources;
[0045] Composite rule: For HTTP POST packets from 193.164.22.0 / 24, if the size exceeds 2MB, forward them to the security detection device; allocate bandwidth to 480p, 720p, and 1080p at a ratio of 5:3:2 for traffic destined for the Youtube IP range.
[0046] In order to distinguish and manage network packets of different types and purposes, you can use formats such as YAML or JSON to define and configure network traffic rules and policies, such as the type, purpose, priority, restriction, action, etc. of network packets. Use libraries such as libyaml or libjson to parse and load network traffic rules and policies and convert them into data structures that eBPF / XDP programs can recognize and process.
[0047] You can use eBPF maps or other data structures, such as arrays, hash tables, queues, stacks, etc., to store and update the classification, statistics, marking, and modification results of network packets. Use the helper functions provided by the bpf_helpers.h header file, such as bpf_map_lookup_elem, bpf_map_update_elem, bpf_map_delete_elem, etc., to operate on eBPF maps or other data structures, such as reading, writing, deleting, etc.
[0048] In order to use eBPF / XDP programs, classify, count, mark, and modify network packets according to the rules and policies of network traffic. Use the auxiliary functions provided by the bpf_helpers.h header file, such as bpf_skb_load_bytes, bpf_skb_store_bytes, bpf_skb_adjust_room, etc. to read, modify, and extend the content of network packets. Use the auxiliary functions provided by the bpf_helpers.h header file, such as bpf_xdp_adjust_meta, bpf_xdp_adjust_tail, etc. to read, modify, and extend the metadata of network packets.
[0049] In order to use mechanisms such as eBPF maps or perf events, the results of classification, statistics, marking, and modification of network packets can be passed to user space or other targets. Use the auxiliary functions provided by the bpf_helpers.h header file, such as bpf_perf_event_output, bpf_perf_event_read, etc., to operate perf events, such as sending, receiving, etc. Use the auxiliary functions provided by the bpf_helpers.h header file, such as bpf_redirect, bpf_redirect_map, etc., to redirect network packets, such as sending to other network cards, network stacks, or discarding them.
[0050] To achieve data visualization, use the libbpf library to read the relevant information of network packets from eBPF maps or perf events and convert them into data structures that can be recognized and processed by GUI or CLI tools. In user space, use a network packet displayer, such as wireshark, tcpdump, iperf or other supported tools to display relevant information of network packets, such as source address, destination address, protocol, port, length, content, label, statistics, etc. The network packet displayer can use a variety of methods, such as graphical interface, command line, web page, etc., to display and analyze the relevant information of network packets.
[0051] Step S30: The eBPF program forwards the network data packet to different targets, such as user space, network stack, other network cards, or discards it according to the type and purpose of the network data packet.
[0052] The eBPF program can forward network packets that need to be further processed or displayed, such as HTTP, HTTPS, SSH, FTP, DNS, etc., to user space, and use mechanisms such as eBPF maps or perf events to pass relevant information of network packets to user space applications such as wireshark, tcpdump, iperf, etc.
[0053] The eBPF program can forward network packets that need to be processed normally, such as ICMP, to the network stack, allowing the network protocol stack of the Linux system to process them.
[0054] The eBPF program can forward network packets that need to be forwarded, such as VPN, to other network cards, allowing other network cards to send them to the destination.
[0055] The eBPF program can directly discard network packets that need to be discarded, such as spam, attacks, etc., to save resources and improve security. It forwards packets that need to be processed in depth to user space (through perf events or eBPFmaps), forwards packets that need to be processed normally to the network stack, sends packets that need to be forwarded to other network cards, and directly discards specific packets according to rules.
[0056] Step S40: Collect and display relevant information of network data packets in the specified target to achieve in-depth analysis and optimization of network traffic.
[0057] Specifically, mechanisms such as eBPF maps or perf events can be used in specified targets such as user space to collect and display relevant information of network packets, such as source address, destination address, protocol, port, length, content, label, statistics, etc.
[0058] Reference Figure 2 In an embodiment of the present invention, a system for analyzing and optimizing network traffic based on Linux is also provided, and the system comprises:
[0059] Linux system, used to load eBPF programs on the Linux system;
[0060] eBPF program, used to intercept and manipulate network packets at the XDP layer;
[0061] A network data packet forwarding device, used to forward network data packets to different targets;
[0062] A network data packet collection device, used to collect relevant information of network data packets in a specified target;
[0063] The network data packet display device is used to display relevant information of the network data packet in the user space.
[0064] See also Figure 3 In the embodiment of the present invention, there is also provided an electronic device 4000 for analyzing and optimizing network traffic based on Linux, and the electronic device 4000 may include: a desktop computer, a notebook computer, a server, etc. with sensor recognition capability. Figure 3 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003. The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a channel for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0065] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.
[0066] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0067] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to this.
[0068] The memory 4003 stores computer-readable instructions, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .
[0069] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. A method for network traffic analysis and optimization based on Linux, characterized in that: The following steps are involved: step S10: Load the eBPF program in the Linux system and use it to operate and intercept network data packets at the XDP layer; Step S20: classify, count, mark and modify the intercepted network data packets according to preset rules and policies; Step S30: forwarding the network data packet to different targets according to the type and purpose of the network data packet; Step S40: Collect and display relevant information of network data packets in the specified target to achieve in-depth analysis and optimization of network traffic.
2. According to a method for analyzing and optimizing network traffic based on Linux in claim 1, it is characterized in that: In step S10, the eBPF program is loaded into the Linux system through the eBPF program loader.
3. The method for analyzing and optimizing network traffic based on Linux according to claim 1, characterized in that: The rules and policies preset in step S20 include data packet type rules, IP address range rules, port range rules, protocol rules, bandwidth limitation rules and compound rules.
4. The method for analyzing and optimizing network traffic based on Linux according to claim 1, characterized in that: In step S20, the network data packet first needs to be converted into a data structure that can be recognized and processed by the eBPF / XDP program, and then the auxiliary function provided by the bpf_helpers.h header file is used to operate the converted data structure.
5. The method for analyzing and optimizing network traffic based on Linux according to claim 1, characterized in that: The different targets in step S30 include user space, network card and network stack.
6. The method for analyzing and optimizing network traffic based on Linux according to claim 5, characterized in that: In step S30, the network data packet may also be discarded.
7. The method for analyzing and optimizing network traffic based on Linux according to claim 1, characterized in that: Before displaying the relevant information of the network data packet in step S40, the network data packet needs to be converted into a data structure that can be recognized and processed by the GUI or CLI tool.
8. The method for analyzing and optimizing network traffic based on Linux according to claim 1, characterized in that: The relevant information of the network data packet displayed in step S40 includes the source address, destination address, protocol, port, length, content, label and statistical data.
9. A network traffic analysis and optimization system based on Linux, characterized in that: include: Linux system, used to load eBPF programs on the Linux system; eBPF program, used to intercept and manipulate network packets at the XDP layer; A network data packet forwarding device, used to forward network data packets to different targets; A network data packet collection device, used to collect relevant information of network data packets in a specified target; The network data packet display device is used to display relevant information of the network data packet in the user space.
10. An electronic device for network traffic analysis and optimization based on Linux, characterized in that: include: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the Linux-based network traffic analysis and optimization method according to any one of claims 1 to 8.
