A traffic mirroring method, apparatus, and computer readable storage medium

Abstract

The application discloses a flow mirroring method and device and a computer readable storage medium, and the method is applied to a linux operating system kernel and comprises the following steps: obtaining a to-be-mirrored skb data packet; determining the transmission direction and protocol type of the to-be-mirrored skb data packet; when the transmission direction of the to-be-mirrored skb data packet is a receiving direction, directly forwarding the to-be-mirrored skb data packet through a pre-created virtual network card; when the transmission direction of the to-be-mirrored skb data packet is a sending direction and the protocol type is not a tcp protocol, asynchronously forwarding the to-be-mirrored skb data packet through the virtual network card; and when the transmission direction of the to-be-mirrored skb data packet is the sending direction and the protocol type is the tcp protocol, re-allocating a structure body and memory for the to-be-mirrored skb data packet to generate a new skb data packet, and asynchronously forwarding the new skb data packet through the virtual network card.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a traffic mirroring method, apparatus, and computer-readable storage medium. Background Technology

[0002] In Linux systems, existing traffic mirroring solutions mainly fall into two categories, but each has significant performance or compatibility limitations.

[0003] The first method involves using libpcap to configure a BPF filter in user space, capturing data packets, encapsulating them, and then forwarding them. This method has weak performance, high CPU consumption, and cannot completely forward all data packets when dealing with large volumes of traffic.

[0004] The second method involves configuring a traffic control (TC) policy on the network interface card (NIC) and then using a Berkeley packet filter (BPF) to filter the packets before forwarding. This method requires configuring a TC policy on the specified NIC, and this policy is visible to the client environment, which may conflict with existing TC policies in some cases. The BPF filter used by TC requires kernel support; older versions of the Linux kernel do not support BPF filters used by TC.

[0005] There is currently no effective solution to the aforementioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to provide a traffic mirroring method, apparatus, and computer-readable storage medium that enables traffic data packets to be forwarded in kernel mode without performance loss due to user-mode operations, without occupying the network card's TC queue, and without interfering with the original system configuration.

[0007] According to one aspect of the present invention, a traffic mirroring method is provided, applied to the Linux operating system kernel, the method comprising: Obtain the skb data package to be mirrored; Determine the transmission direction and protocol type of the skb data packets to be mirrored; When the transmission direction of the skb data packet to be mirrored is the receiving direction, the skb data packet to be mirrored is directly forwarded through the pre-created virtual network card; When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the skb data packet to be mirrored is asynchronously forwarded through the virtual network card; When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the structure and memory of the skb data packet to be mirrored are reallocated to generate a new skb data packet, and the new skb data packet is asynchronously forwarded through the virtual network card.

[0008] Optionally, obtaining the skb data packet to be mirrored includes: The first kernel thread, which is created in advance, retrieves skb data packets from the socket of the af_packet protocol family that the first kernel thread is bound to, and uses them as the skb data packets to be mirrored. Each service network card is bound to a socket, each socket is bound to a first kernel thread and is configured with a BPF filtering statement, and the skb data packets to be mirrored are the skb data packets that have not been filtered out by the BPF filtering statement.

[0009] Optionally, the step of directly forwarding the skb data packet to be mirrored through a pre-created virtual network interface card when the transmission direction of the skb data packet to be mirrored is the receiving direction, and the step of asynchronously forwarding the skb data packet to be mirrored through the virtual network interface card when the transmission direction of the skb data packet to be mirrored is the sending direction and the protocol type is not TCP, includes: When the transmission direction of the skb data packet to be mirrored is the receiving direction, or when the transmission direction of the skb data packet to be mirrored is the sending direction and the protocol type is not TCP, the first kernel thread sends the skb data packet to be mirrored to the virtual network interface card, and the skb data packet to be mirrored is directly forwarded through the pre-created virtual network interface card.

[0010] Optionally, when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the method of reallocating the structure and memory for the skb data packet to be mirrored to generate a new skb data packet, and asynchronously forwarding the new skb data packet through the virtual network card, includes: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the data payload length of the skb data packet to be mirrored is determined by the first kernel thread. The first kernel thread calculates the total length of the preset tunnel encapsulation header length and the data payload length, and allocates a target memory space with a length equal to the total length; wherein, the target memory space includes a prefix space and a suffix space arranged consecutively, the length of the prefix space being the length of the tunnel encapsulation header, and the length of the suffix space being the length of the data payload; A new skb structure is created by the first kernel thread, the new skb structure is associated with the target memory space, and the data pointer of the new skb structure is configured to point to the starting position of the suffix space. The first kernel thread attempts to copy the data payload of the skb data packet to be mirrored into the suffix space. If the copy is successful, the first kernel thread releases the skb data packet to be imaged and sends the skb data packet corresponding to the new skb structure to the virtual network interface card (NIC), which then forwards the skb data packet directly. If the copy fails, the target memory space and the new skb structure are released through the first kernel thread, and the skb data packet to be imaged is discarded.

[0011] Optionally, obtaining the skb data packet to be mirrored includes: A copy of the skb data packet transmitted by the service network card is obtained through the pre-registered packet_type processing function and used as the initial skb data packet; wherein, a packet_type structure is pre-registered for each service network card, and each packet_type structure is configured with the same packet_type processing function; The preset BPF filtering program is run through the packet_type processing function; When the initial skb data packet is not filtered out by the BPF filtering program, the initial skb data packet is identified as the skb data packet to be mirrored.

[0012] Optionally, when the transmission direction of the skb data packet to be mirrored is the receiving direction, directly forwarding the skb data packet to be mirrored through a pre-created virtual network interface card includes: When the transmission direction of the skb data packet to be mirrored is the receiving direction, the packet_type processing function sends the skb data packet to be mirrored to the virtual network interface card (NIC), and the NIC directly forwards the skb data packet to be mirrored.

[0013] Optionally, when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, asynchronously forwarding the skb data packet to be mirrored through the virtual network interface card includes: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the packet_type processing function stores the skb data packet to be mirrored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the skb data packet to be imaged from the percpu_skb queue and sends the read skb data packet to the virtual network card; The virtual network card directly forwards the skb data packets to be imaged.

[0014] Optionally, when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the method of reallocating the structure and memory for the skb data packet to be mirrored to generate a new skb data packet, and asynchronously forwarding the new skb data packet through the virtual network card, includes: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the skb_copy function is called through the packet_type processing function, and the skb_copy function copies the skb data packet to be mirrored to generate a new skb data packet. The new skb data packet is stored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the new skb data packet from the percpu_skb queue and sends the read new skb data packet to the virtual network card; The new skb data packet is forwarded directly through the virtual network card.

[0015] To achieve the above objectives, the present invention further provides a traffic mirroring device deployed in the Linux operating system kernel, the device comprising: The acquisition module is used to acquire the skb data package to be imaged; The determination module is used to determine the transmission direction and protocol type of the skb data packet to be mirrored; The first mirroring module is used to directly forward the skb data packet to be mirrored through a pre-created virtual network card when the transmission direction of the skb data packet to be mirrored is the receiving direction; The second mirroring module is used to asynchronously forward the skb data packet to be mirrored through the virtual network card when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP. The third mirroring module is used to reallocate the structure and memory of the skb data packet to be mirrored to generate a new skb data packet when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP protocol, and asynchronously forward the new skb data packet through the virtual network card.

[0016] To achieve the above objectives, the present invention also provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the traffic mirroring method described above.

[0017] To achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the steps of the traffic mirroring method described above.

[0018] The traffic mirroring method, apparatus, and computer-readable storage medium provided by this invention complete the traffic mirroring process entirely within the operating system kernel. This process is fast, consumes low CPU and memory resources, has a wide range of compatibility with Linux kernel versions, and does not involve modifying the service network card TC configuration, thus having no impact on the original system configuration. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic diagram of the socket-based traffic mirroring method provided in Example 1; Figure 2 This is a schematic diagram of the packet_type-based traffic mirroring method provided in Example 1; Figure 3 A flowchart of the traffic mirroring method provided in Example 1; Figure 4 A block diagram of the flow mirroring device provided in Embodiment 2; Figure 5 A block diagram of a computer device suitable for implementing the traffic mirroring method, provided in Embodiment 3. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0021] Example 1 Embodiment 1 of the present invention provides a traffic mirroring method, which is applied to the Linux operating system kernel and can be used in network security industry scenarios that require traffic mirroring for analysis, such as intrusion detection, data leakage prevention, and behavioral analysis.

[0022] Before implementing the traffic mirroring method of this application, configuration work needs to be performed in advance. Specifically, the configuration work includes: Main configuration functions of user-mode programs: a. Create a virtual network interface card (NIC) for forwarding, which can be a VXLAN or GRE NIC, and add a tc rate limiting filter to this virtual NIC to control the bandwidth of forwarded traffic. This tc rate limiting filter does not affect the system's existing service NICs; it only applies to the created virtual NIC.

[0023] b. Load the kernel driver.

[0024] c. Compile the necessary BPF filtering rule statements into binary data that can be directly used by the kernel driver. These BPF filtering rules are used to prevent packets from being processed recursively and to filter specific packets that need to be forwarded. The filtering conditions can be flexibly configured by the system user.

[0025] d. Send the configuration to the kernel driver via PROCFS, including the name or index number of the service network interface card (NIC) that needs to perform traffic forwarding, the name or index number of the virtual NIC used for forwarding, and the binary data of the BPF filtering rules to be configured on the service NIC. Not only PROCFS can be used, but other communication mechanisms including but not limited to NETLINK, SYSFS, and character devices can also be used.

[0026] Main configuration functions of kernel drivers: a. Register a network device change event notification interface to release the kernel driver's reference to the virtual network interface card (NIC) when it is deleted. This prevents system malfunctions caused by unexpected events such as a system administrator deleting the virtual NIC, and also ensures that kernel threads can safely use the virtual NIC under RCU (Read-Copy Update) protection even if the virtual NIC is not deleted.

[0027] b. Create a PROCFS communication file to receive configurations sent by user-space programs.

[0028] c. Based on the received configuration, execute the configuration operation corresponding to the first scheme or the configuration operation corresponding to the second scheme. The first and second schemes are specific implementation schemes for traffic mirroring, and the detailed steps are described in the following content.

[0029] Configuration steps corresponding to the first option: Obtain the virtual network interface device used for forwarding, and hold a reference to it, wherein the virtual network interface is protected by RCU for subsequent use.

[0030] Create an af_packet protocol family socket for each business network interface card that needs to forward traffic, and bind it to the corresponding business network interface card.

[0031] Bind the binary data of the BPF filter statement to each socket.

[0032] The socket is read repeatedly until it is empty, to ensure that subsequent data packets read meet the BPF filtering conditions.

[0033] A kernel thread is created for each socket, and this kernel thread executes the first scheme in a loop. For example, Figure 1 As shown, the first scheme can be specifically as follows: Read skb data packets from the socket, where the data packets are filtered by a BPF filter attached to the socket, avoiding recursive processing of data packets and supporting forwarding only specific data packets that need to be forwarded; enable the RCU protection critical section; check whether the virtual network interface card is held and whether it is enabled. If the conditions are not met, release the data packets, remove the RCU protection critical section, and exit the loop to terminate the kernel thread; if the conditions are met, determine the transmission direction of the skb data packets. If it is not an outgoing direction or not a TCP protocol data packet, directly execute the steps of forwarding by the virtual network interface card; if it is an outgoing direction and is a TCP protocol data packet, obtain the data payload length len of the skb data packet and add a certain length space (e.g., 64 bytes, this space is used to reserve tunnel header encapsulation). After that, the `alloc_skb` function is called with the increased length to allocate a new skb (called `nskb`). `skb_copy_bits` is then called to attempt to copy the skb's data payload to an offset of 64 bytes from the `nskb`'s data space. If the copy is successful, other members of the `nskb` structure are filled in as needed, the skb is released, and the `nskb` is used to replace the skb variable. If the copy fails, both the skb and `nskb` are released, the current loop ends, and the system waits to receive the next data packet. The skb data packet is then sent through the virtual network interface card (NIC). During this process, the TC rate limiting filter on the virtual NIC performs traffic shaping and rate limiting on all data packets passing through the virtual NIC to avoid excessive use of the host network bandwidth. It also performs corresponding VXLAN or GRE encapsulation on the data packets before forwarding them to the specified receiving address. Finally, the RCU protection critical section is released.

[0034] The configuration steps for the second option are as follows: Obtain the virtual network interface device used for forwarding, and hold a reference to it, wherein the virtual network interface is protected by RCU for subsequent use.

[0035] Preprocess the BPF filter statement into a BPF program.

[0036] Create a per-CPU skb queue for each CPU.

[0037] Create a percpu thread for each CPU to independently process the corresponding CPU's skb queue.

[0038] A `packet_type` structure is registered for each designated network interface card (NIC) used for traffic mirroring. All `packet_type` structures have the same `packet_type` handler function, which is used to receive all skb data packets from the designated NIC. The process involves executing the following steps through the `packet_type` handler function, percpu threads, and skb queues: Figure 2 The second option is shown.

[0039] like Figure 3 As shown, the traffic mirroring method provided in this embodiment includes steps S1 to S5, wherein: Step S1: Obtain the skb data package to be mirrored.

[0040] Backup the skb data packets transmitted on the service network card, and determine whether the backup needs to be filtered out based on the preset BPF rules. If it does not need to be filtered out, the backup is used as the skb data packets to be mirrored.

[0041] Step S2: Determine the transmission direction and protocol type of the skb data packet to be mirrored.

[0042] The transmission direction is either receiving or sending packets, and the protocol type is either TCP or non-TCP. The transmission direction of the skb data packet to be mirrored is the same as that of the original skb data packet transmitted on the corresponding service network interface card, and the protocol type of the skb data packet to be mirrored is the same as that of the original skb data packet transmitted on the corresponding service network interface card.

[0043] Step S3: When the transmission direction of the skb data packet to be mirrored is the receiving direction, the skb data packet to be mirrored is directly forwarded through the pre-created virtual network card.

[0044] For the receiving direction, regardless of whether the transmission process between the skb data packet to be mirrored and the corresponding service network interface card (NIC) is synchronous or asynchronous, no additional processing is required. The skb data packet to be mirrored is directly handed over to the virtual NIC for forwarding. A TC rate limiting filter is added to the virtual NIC to control the bandwidth of the forwarded traffic. This TC rate limiting filter does not affect the original service NICs of the system; it only applies to the created virtual NIC. During the forwarding of skb data packets through the virtual NIC, the TC rate limiting filter on the virtual NIC performs traffic shaping and rate limiting on all data packets passing through the virtual NIC to avoid excessive consumption of host network bandwidth. It also performs appropriate VXLAN or GRE encapsulation on the data packets and finally forwards them to the designated receiving address.

[0045] Step S4: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the skb data packet to be mirrored is asynchronously forwarded through the virtual network card.

[0046] When skb packets are sent outwards, they need to be processed asynchronously by a thread independent of the original transmission flow of the service network interface card (NIC). If the virtual NIC is called synchronously during the lock holding period of the outward path, and the virtual NIC's sending path may again pass through the original service NIC due to routing or other reasons, it will attempt to acquire the already held lock, leading to a deadlock. The asynchronous forwarding mechanism breaks this circular dependency and ensures system stability. For example, if outward skb packets are not processed asynchronously, the following might happen: When service NIC 1 sends a packet, the kernel acquires a lock and enters the packet sending process. If the skb packet is sent directly to the virtual NIC through the synchronous process, after the virtual NIC encapsulates the skb packet, it may determine through the system routing table that it was sent through service NIC 1. At this point, it will return to the outward sending process of service NIC 1, and then lock service NIC 1 again, resulting in a deadlock. Since asynchronous processing does not involve locking, outward skb packets need to be processed asynchronously to avoid deadlocks.

[0047] Step S5: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the structure and memory of the skb data packet to be mirrored are reallocated to generate a new skb data packet, and the new skb data packet is asynchronously forwarded through the virtual network card.

[0048] The skb packets used for traffic forwarding are backups of the original skb packets transmitted on the service network interface card (NIC). These forwarded skb packets share the same data payload memory as the original skb packets. When the skb packets are outbound and use the TCP protocol, due to a kernel bug, the original TCP packets may modify the shared data payload memory under certain conditions. This modification leads to inconsistencies between the data in the forwarded skb packets and the overall skb structure, resulting in a mismatch between the shared data payload memory and the size recorded in the skb structure. If the skb packets used for traffic mirroring are directly handed over to the virtual NIC for transmission, the system may crash during transmission due to these inconsistencies. Therefore, this embodiment reallocates the skb structure and data memory through a copying method before handing over the skb packets. The copying method can detect the aforementioned inconsistencies and will cause copying failure when such inconsistencies exist, but it will not cause a system crash. Therefore, to avoid system crashes, when the transmission direction of the skb packets to be mirrored is outbound and the protocol type is TCP, the structure and memory of the skb packets to be mirrored are reallocated to generate new skb packets.

[0049] This embodiment applies the traffic mirroring method to the Linux operating system kernel, clarifying the execution context of this method. This is the foundation that distinguishes this solution from traditional low-performance solutions, directly avoiding the costly "kernel-user mode" context switching and data copying required by user-mode (such as libpcap) solutions as described in the background. Since the packet receiving path typically does not involve device transmit lock contention that may exist in the outgoing path within the kernel network protocol stack, the direct forwarding approach is simple in logic, has the lowest latency, and can efficiently handle mirrored traffic flowing to the local machine without introducing additional queue and scheduling overhead. For outgoing asynchronous forwarding processes that are not TCP protocols, the deadlock risk when mirroring outgoing traffic in kernel mode is resolved. For outgoing asynchronous forwarding processes that are TCP protocols, by reallocating structures and memory, an isolated and secure data copy is created for forwarding, completely avoiding the system crash risk caused by shared memory. This embodiment creatively combines high-performance kernel-mode processing, deadlock-avoiding asynchronous mechanisms, and memory isolation to ensure stable TCP processing through the aforementioned precise differentiated processing logic. It achieves the high-performance advantages of kernel-mode forwarding while avoiding intrusion into outgoing paths and potential deadlocks through asynchronous forwarding. Furthermore, it addresses deep-seated memory consistency issues through TCP packet copying operations. Moreover, since this method does not rely on or modify the TCP configuration on the service network interface card, it possesses strong compatibility and deployment flexibility, effectively overcoming the inherent defects of the two existing solutions described in the background section.

[0050] The specific processing flow of the traffic mirroring method corresponding to the first scheme includes the following steps.

[0051] Optionally, obtaining the skb data packet to be mirrored includes: The first kernel thread, which is created in advance, retrieves skb data packets from the socket of the af_packet protocol family that the first kernel thread is bound to, and uses them as the skb data packets to be mirrored. Each service network card is bound to a socket, each socket is bound to a first kernel thread and is configured with a BPF filtering statement, and the skb data packets to be mirrored are the skb data packets that have not been filtered out by the BPF filtering statement.

[0052] Specifically, the `af_packet` socket is an interface provided by the Linux kernel that allows direct reception of link-layer data frames in kernel mode or user mode. In this implementation, during kernel driver initialization, a socket is created for each service network interface card (NIC) that needs to be monitored, and the socket is bound to that NIC. Simultaneously, a dedicated first kernel thread is pre-created for each socket to cyclically read data packets from the queue of its bound socket. The kernel driver pre-attaches a user-space compiled BPF bytecode program to each socket; all data packets are checked by this BPF before entering the socket to determine whether to allow the packet to enter the socket queue or discard it.

[0053] Optionally, the step of directly forwarding the skb data packet to be mirrored through a pre-created virtual network interface card when the transmission direction of the skb data packet to be mirrored is the receiving direction, and the step of asynchronously forwarding the skb data packet to be mirrored through the virtual network interface card when the transmission direction of the skb data packet to be mirrored is the sending direction and the protocol type is not TCP, includes: When the transmission direction of the skb data packet to be mirrored is the receiving direction, or when the transmission direction of the skb data packet to be mirrored is the sending direction and the protocol type is not TCP, the first kernel thread sends the skb data packet to be mirrored to the virtual network interface card, and the skb data packet to be mirrored is directly forwarded through the pre-created virtual network interface card.

[0054] Specifically, direct forwarding refers to the behavior of the first kernel thread, within its own execution context, directly calling the network device interface to submit the skb to the virtual network card driver. Since the skb packet to be mirrored is not processed in its original path where it might hold locks, but is first routed to the socket queue and then asynchronously consumed and processed by an independent first kernel thread, this direct forwarding action itself occurs within a securely asynchronous execution context. It does not interfere with or block the original packet receiving process, thus avoiding the risk of deadlock.

[0055] Optionally, when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the method of reallocating the structure and memory for the skb data packet to be mirrored to generate a new skb data packet, and asynchronously forwarding the new skb data packet through the virtual network card, includes: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the data payload length of the skb data packet to be mirrored is determined by the first kernel thread. The first kernel thread calculates the total length of the preset tunnel encapsulation header length and the data payload length, and allocates a target memory space with a length equal to the total length; wherein, the target memory space includes a prefix space and a suffix space arranged consecutively, the length of the prefix space being the length of the tunnel encapsulation header, and the length of the suffix space being the length of the data payload; A new skb structure is created by the first kernel thread, the new skb structure is associated with the target memory space, and the data pointer of the new skb structure is configured to point to the starting position of the suffix space. The first kernel thread attempts to copy the data payload of the skb data packet to be mirrored into the suffix space. If the copy is successful, the first kernel thread releases the skb data packet to be imaged and sends the skb data packet corresponding to the new skb structure to the virtual network interface card (NIC), which then forwards the skb data packet directly. If the copy fails, the target memory space and the new skb structure are released through the first kernel thread, and the skb data packet to be imaged is discarded.

[0056] Specifically, the target memory space is logically divided into two parts: a prefix space for subsequent tunnel header encapsulation and a suffix space for storing the copied data payload. The data pointer of the new skb structure is configured to point to the beginning of the suffix space. The effective data payload representing this new skb structure will begin from this suffix space, and the prefix space becomes its header space. During the copying attempt, if an anomaly is detected in the original skb data (such as a discrepancy between the actual data length and the structure record, i.e., a memory inconsistency problem), the copy fails; or if a normal resource shortage is detected (such as a memory error), the copy also fails. If the copy fails, as a safety fallback mechanism, all allocated resources are immediately released, and the skb data packet to be mirrored is discarded. If the copy succeeds, the new skb data packet is used to execute the subsequent traffic mirroring process.

[0057] This embodiment creates an independent data copy through a copy operation, completely severing any potential memory sharing between the new image package and the original outgoing TCP package. This fundamentally avoids the risk of system crashes due to modifications to the original package's memory. Furthermore, the copy failure branch provides a safe exit point. When encountering insufficient memory or abnormal data packets, the system chooses to discard the current data packet and release resources instead of risking further processing of potentially crash-causing abnormal data, thus ensuring the overall robustness of the system.

[0058] The first scheme completes the entire forwarding process within the kernel, without going through user space. No TCP configuration is required on the service network interface card (NIC), and the system NIC's existing TCP queue does not need modification. A BPF filter is used to prevent recursion, and filtering conditions can be flexibly configured. Furthermore, a first kernel thread receives data packets from the socket queue, and the queue and the original data packets sent out by the service NIC are processed asynchronously, avoiding kernel deadlock. After receiving the data packet skb, the first kernel thread re-copys the data payload of the outgoing TCP packet and handles copy failures, preventing kernel crashes under certain conditions.

[0059] The specific processing flow of the traffic mirroring method corresponding to the second scheme includes the following steps.

[0060] Optionally, obtaining the skb data packet to be mirrored includes: A copy of the skb data packet transmitted by the service network card is obtained through the pre-registered packet_type processing function and used as the initial skb data packet; wherein, a packet_type structure is pre-registered for each service network card, and each packet_type structure is configured with the same packet_type processing function; The preset BPF filtering program is run through the packet_type processing function; When the initial skb data packet is not filtered out by the BPF filtering program, the initial skb data packet is identified as the skb data packet to be mirrored.

[0061] Specifically, the kernel driver registers a `packet_type` structure for each service network interface card (NIC) that needs to be monitored. All `packet_type` structures point to the same `packet_type` processing function. When a data packet passes through a specified service NIC, the registered `packet_type` processing function is called, passing a copy of the data packet as a parameter. This copy is the initial skb data packet. Further, this initial skb data packet is checked using BPF (Browser-Based Function) analysis. Only initial skb data packets that successfully pass the BPF check are included in the subsequent processing flow as skb data packets to be mirrored. It should be noted that the capture and initial filtering here are completed synchronously within the original execution path of the protocol stack.

[0062] Optionally, when the transmission direction of the skb data packet to be mirrored is the receiving direction, directly forwarding the skb data packet to be mirrored through a pre-created virtual network interface card includes: When the transmission direction of the skb data packet to be mirrored is the receiving direction, the packet_type processing function sends the skb data packet to be mirrored to the virtual network interface card (NIC), and the NIC directly forwards the skb data packet to be mirrored.

[0063] Specifically, there is no need to further restrict whether the packet receiving direction is synchronous or asynchronous processing; it can simply be forwarded directly by the virtual network interface card (NIC) within the current processing flow. In this embodiment, since the packet_type processing function's actions of capturing, filtering, and forwarding data packets in the receiving direction are completed continuously within the same execution context as the data packets transmitted by the service NIC, the data processing flow in the receiving direction is synchronous with the original skb data packets transmitted by the service NIC.

[0064] Optionally, when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, asynchronously forwarding the skb data packet to be mirrored through the virtual network interface card includes: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the packet_type processing function stores the skb data packet to be mirrored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the skb data packet to be imaged from the percpu_skb queue and sends the read skb data packet to the virtual network card; The virtual network card directly forwards the skb data packets to be imaged.

[0065] Specifically, each CPU is bound to a percpu_skb queue and a second kernel thread. Each second kernel thread is used to read skb packets from the corresponding percpu_skb queue and send the read skb packets to the virtual network interface card.

[0066] When the packet_type handler determines in the synchronous context that a packet is outgoing and not a TCP protocol, it does not forward it immediately. Instead, it performs a crucial operation: placing the packet into a percpu_skb queue dedicated to the CPU core currently executing the handler, in order to achieve asynchronous processing of outgoing non-TCP protocol packets.

[0067] Optionally, when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the method of reallocating the structure and memory for the skb data packet to be mirrored to generate a new skb data packet, and asynchronously forwarding the new skb data packet through the virtual network card, includes: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the skb_copy function is called through the packet_type processing function, and the skb_copy function copies the skb data packet to be mirrored to generate a new skb data packet. The new skb data packet is stored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the new skb data packet from the percpu_skb queue and sends the read new skb data packet to the virtual network card; The new skb data packet is forwarded directly through the virtual network card.

[0068] Specifically, copying the skb data packet to be mirrored to generate a new skb data packet using the `skb_copy` function includes: attempting to copy the skb data packet to be mirrored using the `skb_copy` function; if the copy is successful, a new skb data packet is obtained and discarded; if the copy fails, the skb data packet to be mirrored is discarded directly. By calling `skb_copy` to create an independent copy, the memory sharing relationship between the mirrored data packet and the original protocol stack processing path is completely severed, eliminating the potential for system instability caused by memory sharing at its source. After generating the independent copy, the `packet_type` processing function puts the new skb data packet into the `percpu_skb` queue bound to the current CPU, achieving asynchronous processing.

[0069] It should be noted that, because the second approach synchronizes the copying process of the skb data packet to be mirrored with the original skb data packet of the service network card, it means that the skb data packet to be mirrored will not be modified by other processes. Therefore, it does not require the complex copying process of the first approach; it only needs to directly call the skb_copy function to attempt to copy the skb data packet to be mirrored. If the first approach were to call the skb_copy function to attempt to copy the skb data packet to be mirrored, and there were problems with the skb data packet to be mirrored, it would very likely cause a system crash. Therefore, the first approach uses other methods for copying.

[0070] This embodiment achieves two advantages: firstly, it solves the data memory security and consistency problem immediately in the synchronous context by copying first and then enqueuing asynchronously, so that even if the copy fails, it only affects the current data packet and will not endanger the system; secondly, it solves the deadlock problem in the execution context, so that the new copy obtained after successful copying is handed over to an absolutely safe execution environment for asynchronous forwarding.

[0071] The second approach also completes the forwarding process entirely within the kernel, without going through user space. No TCP configuration is required on the service network interface card (NIC), and the system NIC's existing TCP queues do not need modification. A BPF filter is used to prevent recursion, and filtering conditions can be flexibly configured. Furthermore, all outgoing packets, after being received by the `packet_type` processing function, are processed by a second kernel thread via an asynchronous queue, asynchronously with the original packets sent out by the service NIC, avoiding kernel deadlock. The per-CPU thread ensures that forwarded packets and original packets are processed on the same CPU, effectively distributing CPU usage. Outgoing TCP packets undergo skb structure and data memory reallocation before being inserted into the queue, preventing kernel crashes under certain conditions. For incoming packets, after being received by the `packet_type` processing function, they are sent directly to the virtual NIC for forwarding without going through a queue, minimizing CPU resource consumption.

[0072] Example 2 This invention provides a traffic mirroring device deployed in the Linux operating system kernel, such as... Figure 4 As shown, the flow mirroring device 40 specifically includes the following components: Module 401 is used to acquire the skb data packet to be mirrored; The determining module 402 is used to determine the transmission direction and protocol type of the skb data packet to be mirrored; The first mirroring module 403 is used to directly forward the skb data packet to be mirrored through a pre-created virtual network card when the transmission direction of the skb data packet to be mirrored is the receiving direction; The second mirroring module 404 is used to asynchronously forward the skb data packet to be mirrored through the virtual network card when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP. The third mirroring module 405 is used to reallocate the structure and memory of the skb data packet to be mirrored to generate a new skb data packet when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP protocol, and asynchronously forward the new skb data packet through the virtual network card.

[0073] Optionally, the acquisition module is specifically used for: The first kernel thread, which is created in advance, retrieves skb data packets from the socket of the af_packet protocol family that the first kernel thread is bound to, and uses them as the skb data packets to be mirrored. Each service network card is bound to a socket, each socket is bound to a first kernel thread and is configured with a BPF filtering statement, and the skb data packets to be mirrored are the skb data packets that have not been filtered out by the BPF filtering statement.

[0074] Optionally, the first mirror module and the second mirror module are specifically used for: When the transmission direction of the skb data packet to be mirrored is the receiving direction, or when the transmission direction of the skb data packet to be mirrored is the sending direction and the protocol type is not TCP, the first kernel thread sends the skb data packet to be mirrored to the virtual network interface card, and the skb data packet to be mirrored is directly forwarded through the pre-created virtual network interface card.

[0075] Optionally, the third mirror module is specifically used for: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the data payload length of the skb data packet to be mirrored is determined by the first kernel thread. The first kernel thread calculates the total length of the preset tunnel encapsulation header length and the data payload length, and allocates a target memory space with a length equal to the total length; wherein, the target memory space includes a prefix space and a suffix space arranged consecutively, the length of the prefix space being the length of the tunnel encapsulation header, and the length of the suffix space being the length of the data payload; A new skb structure is created by the first kernel thread, the new skb structure is associated with the target memory space, and the data pointer of the new skb structure is configured to point to the starting position of the suffix space. The first kernel thread attempts to copy the data payload of the skb data packet to be mirrored into the suffix space. If the copy is successful, the first kernel thread releases the skb data packet to be imaged and sends the skb data packet corresponding to the new skb structure to the virtual network interface card (NIC), which then forwards the skb data packet directly. If the copy fails, the target memory space and the new skb structure are released through the first kernel thread, and the skb data packet to be imaged is discarded.

[0076] Optionally, the acquisition module is specifically used for: A copy of the skb data packet transmitted by the service network card is obtained through the pre-registered packet_type processing function and used as the initial skb data packet; wherein, a packet_type structure is pre-registered for each service network card, and each packet_type structure is configured with the same packet_type processing function; The preset BPF filtering program is run through the packet_type processing function; When the initial skb data packet is not filtered out by the BPF filtering program, the initial skb data packet is identified as the skb data packet to be mirrored.

[0077] Optionally, the first mirror module is specifically used for: When the transmission direction of the skb data packet to be mirrored is the receiving direction, the packet_type processing function sends the skb data packet to be mirrored to the virtual network interface card (NIC), and the NIC directly forwards the skb data packet to be mirrored.

[0078] Optionally, the second mirror module is specifically used for: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the packet_type processing function stores the skb data packet to be mirrored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the skb data packet to be imaged from the percpu_skb queue and sends the read skb data packet to the virtual network card; The virtual network card directly forwards the skb data packets to be imaged.

[0079] Optionally, the third mirror module is specifically used for: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the skb_copy function is called through the packet_type processing function, and the skb_copy function copies the skb data packet to be mirrored to generate a new skb data packet. The new skb data packet is stored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the new skb data packet from the percpu_skb queue and sends the read new skb data packet to the virtual network card; The new skb data packet is forwarded directly through the virtual network card.

[0080] Example 3 This embodiment also provides a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including a standalone server or a server cluster composed of multiple servers), etc., capable of executing programs. Figure 5 As shown, the computer device 50 in this embodiment includes, but is not limited to, a memory 501 and a processor 502 that are communicatively connected to each other via a system bus. It should be noted that... Figure 5 Only a computer device 50 with components 501-502 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0081] In this embodiment, the memory 501 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 501 may be an internal storage unit of the computer device 50, such as the hard disk or memory of the computer device 50. In other embodiments, the memory 501 may also be an external storage device of the computer device 50, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 50. Of course, the memory 501 may include both the internal storage unit and the external storage device of the computer device 50. In this embodiment, the memory 501 is typically used to store the operating system and various application software installed on the computer device 50. In addition, the memory 501 may also be used to temporarily store various types of data that have been output or will be output.

[0082] In some embodiments, processor 502 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 502 is typically used to control the overall operation of computer device 50.

[0083] Specifically, in this embodiment, the processor 502 is used to execute the program of the traffic mirroring method stored in the memory 501.

[0084] For a detailed description of the above method steps, please refer to Example 1. This example will not be repeated here.

[0085] Example 4 This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the computer program is executed by a processor, it is used to implement the steps of the traffic mirroring method.

[0086] For a detailed description of the above method steps, please refer to Example 1. This example will not be repeated here.

[0087] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0088] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0090] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A traffic mirroring method, characterized in that, Applied to the Linux operating system kernel, the method includes: Obtain the skb data package to be mirrored; Determine the transmission direction and protocol type of the skb data packets to be mirrored; When the transmission direction of the skb data packet to be mirrored is the receiving direction, the skb data packet to be mirrored is directly forwarded through the pre-created virtual network card; When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the skb data packet to be mirrored is asynchronously forwarded through the virtual network card; When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the structure and memory of the skb data packet to be mirrored are reallocated to generate a new skb data packet, and the new skb data packet is asynchronously forwarded through the virtual network card.

2. The traffic mirroring method according to claim 1, characterized in that, The process of obtaining the skb data packet to be mirrored includes: The first kernel thread, which is created in advance, retrieves skb data packets from the socket of the af_packet protocol family that the first kernel thread is bound to, and uses them as the skb data packets to be mirrored. Each service network card is bound to a socket, each socket is bound to a first kernel thread and is configured with a BPF filtering statement, and the skb data packets to be mirrored are the skb data packets that have not been filtered out by the BPF filtering statement.

3. The traffic mirroring method according to claim 2, characterized in that, The step of forwarding the skb data packet to be mirrored directly through a pre-created virtual network interface card (NIC) when the transmission direction of the skb data packet to be mirrored is the receiving direction, and the step of asynchronously forwarding the skb data packet to be mirrored through the virtual NIC when the transmission direction of the skb data packet to be mirrored is the sending direction and the protocol type is not TCP, includes: When the transmission direction of the skb data packet to be mirrored is the receiving direction, or when the transmission direction of the skb data packet to be mirrored is the sending direction and the protocol type is not TCP, the first kernel thread sends the skb data packet to be mirrored to the virtual network interface card, and the skb data packet to be mirrored is directly forwarded through the pre-created virtual network interface card.

4. The traffic mirroring method according to claim 2, characterized in that, When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the structure and memory of the skb data packet to be mirrored are reallocated to generate a new skb data packet, and the new skb data packet is asynchronously forwarded through the virtual network card, including: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the data payload length of the skb data packet to be mirrored is determined by the first kernel thread. The first kernel thread calculates the total length of the preset tunnel encapsulation header length and the data payload length, and allocates a target memory space with a length equal to the total length; wherein, the target memory space includes a prefix space and a suffix space arranged consecutively, the length of the prefix space being the length of the tunnel encapsulation header, and the length of the suffix space being the length of the data payload; A new skb structure is created by the first kernel thread, the new skb structure is associated with the target memory space, and the data pointer of the new skb structure is configured to point to the starting position of the suffix space. The first kernel thread attempts to copy the data payload of the skb data packet to be mirrored into the suffix space. If the copy is successful, the first kernel thread releases the skb data packet to be imaged and sends the skb data packet corresponding to the new skb structure to the virtual network interface card (NIC), which then forwards the skb data packet directly. If the copy fails, the target memory space and the new skb structure are released through the first kernel thread, and the skb data packet to be imaged is discarded.

5. The traffic mirroring method according to claim 1, characterized in that, The process of obtaining the skb data packet to be mirrored includes: A copy of the skb data packet transmitted by the service network card is obtained through the pre-registered packet_type processing function and used as the initial skb data packet; wherein, a packet_type structure is pre-registered for each service network card, and each packet_type structure is configured with the same packet_type processing function; The preset BPF filtering program is run through the packet_type processing function; When the initial skb data packet is not filtered out by the BPF filtering program, the initial skb data packet is identified as the skb data packet to be mirrored.

6. The traffic mirroring method according to claim 5, characterized in that, When the transmission direction of the skb data packet to be mirrored is the receiving direction, the skb data packet to be mirrored is directly forwarded through a pre-created virtual network interface card, including: When the transmission direction of the skb data packet to be mirrored is the receiving direction, the packet_type processing function sends the skb data packet to be mirrored to the virtual network interface card (NIC), and the NIC directly forwards the skb data packet to be mirrored.

7. The traffic mirroring method according to claim 5, characterized in that, When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the skb data packet to be mirrored is asynchronously forwarded through the virtual network card, including: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP, the packet_type processing function stores the skb data packet to be mirrored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the skb data packet to be imaged from the percpu_skb queue and sends the read skb data packet to the virtual network card; The virtual network card directly forwards the skb data packets to be imaged.

8. The traffic mirroring method according to claim 5, characterized in that, When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the structure and memory of the skb data packet to be mirrored are reallocated to generate a new skb data packet, and the new skb data packet is asynchronously forwarded through the virtual network card, including: When the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP, the skb_copy function is called through the packet_type processing function, and the skb_copy function copies the skb data packet to be mirrored to generate a new skb data packet. The new skb data packet is stored in the percpu_skb queue bound to the CPU that executes the packet_type processing function. The second kernel thread bound to the CPU reads the new skb data packet from the percpu_skb queue and sends the read new skb data packet to the virtual network card; The new skb data packet is forwarded directly through the virtual network card.

9. A flow mirroring device, characterized in that, Deployed within the Linux operating system kernel, the device includes: The acquisition module is used to acquire the skb data package to be imaged; The determination module is used to determine the transmission direction and protocol type of the skb data packet to be mirrored; The first mirroring module is used to directly forward the skb data packet to be mirrored through a pre-created virtual network card when the transmission direction of the skb data packet to be mirrored is the receiving direction; The second mirroring module is used to asynchronously forward the skb data packet to be mirrored through the virtual network card when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is not TCP. The third mirroring module is used to reallocate the structure and memory of the skb data packet to be mirrored to generate a new skb data packet when the transmission direction of the skb data packet to be mirrored is outward and the protocol type is TCP protocol, and to asynchronously forward the new skb data packet through the virtual network card.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it is used to implement the steps of the method according to any one of claims 1 to 8.

Images