A real-time video dirty data cleaning method and device enhanced by an operating system kernel

Abstract

This application provides a real-time video dirty data cleaning method and apparatus enhanced with an operating system kernel. The method includes: obtaining a dual-dimensional video stream blacklist based on the binding relationship between the Synchronization Source Identifier (SSRC) of Real-Time Transport Protocol (RTP) packets and the media stream through verification processing by an application-layer streaming media service; obtaining a kernel-level blacklist accessible by an eBPF program based on the video stream blacklist through synchronization processing of a circular buffer; performing packet filtering on data packets in the network interface card (NIC) queue based on the kernel-level blacklist, and performing matching filtering on the data packets through an eBPF program. During the matching filtering process, atomic state flags are used to synchronize the blacklist rules with the timing alignment of data packet processing, and the data packet is discarded or allowed based on the timing alignment result. This application achieves low-latency, low-overhead dirty data cleaning while ensuring the accuracy of filtering.

Description

Technical Field

[0001] This invention belongs to the field of operating system technology, and in particular relates to a method and apparatus for real-time video dirty data cleaning with operating system kernel enhancement. Background Technology

[0002] Existing real-time video streaming and data cleaning solutions generally rely on the application layer for validity verification. Network data packets must undergo end-to-end processing, including network card driver, NAPI polling, kernel network protocol stack, and soft interrupt scheduling, before reaching the user-space streaming media service. During this process, a large number of invalid, dirty data packets continuously consume kernel memory cache and CPU interrupt resources, increasing system load and exacerbating latency and jitter in normal video streaming. To proactively intercept invalid data, existing technologies attempt to synchronize filtering rules to kernel space, such as using eBPF programs to perform pre-filtering of data packets at XDP hook points. However, when application-layer dynamically generated blacklist rules need to be passed across layers to the kernel-space eBPF program, there is a natural asynchronous window between rule synchronization and data packet arrival. In high-speed network environments, this asynchronous window means that the blacklist rules referenced during data packet filtering may not be the latest version, resulting in timing misalignment issues and causing dirty data to be missed or legitimate data streams to be mistakenly blocked. Therefore, in real-time video streaming scenarios, how to ensure the timing consistency between application-layer dynamic rules and kernel-space filtering execution while reducing kernel protocol stack processing overhead has become a pressing technical problem. Summary of the Invention

[0003] The purpose of this application is to provide a real-time video dirty data cleaning method and apparatus with operating system kernel enhancement, so as to achieve early interception of dirty data before it enters the kernel network protocol stack in real-time video streaming scenarios to reduce system resource consumption, and to ensure the timing consistency between the blacklist rules dynamically generated by the application layer and the kernel-mode data packet filtering, so as to avoid inaccurate filtering due to rule synchronization delay.

[0004] This application discloses a real-time video dirty data cleaning method enhanced with an operating system kernel, including: Based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-time Transport Protocol (RTP) packet and the media stream, a two-dimensional video stream blacklist of SSRC and network link is obtained through the verification processing of the application layer streaming media service. Based on the video stream blacklist, a kernel-mode blacklist accessible to the eBPF program is obtained through synchronous processing of the circular buffer. Based on the kernel-mode blacklist, packet filtering based on high-speed data path XDP preprocessing is performed on the packets in the network card queue, and matching filtering is performed on the packets through the eBPF program. In the matching filtering process, the blacklist rules are synchronized with the timing of packet processing through atomic state flags, and the packet is dropped or allowed based on the timing alignment result.

[0005] Optionally, packet filtering based on XDP preprocessing is performed on packets in the network interface card queue according to the kernel-mode blacklist, and matching filtering is performed on the packets through an eBPF program. During the matching filtering process, the blacklist rules are synchronized with the timing of packet processing through atomic state flags. The packet dropping or allowing processing is obtained based on the timing alignment result, including: Based on the raw data packets in the network interface card queue, the data packets to be filtered are obtained through the capture and processing of the eBPF program mounted on the XDP hook point; Based on the data packet to be filtered, the binary features are obtained by parsing the protocol header of the eBPF program; Based on the characteristics of the binary tuple, the packet handling result is obtained through matching processing of the kernel-mode blacklist; Based on the packet handling result, the packet is either dropped or allowed through the execution process of the kernel network subsystem. The timing alignment of blacklist rule synchronization and data packet processing through atomic state markers includes: obtaining atomic state markers through state configuration processing based on the extended fields of the network interface card queue structure; aligning the timing of blacklist rule synchronization and data packet processing through the atomic state markers based on the data packets arriving at the network interface card queue; and completing the matching filtering based on the timing alignment result.

[0006] Optionally, based on the data packet to be filtered, the tuple features are obtained through protocol header parsing processing of the eBPF program, including: Based on the IP header of the data packet to be filtered, the source IP address is obtained through field extraction processing; The source port is obtained by extracting fields from the transport layer header of the data packet to be filtered. The binary feature is obtained by combining the source IP address and source port. Based on the characteristics of the binary tuple, blacklist matching is completed through query processing using the kernel-mode hash storage structure.

[0007] Optionally, the synchronization of blacklist rules and the timing alignment of data packet processing are achieved through the atomic state markers, including: Based on the initial idle state of the atomic state marker, the state is set to either a rule synchronization state or a processing state through atomic operations; If XDP has returned the XDP_DROP instruction during data processing, the data packet will no longer enter the subsequent processing flow of the new application programming interface NAPI. Based on the data packet filtering completion result, the atomic state flag is set to the idle state through the atomic operation reset process.

[0008] Optionally, based on the video stream blacklist, a kernel-mode blacklist accessible to the eBPF program is obtained through synchronous processing of the circular buffer, including: Based on the kernel-mode contiguous physical memory allocation rules, a static fixed-length circular buffer is obtained through memory allocation processing; Based on the entry data of the video stream blacklist, the rules to be synchronized in the circular buffer are obtained through user-mode atomic write processing; Based on the rules to be synchronized, the kernel-mode available blacklist rules are obtained through atomic reading and processing by the eBPF program. Based on the kernel-mode available blacklist rules, the updated kernel-mode blacklist is obtained through storage update processing.

[0009] Optionally, the rules to be synchronized in the circular buffer are obtained through user-space atomic write processing, and the kernel-space available blacklist rules are obtained through eBPF program atomic read processing, including: The current write position is obtained through atomic loading based on the initial write index value of the circular buffer. Data storage is completed by writing blacklist entries based on the current write position. Update the write index using atomic storage processing based on the data storage completion status. The current read position is obtained through atomic loading based on the initial value of the read index of the circular buffer. Based on the current read position, the kernel-state available blacklist rules are obtained through atomic read operations.

[0010] Optionally, based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-time Transport Protocol (RTP) packet and the media stream, a two-dimensional video stream blacklist consisting of the SSRC and the network link is obtained through verification processing by the application-layer streaming media service. This blacklist includes: Based on the received data from the preset network port monitored by the application layer streaming media service, the SSRC is obtained through RTP packet parsing. Based on the SSRC, the binding status is obtained through media stream binding relationship query processing; Based on the binding status, unbound SSRCs and their corresponding network link information are written into the video stream blacklist through entry addition processing; Based on the video stream blacklist, SSRC entries that have been active for a longer than a preset duration are removed through timed scanning.

[0011] Optionally, in the matching filtering process, blacklist rule synchronization and data packet processing timing alignment are achieved through atomic state marking, including: When the atomic state is marked as being in a state of rule synchronization, the arriving data packets are temporarily stored in the NAPI queue; After the atomic state flag is switched to an idle state, the temporary data packet is retrieved from the NAPI queue and the matching filter is executed.

[0012] Optionally, performing packet filtering based on XDP preprocessing on packets in the network interface card queue according to the kernel-mode blacklist, and performing matching filtering on packets through an eBPF program, further includes: When the matching filter returns the XDP_DROP instruction, the kernel directly reclaims the network card DMA buffer occupied by the data packet, and the data packet is not allocated a socket buffer structure and does not enter the kernel protocol stack.

[0013] Another aspect of this application discloses a real-time video dirty data cleaning device with an enhanced operating system kernel, including a blacklist generation unit, a blacklist synchronization unit, and a data packet filtering unit; The blacklist generation unit is used to generate a two-dimensional video stream blacklist based on the binding relationship between the SSRC of the Real-Time Transport Protocol (RTP) packet and the media stream; The blacklist synchronization unit is used to synchronize the video stream blacklist into a kernel-mode blacklist accessible by the eBPF program via a circular buffer. The packet filtering unit is used to perform packet filtering based on XDP preprocessing on packets in the network card queue according to the kernel-mode blacklist, and to perform matching filtering on packets through the eBPF program. During the filtering process, the blacklist rules are synchronized with the timing of packet processing through atomic state flags, and the packet is dropped or allowed based on the timing alignment result.

[0014] The operating system kernel-enhanced real-time video dirty data cleaning method provided in this application first generates a two-dimensional blacklist based on the binding relationship between the SSRC of RTP packets and the media stream by the application-layer streaming media service, refining the legality judgment results at the business level into precise filtering rules. Then, through a circular buffer, these rules are atomically synchronized from user space to a storage structure accessible to the kernel-space eBPF program. The synchronization process is non-blocking and zero-copy, reducing rule update latency to the microsecond level. When a data packet arrives at the network interface card (NIC) queue, the eBPF program mounted on the XDP hook point directly captures the raw data packet, initiating the filtering process before the packet enters the kernel protocol stack. During this process, atomic state flags configured in the NIC queue structure coordinate the rule synchronization state and data processing state: when the atomic state flag indicates that a rule is in synchronization, the eBPF program proactively stores the data packet temporarily in the NAPI queue and waits for synchronization to complete, thus avoiding the use of an incomplete blacklist for filtering; after the rule synchronization is complete and the flag turns to an idle state, the data packet is retrieved from the queue, matched against the latest blacklist, and then discarded or allowed. Thus, the three stages of application layer rule generation, kernel-mode lock-free synchronization, and XDP pre-filtering interact with each other, not only intercepting invalid data packets outside the kernel network protocol stack to eliminate their invalid occupation of the protocol stack and interrupt resources, but also eliminating the risk of rule inconsistency introduced by asynchronous synchronization through the timing alignment mechanism. While achieving low-latency and low-overhead dirty data cleaning, it also ensures the accuracy of filtering, thereby improving the stability and service quality of real-time video stream transmission as a whole. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a real-time video dirty data cleaning method with operating system kernel enhancement provided in an embodiment of this application; Figure 2 This is a flowchart illustrating how, in this embodiment of the application, a dual-dimensional video stream blacklist is obtained by processing the application-layer streaming media service to determine the SSRC and network link based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-time Transport Protocol (RTP) packet and the media stream. Figure 3 This is a flowchart illustrating how, according to an embodiment of this application, a kernel-mode blacklist accessible to an eBPF program is obtained through synchronous processing of a circular buffer based on the video stream blacklist. Figure 4This is a flowchart illustrating how the user-space atomic write process obtains the rules to be synchronized in the circular buffer, and how the eBPF program atomic read process obtains the kernel-space available blacklist rules. Figure 5 This is a sub-flowchart of embodiment S300 of this application; Figure 6 This is a flowchart illustrating how the data packet to be filtered is parsed and processed by the eBPF program to obtain the tuple features according to an embodiment of this application. Figure 7 This is a flowchart illustrating the timing alignment of blacklist rules and data packet processing achieved through the atomic state markers in this embodiment of the application. Figure 8 This is a structural block diagram of the real-time video dirty data cleaning device with operating system kernel enhancement provided in this application embodiment. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to embodiments. The described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] This application provides a real-time video dirty data cleaning method enhanced by the operating system kernel. This method can be applied to real-time video streaming scenarios, such as performing legality verification and cleaning of Real-Time Transport Protocol (RTP) video streams pushed by connected network cameras in a video surveillance platform. This method is completed by the cooperation of application layer streaming media service, kernel-mode lockless ring buffer, extended Berkeley packet filter (eBPF) program mounted on high-speed data path XDP hook point, and kernel network subsystem.

[0019] Please refer to Figure 1 , Figure 1 A flowchart of the real-time video dirty data cleaning method with operating system kernel enhancement provided in this disclosure is shown below. Figure 1 As shown, it includes the following steps: S100, based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-time Transport Protocol (RTP) packet and the media stream, obtains a two-dimensional video stream blacklist of SSRC and network link through the verification processing of the application layer streaming media service; S200, Based on the video stream blacklist, obtain the kernel-mode blacklist accessible by the eBPF program through synchronous processing of the circular buffer; S300, based on the kernel-mode blacklist, performs packet filtering on packets in the network card queue based on high-speed data path XDP preprocessing, and performs matching filtering on packets through the eBPF program. In the matching filtering process, the blacklist rules are synchronized with the timing of packet processing through atomic state flags, and the packet is dropped or allowed based on the timing alignment result.

[0020] The above three steps are executed sequentially, forming a complete processing chain from discovering dirty data to generating rules, synchronizing rules across layers, and then filtering in the kernel early and ensuring rule consistency.

[0021] The specific implementation methods for each step are described in detail below.

[0022] In some embodiments of this application, based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-Time Transport Protocol (RTP) packet and the media stream, a dual-dimensional video stream blacklist consisting of the SSRC and the network link is obtained through verification processing by the application-layer streaming media service. This step is performed by the streaming media service process running in user space. After startup, the streaming media service creates a socket and binds it to a preset network port, which is used to receive RTP video stream data from external devices. The service can simultaneously listen to connection-oriented TCP transmissions and connectionless UDP transmissions to adapt to the access requirements of devices in different network environments. When a data packet arrives at this port, the application layer performs RTP packet parsing processing on the data packet. This parsing process includes verifying whether the version number, padding bits, extension bits, and the CSRC counter field in the RTP fixed header conform to the protocol definition. If the verification fails, the data packet is directly discarded. After successful verification, the SSRC field is read from the RTP packet header. This field is a 32-bit unsigned integer used to uniquely identify the source of a media stream in the RTP session.

[0023] The application layer maintains a binding table between media streams and SSRCs. In a preferred implementation, this table is stored in memory using a hash table structure, with the SSRC as the key and the corresponding media stream object, network connection information, device identifier, and binding timestamp as the value. Whenever a new legitimate media stream is established, the SSRC value negotiated during signaling interaction is registered in this table, forming a binding relationship. After extracting the SSRC from a data packet, the application layer queries the binding table using that SSRC to obtain the binding status. If a valid entry is found, the binding status is "bound," indicating that the data packet belongs to a legitimate video stream, and subsequent media data parsing, format conversion, and distribution proceed according to the normal process. If no entry is found, the binding status is "unbound," indicating that the SSRC is an unnegotiated and illegal identifier. At this point, the application layer constructs a blacklist entry from the SSRC, the source IP address, and the source port of the data packet, and writes it into the video stream blacklist structure. Thus, each entry in the blacklist simultaneously records the illegal SSRC and its corresponding network link information, forming a two-dimensional video stream blacklist based on both SSRC and network link information.

[0024] The two-dimensional blacklist operates on the following logic: When determining if a data packet is dirty, it is considered dirty if its SSRC (SSRC Context Value) matches the SSRC dimension in the blacklist, or if its source IP address and source port match the network link dimension in the blacklist. Only when neither dimension matches is the data packet considered legitimate. This two-dimensional design effectively addresses situations where a single identifier is forged. For example, if an abnormal device forges a legitimate SSRC value, but the packet's source IP address and port are already blacklisted in the network link dimension, the packet will still be considered dirty and discarded.

[0025] To prevent the blacklist from growing indefinitely over time, the application layer initiates a background, scheduled scanning task. This task iterates through all entries in the blacklist at fixed time intervals, subtracting the entry's addition time or most recent hit time from the current system time. If the difference exceeds a preset duration, such as three minutes, the entry is automatically removed from the blacklist. This timeout recycling mechanism keeps the blacklist lightweight while allowing external devices that have resumed normal behavior to re-establish legitimate connections using their original SSRC or IP port after the timeout.

[0026] In some embodiments, such as Figure 2 As shown, based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-time Transport Protocol (RTP) packet and the media stream, a two-dimensional video stream blacklist consisting of the SSRC and the network link is obtained through the verification processing of the application-layer streaming media service. This blacklist includes: S110, based on the received data from the preset network port monitored by the application layer streaming media service, obtains SSRC through RTP packet parsing; S120, Based on the SSRC, the binding status is obtained through media stream binding relationship query processing; S130, based on the binding status, write the unbound SSRC and corresponding network link information into the video stream blacklist through entry addition processing; S140, based on the video stream blacklist, remove SSRC entries with a survival time exceeding a preset duration through timed scanning.

[0027] In practice, for data streams received via TCP, the application layer allocates an independent processing context for each newly established TCP connection. Data in the TCP stream is typically transmitted as RTP packets in frame format. The first two bytes of each frame store the length of subsequent RTP packets in network byte order, followed by the RTP packet content of that length. The application layer cyclically reads these two byte length values ​​and then reads the corresponding length of RTP data for parsing. If no valid RTP packets are successfully read from the TCP connection within a preset time limit, such as ten seconds, the application layer actively closes the TCP connection to release resources. When an illegal SSRC is detected, in addition to adding the corresponding SSRC and network link information to the blacklist, the application layer also actively terminates the TCP connection. For datagrams received via UDP, the application layer directly obtains the RTP packet content and source address information from the receive call, performing RTP parsing and binding status queries. If the same source IP and port repeatedly send packets carrying illegal SSRCs, that network link can be directly added to the blacklist of network links.

[0028] In some embodiments of this application, after the video stream blacklist is generated at the application layer, the rules need to be transmitted to the kernel space through a blacklist synchronization mechanism for use by the eBPF program. Based on the video stream blacklist, a kernel-space blacklist accessible to the eBPF program is obtained through synchronization processing of a circular buffer. This synchronization step uses a static, fixed-length circular buffer located in the kernel space as the data channel. This circular buffer is created based on the kernel-provided contiguous physical memory allocation mechanism, and its length is set to a power of two number of entry units. Each entry unit has a fixed size, sufficient to hold at least one complete blacklist rule, including fields such as SSRC, source IP address, source port, and timestamp. Setting the length to a power of two facilitates the use of bitwise AND operations for index wrapping of read and write positions, avoiding division and modulo operations. This is beneficial for efficient implementation within the limited instruction set of the eBPF program, while ensuring that the boundaries of memory access are always within the range that the verifier can statically infer.

[0029] In this process, the application layer, acting as the producer, maps the physical memory of the circular buffer to the user-space virtual address space through a memory mapping mechanism, thereby gaining direct access to this memory region. When a new blacklist entry needs to be issued, the user-space program first uses an atomic operation to load the current value of the write index from the shared control area of ​​the circular buffer to obtain the current write position. Subsequently, the assembled blacklist entry structure is copied to the data area offset corresponding to the write position according to a predefined fixed-length format. After the copy operation is completed, the write index is updated to the next position through an atomic operation. The entire process relies entirely on atomic instruction coordination, eliminating the risk of blocking or deadlock caused by using locks.

[0030] In this embodiment, the kernel-mode eBPF consumer program is responsible for reading new rules from the circular buffer and being triggered to run at an appropriate time. The triggering time can be an event callback indicating that data in the circular buffer is available, or a periodic check triggered by a timer. The consumer program also uses atomic operations to load the read index, reads a fixed-length blacklist entry from the data area corresponding to the current read index, parses it into a kernel-mode recognizable rule representation, and then calls the eBPF map update helper function to write the entry as a key-value pair into a dedicated eBPF hash table for storing the blacklist, thereby obtaining or updating the kernel-mode blacklist accessible to the eBPF program. After reading is complete, the consumer program atomically updates the read index.

[0031] The advantage of this lock-free ring buffer synchronization mechanism lies in the fact that the entire chain from rule generation in user space to its loading and activation in kernel space is completed within microseconds, without any system calls, context switching, or lock contention. Compared to traditional synchronization methods using netlink messages or sysfs file interfaces, this solution avoids the overhead of task wake-up and message queuing, and is better suited to the high throughput and low latency requirements of real-time video streaming scenarios.

[0032] In some embodiments of this application, such as Figure 3 As shown, based on the video stream blacklist, a kernel-mode blacklist accessible to the eBPF program is obtained through synchronous processing of the circular buffer, including: S210, according to the kernel-mode continuous physical memory allocation rules, obtains a static fixed-length circular buffer through memory allocation processing; S220, Based on the entry data of the video stream blacklist, the synchronization rules in the circular buffer are obtained through user-mode atomic write processing; S230, Based on the rules to be synchronized, the kernel-mode available blacklist rules are obtained by atomic reading and processing through the eBPF program; S240, Based on the kernel-mode available blacklist rules, the updated kernel-mode blacklist is obtained through storage update processing.

[0033] For specific operations on atomic writing and atomic reading, please refer to [link / reference]. Figure 4 ,like Figure 4 As shown, the rules to be synchronized in the circular buffer are obtained through user-space atomic write processing, and the kernel-space available blacklist rules are obtained through eBPF program atomic read processing, including: S221, based on the initial value of the write index of the circular buffer, the current write position is obtained through atomic loading; S222, Data storage is completed by writing blacklist entries according to the current write position; S223, Update the write index through atomic storage processing based on the data storage completion status; S224, Based on the initial value of the read index of the circular buffer, the current read position is obtained through atomic loading; S225, based on the current read position, the kernel-mode available blacklist rules are obtained through atomic read operations. Specifically, the difference between the current read position and the write index position is used to determine whether there is readable data. If so, a fixed-length entry is read from the read index position. After reading, the read index is updated through atomic storage processing. Since the length of the circular buffer is an integer power of two, the corresponding true offset can be obtained by performing bitwise AND mask operations before reading and writing the index. This fixed-boundary access mode meets the memory safety requirements of the eBPF verifier.

[0034] In one alternative implementation, the circular buffer can directly use a kernel-provided map of type BPF_MAP_TYPE_RINGBUF. This map is created with the size of each element and the maximum number of entries specified. It is allocated by the kernel in contiguous physical memory and the data area is automatically mapped to user space. User-mode programs submit blacklisted entries to the buffer using a circular buffer helper function; kernel-mode eBPF consumer programs retrieve entries using the corresponding circular buffer read function. This implementation leverages the kernel's built-in circular buffer management logic, further simplifying the implementation of lock-free synchronization.

[0035] In some embodiments, packet filtering based on XDP preprocessing is performed on packets in the network card queue according to the kernel-mode blacklist, and matching filtering is performed on the packets through the eBPF program. The method further includes: when the matching filtering returns the XDP_DROP instruction, the kernel directly reclaims the network card DMA buffer occupied by the packet, and the packet is not allocated a socket buffer structure and does not enter the kernel protocol stack.

[0036] Once the kernel-mode blacklist is ready, early filtering can be implemented on data packets in the network interface card (NIC) queue. Based on the kernel-mode blacklist, packet filtering based on XDP preprocessing is performed on the data packets in the NIC queue, and matching filtering is performed on the data packets through an eBPF program. During the matching filtering process, atomic state flags are used to synchronize the blacklist rules with the timing alignment of data packet processing. Based on the timing alignment result, the data packet is either dropped or allowed.

[0037] The XDP hook point is located before the network interface card (NIC) driver submits data packets to the kernel network protocol stack, representing the earliest stage of packet processing by the kernel network subsystem. In this embodiment, the compiled eBPF program is mounted to the NIC device's XDP hook via a loading mechanism and runs in native driver mode, allowing direct reading of raw data packets from the NIC's DMA buffer. When a data packet arrives at the NIC, the hardware places it into a pre-allocated kernel buffer via DMA, subsequently triggering NAPI polling. In the NAPI polling callback, the NIC driver passes the data packet to the mounted XDPeBPF program. At this point, the data packet has not yet been allocated a socket buffer sk_buff structure and has not been interpreted by any protocol layer; therefore, discarding dirty data packets here maximizes system resource conservation.

[0038] In some embodiments, please refer to Figure 5 , Figure 5 For example, the sub-flowchart of S300, Figure 5 As shown, packet filtering based on XDP preprocessing is performed on packets in the network interface card queue according to the kernel-mode blacklist, and matching filtering is performed on the packets through the eBPF program. In the matching filtering process, the blacklist rules are synchronized with the timing of packet processing through atomic state flags. The packet dropping or allowing processing is obtained according to the timing alignment result, including: S310 obtains the data packets to be filtered based on the original data packets in the network card queue through the capture and processing of the eBPF program mounted on the XDP hook point; S320, Based on the data packet to be filtered, the binary feature is obtained by parsing the protocol header of the eBPF program; S330, Based on the characteristics of the binary tuple, the packet handling result is obtained through matching processing of the kernel-mode blacklist; S340, based on the data packet handling result, the kernel network subsystem executes the process to discard or allow the data packet; The timing alignment of blacklist rule synchronization and data packet processing through atomic state markers includes: obtaining atomic state markers through state configuration processing based on the extended fields of the network interface card queue structure; aligning the timing of blacklist rule synchronization and data packet processing through the atomic state markers based on the data packets arriving at the network interface card queue; and completing the matching filtering based on the timing alignment result.

[0039] Specifically, after obtaining the packet start and end pointers from the input parameters, the eBPF program first determines the network layer protocol type based on the protocol type field in the link layer header, and then locates the IP header and transport layer header layer by layer. For IPv4 packets, it reads the 4-byte source IP address 12 bytes from the IP header offset; for IPv6 packets, it reads the 16-byte source IPv6 address 8 bytes from the offset. Next, it identifies the transport layer protocol based on the protocol field in the IP header; for example, protocol number 17 represents UDP, and protocol number 6 represents TCP. After skipping the optional fields and length of the IP header, it locates the beginning of the transport layer header and reads the 2-byte source port byte 0 bytes from the offset. The read source IP address and source port are combined into a binary feature value, which is used as the keyword for blacklist lookup.

[0040] After obtaining the binary tuple characteristic, the eBPF program uses this characteristic as the key to call the eBPF hash table lookup helper function to search the kernel-mode blacklist hash table maintained by the consumer program. If a match is found and the entry has not expired, the data packet is determined to be dirty data, and the result is to discard it; the eBPF program returns the XDP_DROP instruction. If no match is found, the result is to allow the packet, and the XDP_PASS instruction is returned. After returning the XDP_DROP instruction, the network card driver directly reclaims the DMA buffer occupied by the data packet, does not allocate an sk_buff structure, and the data packet does not enter the kernel network protocol stack. Upon returning XDP_PASS, the driver allocates an sk_buff and sends the data packet into the NAPI subsequent processing flow and protocol stack.

[0041] In some embodiments of this application, such as Figure 6 As shown, based on the data packet to be filtered, the tuple features are obtained through protocol header parsing processing by the eBPF program, including: S321, Based on the IP header of the data packet to be filtered, the source IP address is obtained through field extraction processing; S322, Based on the transport layer header of the data packet to be filtered, the source port is obtained through field extraction processing; S323, Based on the source IP address and source port, the binary feature is obtained through combination processing; S324, Based on the characteristics of the binary tuple, the blacklist matching is completed through query processing of the kernel-mode hash storage structure.

[0042] In practice, the byte sequence of the source IP address and two bytes of the source port can be concatenated to form a fixed-length byte array as the hash key. The key structure remains consistent between the eBPF program and the data update side. Since eBPF hash table lookups have constant time complexity, this matching process has minimal impact on packet throughput.

[0043] In actual operation, blacklist rules are dynamically updated via a lock-free circular buffer, with user-space writing and kernel-space consumption occurring asynchronously and in parallel. At the point when the XDP eBPF program executes filtering, a new blacklist entry may have been written to the circular buffer by the user but not yet loaded into the hash table by the consumer program. In this case, the kernel-space blacklist retrieved by the eBPF program may not be the latest version, potentially allowing dirty packets that should have been intercepted to pass through. This timing misalignment between rules and processing caused by asynchronous synchronization impacts filtering quality in high-throughput scenarios.

[0044] To address this issue, this application extends atomic state flags at the network interface card (NIC) queue level, using these flags to achieve rule synchronization and timing alignment of packet processing.

[0045] In some embodiments of this application, an atomic state flag is obtained through state configuration processing based on the extended fields of the network interface card queue structure; based on the data packets arriving at the network interface card queue, the timing alignment of blacklist rules with data packet processing is achieved through the atomic state flag; based on the timing alignment result, data packet filtering is completed through the eBPF program; based on the data packet filtering completion status, the queue is restored to an idle state through the release processing of the atomic state flag.

[0046] In some embodiments, the atomic state flag can be implemented as an integer atomic variable. Each network interface card (NIC) queue can independently maintain its own atomic state flag, which is stored in a kernel data structure associated with the queue. As another more portable implementation, the atomic state flag can also exist as a single-eBPF array map, which can be accessed by both the application layer and the XDP eBPF program. The application layer modifies the state value using `bpf_map_update_elem`, and the eBPF program reads the state value using `bpf_map_lookup_elem`, thus completing state transfer without modifying the internal kernel data structures.

[0047] An atomic state flag has at least two state values: idle and rule synchronization in progress. The idle state indicates that no rule synchronization operation is currently in progress, the kernel-mode blacklist rules are in a stable and consistent state, and filtering can be performed directly. The rule synchronization in progress state indicates that blacklist rules are currently being synchronized from user space to the eBPF hash table via the circular buffer; at this time, the contents of the hash table may not yet reflect the latest issued rules. Before rule synchronization begins, the application layer or management component changes the flag from the idle state to the rule synchronization in progress state through an atomic operation. After rule synchronization is complete, i.e., after the consumer program has read all relevant entries in the circular buffer and updated the hash table, the flag is reset to the idle state through an atomic operation.

[0048] Before entering the filtering logic, the XDP eBPF program performs an atomic load read on the atomic state flag. If the read value is in the idle state, it indicates that the current rule is stable, and the program continues to execute the aforementioned tuple extraction, hash lookup, and decision. If the read value is in the rule synchronization state, it means that the hash table may differ from the rules to be consumed in the circular buffer, and a decision should not be made directly based on the current hash table. In this case, the eBPF program adopts a delay strategy.

[0049] In some embodiments of this application, during the matching filtering process, the timing alignment of blacklist rule synchronization and data packet processing is achieved through atomic state markers, including: when the atomic state marker is in the rule synchronization state, the arriving data packets are temporarily stored in the NAPI queue; after the atomic state marker changes to the idle state, the temporarily stored data packets are retrieved from the NAPI queue and the matching filtering is performed.

[0050] In this process, when the atomic state is marked as "rule synchronization in progress," arriving data packets are temporarily stored in the NAPI queue. Once the atomic state changes to the idle state, the temporarily stored data packets are retrieved from this queue and matching filtering is performed. Temporary storage can be implemented in several ways. One method is that the XDP eBPF program returns XDP_PASS for data packets arriving in the "rule synchronization in progress" state, while simultaneously setting a pending flag in the associated metadata or sk_buff flag field of the data packet. This flag is then passed to another eBPF program at the flow control (TC) entry layer to query the blacklist hash table again. When the state returns to idle, the TC layer eBPF program recognizes the pending flag, re-executes the hash query, and decides whether to discard or allow the packet. Another method is that the XDP eBPF program redirects the data packet to a dedicated temporary CPU queue, and after the state returns to idle, the queue consumer program injects the data packet back into the filtering path. Regardless of the temporary storage strategy used, the core principle is that the final decision on the data packet is postponed until rule synchronization is complete, thus ensuring that the filtering is based on a complete and consistent kernel-level blacklist.

[0051] In some embodiments of this application, such as Figure 7 As shown, the timing alignment of blacklist rule synchronization and data packet processing is achieved through the atomic state flags, including: S351, based on the initial idle state of the atomic state marker, the state is set to the rule synchronization state or the processing state through atomic operations; S352, If XDP has returned the XDP_DROP instruction during data processing, the data packet will no longer enter the subsequent processing flow of the new application programming interface NAPI. S353, based on the data packet filtering completion result, the atomic state flag is set to the idle state through the atomic operation reset process.

[0052] The state transition process for timing alignment via atomic state flags further includes: setting the state to either "Rule Synchronization in Progress" or "Processing in Progress" through atomic operations based on the initial idle state of the atomic state flag; if XDP has returned an XDP_DROP instruction, the data packet will no longer enter the subsequent NAPI processing flow; and resetting the atomic state flag to the idle state through atomic operations based on the data packet filtering completion result. In the simplified scheme using a single busy / idle flag, the "Rule Synchronization in Progress" and "Processing in Progress" states can be merged into a single busy state, in which new data packets are temporarily deferred. When using a finer-grained state division, the "Processing in Progress" state is used to indicate that the queue is being exclusively processed by the filtering program to prevent concurrent processing flows from interfering with the atomicity of the state transition.

[0053] In some embodiments of this application, when the matching filter returns an XDP_DROP instruction, the kernel directly reclaims the network card DMA buffer occupied by the data packet. The data packet is not allocated a socket buffer sk_buff structure and is not entered into the kernel protocol stack. This ensures that the resource consumption of data packets judged as dirty data in the entire system is minimized, and they only occupy the network card hardware and DMA buffer at a brief stage.

[0054] During the research and development process, the inventors noticed that combining application-layer dynamic blacklists with kernel XDP filtering schemes presents technical challenges not fully revealed by existing technologies. In conventional XDP filtering schemes, filtering rules are typically pre-configured by administrators with very low update frequency; occasional brief inconsistencies during updates have little impact on the overall filtering results. However, in the context of real-time video stream dirty data cleaning, since abnormal streams can occur at any time, the application layer must immediately add newly discovered dirty data sources to the blacklist and expect immediate interception in the kernel. To address this, this application uses a lock-free circular buffer to compress synchronization latency to the microsecond level. However, further analysis revealed that even with extremely short synchronization paths, the parallel nature of asynchronous writing and consumption still determines a physically unavoidable inconsistency window. Under 10 Gigabit or higher network conditions, a considerable number of data packets can still flood into this window, potentially including dirty data packets that need to be intercepted. Simply increasing the synchronization speed cannot fundamentally close this window. Based on this understanding, this application introduces an atomic state marking mechanism, placing it on the processing path of the network interface card (NIC) queue. This allows the XDP filtering program to check whether a rule update is in progress before executing rule-related decisions. If so, it proactively postpones the decision, logically avoiding erroneous allowances based on outdated rules during rule changes. This technical solution is not a conventional supplement to XDP filtering technology, but rather a timing coordination method specifically designed to address the problem of instantaneous rule inconsistencies in the specific architecture of cross-layer asynchronous collaboration. In existing XDP technical documentation, the common approach is to emphasize statelessness and one-time processing, avoiding the maintenance of state and cross-event synchronization on the data path. However, this application introduces a very lightweight state awareness on the data path to cope with the special fluctuation characteristics of application-layer dynamic rules. This design approach is not easily conceived in the current technological environment.

[0055] To enable those skilled in the art to better understand the overall solution of this application, a comprehensive application example is provided below. In a large-scale video surveillance system, thousands of network cameras establish sessions with a streaming media server via signaling protocols and negotiate the SSRC of RTP streams. After negotiation, the cameras continuously push RTP video data packets. The application layer service of the streaming media server listens on a preset port, extracts the SSRC for each RTP packet, and queries the binding relationship table. When a camera starts sending an unnegotiated SSRC due to a firmware anomaly, the application layer detects the unbound state and immediately generates a blacklist entry containing the illegal SSRC, the camera's source IP, and port. The application layer sets the atomic state flag corresponding to the network card queue to the rule synchronization state through atomic operations, and then puts the entry into the lockless circular buffer through atomic write. The kernel-mode eBPF consumer program is awakened by the circular buffer event, reads the entry, calls bpf_map_update_elem to add it to the blacklist hash table, and then resets the atomic state flag to idle. Within a window of tens of microseconds when the state is marked as "rule synchronization," the XDP eBPF program employs a temporary storage strategy for arriving packets, such as returning XDP_PASS and marking them as pending. The TC layer eBPF program then re-examines these packets once the state returns to idle. All packets arriving after this window are filtered directly by the XDP eBPF program based on the latest hash table, and packets matching the rules are discarded with XDP_DROP. Thus, non-negotiated abnormal video streams are continuously intercepted before entering the kernel protocol stack, ensuring that legitimate video streams maintain stable transmission.

[0056] Several parallel implementation methods are provided below to demonstrate the applicability of the technical solution of this application in different scenarios.

[0057] In one parallel embodiment of this application, a more conservative approach is adopted for the temporary storage strategy of atomic state markers. When the XDP eBPF program detects that the atomic state marker is in the rule synchronization state, it no longer returns XDP_PASS to temporarily store the data packet, but directly returns the XDP_DROP instruction to discard the current data packet. Although this method will discard a small number of potentially legitimate data packets within the synchronization window, it ensures that no dirty data packet can pass through XDP filtering due to rule inconsistency. This strategy is suitable for scenarios with strict security requirements and the ability to tolerate brief packet loss, such as high-security security monitoring systems. In this embodiment, since secondary checks at the TC layer are not required, the system structure is also relatively simplified.

[0058] In another parallel embodiment of this application, the atomic state flag is maintained by a user-space program through an independent management thread and is not bound to a single circular buffer write operation. The management thread periodically checks whether there are new blacklist entries to be issued. If so, it sets the state flag to rule synchronization in batches, writes a batch of entries to the circular buffer, and restores the state flag after the consumer completes the update. This batch synchronization method can reduce the frequency of state switching, reduce the pressure on the cache consistency protocol caused by frequent atomic operations, and is suitable for periods when blacklist updates are relatively frequent.

[0059] In another parallel embodiment of this application, the two-dimensional matching logic for the video stream blacklist is stored in two hash tables within the kernel-mode eBPF hash table. The first hash table uses SSRC as the key, and the second hash table uses a tuple of source IP and source port as the key. During filtering, the XDP eBPF program queries both hash tables sequentially; if a match is found in either table, the data is determined to be dirty. Updates to both hash tables are synchronized through a circular buffer and share the same atomic state flag. This partitioned storage method facilitates setting different expiration policies for different dimensions; for example, the timeout for the SSRC dimension can be set to three minutes, and the timeout for the network link dimension can be set to five minutes, thus more flexibly adapting to the characteristics of dirty data from different sources.

[0060] In one embodiment of this application, regarding the specific implementation of the eBPF program satisfying the verifier's security constraints, static boundary checks are added to all packet parsing paths. For example, before reading the version field and source IP address in the IP header, it is checked whether the current pointer plus the Ethernet header length and the required offset of the IP header are strictly less than the end-of-packet pointer; before reading the transport layer source port, the protocol field in the IP header is checked to confirm the transport layer protocol type, and the complete length of the UDP or TCP header is verified to be within the packet's range. The read and write indices of the circular buffer are both wrapped around using bitwise AND operations modulo two, ensuring that every memory access in the eBPF program is within boundaries that can be determined at compile time. These measures enable the eBPF program to include the necessary filtering and status checking logic while successfully passing the kernel verifier's security checks.

[0061] This application also provides a real-time video dirty data cleaning device with an enhanced operating system kernel, such as... Figure 8 As shown, the device includes a blacklist generation unit 10, a blacklist synchronization unit 20, and a data packet filtering unit 30.

[0062] The blacklist generation unit 10 is used to generate a two-dimensional video stream blacklist based on the SSRC and network link binding relationship of the Real-Time Transport Protocol (RTP) packets. In specific implementations, this unit can be integrated as a software module into the streaming media service program, responsible for functions such as listening to network ports, performing RTP parsing, querying and maintaining the binding relationship table, generating blacklist entries, and running background periodic scans to remove expired entries.

[0063] The blacklist synchronization unit 20 is used to synchronize the video stream blacklist into a kernel-mode blacklist accessible by the eBPF program via a circular buffer. This unit includes a user-mode circular buffer write routine and a kernel-mode eBPF consumer program. The former is responsible for writing blacklist entries to the circular buffer through atomic operations, while the latter is responsible for reading entries from the circular buffer and updating the eBPF blacklist hash table through atomic operations. At the same time, the two jointly manage atomic state flags.

[0064] The packet filtering unit 30 is used to perform XDP-based preprocessing-based packet filtering on packets in the network interface card (NIC) queue according to the kernel-mode blacklist, and to perform matching filtering on packets through an eBPF program. During the filtering process, atomic state flags are used to synchronize the blacklist rules with the timing alignment of packet processing. Based on the timing alignment result, the packet is either dropped or allowed. This unit is implemented as an eBPF program mounted on the NIC XDP hook. When implementing a temporary storage policy, it may also include an auxiliary eBPF program mounted on the TC entry hook, as well as an eBPF map for storing atomic states.

[0065] When the device starts up, the blacklist synchronization unit 20 initializes the circular buffer and status flags, the packet filtering unit 30 loads the eBPF program and binds it to the designated network interface card, and the blacklist generation unit 10 begins listening to and processing inbound video data. During operation, the blacklist generation unit 10 generates blacklist entries, the blacklist synchronization unit 20 is responsible for switching status flags and transferring entries across layers, and the packet filtering unit 30 performs filtering on the data plane according to the current status and the latest rules. The three work together to achieve efficient cleaning of dirty data.

[0066] The blacklist generation unit 10, blacklist synchronization unit 20, and packet filtering unit 30 in the aforementioned device can all be implemented in software, for example, as user-mode processes, kernel eBPF bytecode, and a combination of kernel eBPF bytecode, respectively. They exchange data through the shared memory mechanism and circular buffer provided by eBPF maps. Furthermore, some logic in the packet filtering unit can also be offloaded by programmable hardware as needed, for example, filtering can be performed directly on a smart network interface card that supports XDP hardware offloading. The technical solution of this application is also applicable to this deployment method.

[0067] The specific embodiments of this application have been described in detail above. These embodiments are used to illustrate the technical solutions of this application, and not to limit its scope of protection. Those skilled in the art will understand that various improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A real-time video dirty data cleaning method enhanced with operating system kernel, characterized in that, include: Based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-time Transport Protocol (RTP) packet and the media stream, a two-dimensional video stream blacklist of SSRC and network link is obtained through the verification processing of the application layer streaming media service. Based on the video stream blacklist, a kernel-mode blacklist accessible to the eBPF program is obtained through synchronous processing of the circular buffer. Based on the kernel-mode blacklist, packet filtering based on high-speed data path XDP preprocessing is performed on the packets in the network card queue, and matching filtering is performed on the packets through the eBPF program. In the matching filtering process, the blacklist rules are synchronized with the timing of packet processing through atomic state flags, and the packet is dropped or allowed based on the timing alignment result.

2. The method according to claim 1, characterized in that, Based on the kernel-mode blacklist, packet filtering based on XDP preprocessing is performed on packets in the network interface card queue, and matching filtering is performed on the packets through an eBPF program. During the matching filtering process, atomic state flags are used to synchronize the blacklist rules with the timing alignment of packet processing. Based on the timing alignment result, packet dropping or allowing is determined, including: Based on the raw data packets in the network interface card queue, the data packets to be filtered are obtained through the capture and processing of the eBPF program mounted on the XDP hook point; Based on the data packet to be filtered, the binary features are obtained by parsing the protocol header of the eBPF program; Based on the characteristics of the binary tuple, the packet handling result is obtained through matching processing of the kernel-mode blacklist; Based on the packet handling result, the packet is either dropped or allowed through the execution process of the kernel network subsystem. The timing alignment of blacklist rule synchronization and data packet processing through atomic state markers includes: obtaining atomic state markers through state configuration processing based on the extended fields of the network interface card queue structure; aligning the timing of blacklist rule synchronization and data packet processing through the atomic state markers based on the data packets arriving at the network interface card queue; and completing the matching filtering based on the timing alignment result.

3. The method according to claim 2, characterized in that, Based on the data packet to be filtered, the binary features are obtained through protocol header parsing processing by the eBPF program, including: Based on the IP header of the data packet to be filtered, the source IP address is obtained through field extraction processing; The source port is obtained by extracting fields from the transport layer header of the data packet to be filtered. The binary feature is obtained by combining the source IP address and source port. Based on the characteristics of the binary tuple, blacklist matching is completed through query processing using the kernel-mode hash storage structure.

4. The method according to claim 2, characterized in that, The synchronization of blacklist rules and the timing alignment of data packet processing are achieved through the atomic state flags, including: Based on the initial idle state of the atomic state marker, the state is set to either a rule synchronization state or a processing state through atomic operations; If XDP has returned the XDP_DROP instruction during data processing, the data packet will no longer enter the subsequent processing flow of the new application programming interface NAPI. Based on the data packet filtering completion result, the atomic state flag is set to the idle state through the atomic operation reset process.

5. The method according to claim 1, characterized in that, Based on the video stream blacklist, a kernel-mode blacklist accessible to eBPF programs is obtained through synchronous processing of the circular buffer, including: Based on the kernel-mode contiguous physical memory allocation rules, a static fixed-length circular buffer is obtained through memory allocation processing; Based on the entry data of the video stream blacklist, the rules to be synchronized in the circular buffer are obtained through user-mode atomic write processing; Based on the rules to be synchronized, the kernel-mode available blacklist rules are obtained through atomic reading and processing by the eBPF program. Based on the kernel-mode available blacklist rules, the updated kernel-mode blacklist is obtained through storage update processing.

6. The method according to claim 5, characterized in that, The rules to be synchronized in the circular buffer are obtained through user-space atomic write processing, and the kernel-space available blacklist rules are obtained through eBPF program atomic read processing, including: The current write position is obtained through atomic loading based on the initial write index value of the circular buffer. Data storage is completed by writing blacklist entries based on the current write position. Update the write index using atomic storage processing based on the data storage completion status. The current read position is obtained through atomic loading based on the initial value of the read index of the circular buffer. Based on the current read position, the kernel-state available blacklist rules are obtained through atomic read operations.

7. The method according to claim 1, characterized in that, Based on the binding relationship between the Synchronization Source Identifier (SSRC) of the Real-time Transport Protocol (RTP) packet and the media stream, a two-dimensional video stream blacklist is obtained through the verification processing of the application-layer streaming media service, consisting of both the SSRC and the network link. This blacklist includes: Based on the received data from the preset network port monitored by the application layer streaming media service, the SSRC is obtained through RTP packet parsing. Based on the SSRC, the binding status is obtained through media stream binding relationship query processing; Based on the binding status, unbound SSRCs and their corresponding network link information are written into the video stream blacklist through entry addition processing; Based on the video stream blacklist, SSRC entries that have been active for a longer than a preset duration are removed through timed scanning.

8. The method according to claim 1, characterized in that, In the matching and filtering process, blacklist rules are synchronized and the timing of data packet processing is aligned through atomic state flags, including: When the atomic state is marked as being in a state of rule synchronization, the arriving data packets are temporarily stored in the NAPI queue; After the atomic state flag is switched to an idle state, the temporary data packet is retrieved from the NAPI queue and the matching filter is executed.

9. The method according to claim 1, characterized in that, Based on the kernel-mode blacklist, packet filtering based on XDP preprocessing is performed on packets in the network interface card queue, and matching filtering is performed on packets through the eBPF program. The process also includes: When the matching filter returns the XDP_DROP instruction, the kernel directly reclaims the network card DMA buffer occupied by the data packet, and the data packet is not allocated a socket buffer structure and does not enter the kernel protocol stack.

10. A real-time video dirty data cleaning device with enhanced operating system kernel, characterized in that, It includes a blacklist generation unit, a blacklist synchronization unit, and a data packet filtering unit; The blacklist generation unit is used to generate a two-dimensional video stream blacklist based on the binding relationship between the SSRC of the Real-Time Transport Protocol (RTP) packet and the media stream; The blacklist synchronization unit is used to synchronize the video stream blacklist into a kernel-mode blacklist accessible by the eBPF program via a circular buffer. The packet filtering unit is used to perform packet filtering based on XDP preprocessing on packets in the network card queue according to the kernel-mode blacklist, and to perform matching filtering on packets through the eBPF program. During the filtering process, the blacklist rules are synchronized with the timing of packet processing through atomic state flags, and the packet is dropped or allowed based on the timing alignment result.

Images