Neural network data packet classification method and system based on tuple structure and hardware acceleration

Abstract

The invention provides a tuple structure and hardware acceleration neural network data packet classification method and system, and the method comprises the steps: reconstructing a data packet classification problem into a high-dimensional multi-class classification problem, and generating a unique tuple index for each data packet through a rule priority mechanism; constructing a semi-structured model based on the unique tuple index; the semi-structured model comprises a neural network classification model and a tuple space search structure; and performing classification on the CPU-GPU hybrid streaming framework, predicting a tuple index by the GPU based on a neural network classification model, calling a tuple space search structure by the CPU based on the index to perform accurate search, and outputting a data packet classification result. According to the method, complete coverage of a large-scale rule set is achieved through a single model, rule copying is not needed, the model complexity is remarkably reduced through TSS middleware, the hardware potential is fully released through CPU-GPU assembly line collaborative design, and therefore the classification throughput and stability are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, specifically to a tuple-structured and hardware-accelerated neural network packet classification method and system. Background Technology

[0002] Packet classification is a core function of modern network systems (such as routers, firewalls, and intrusion detection systems). It examines and classifies network packets according to a predefined set of rules (including source / destination IP, source / destination port, protocol, etc.) to perform corresponding actions (such as forwarding, dropping, and QoS marking). With the explosive growth of network bandwidth (reaching 400G / 800G) and service complexity, packet classification faces the dual challenges of rule set size (reaching millions) and processing speed.

[0003] Traditional software-based packet classification algorithms are mainly divided into two categories: 1. Decision Tree Methods (such as HiCuts, HyperCuts, CutSplit): These methods construct decision trees by recursively partitioning a multi-dimensional rule space. They are fast because they typically only require traversing one path during lookup. However, their core drawback lies in "rule copying." When a rule definition spans multiple partitioned regions, the rule must be copied to all the leaf nodes it covers. This not only leads to a dramatic increase in memory consumption but also makes rule updates extremely difficult and slow, as updating a rule may require traversing and modifying multiple positions in the tree, and may even trigger subtree reconstruction.

[0004] 2. Hash table methods (such as TSS, PSTSS, DBTable): These methods group rules into different "tuples" based on specific fields of the rules (such as prefix length) and build a hash table for each tuple. These methods naturally support efficient rule updates (O(1) time). However, their lookup performance is unstable. A packet may potentially match rules in multiple tuples, and in the worst case, the classifier may need to probe the hash tables of multiple tuples, leading to unpredictable lookup latency, and hash collisions further degrade performance.

[0005] In recent years, machine learning-based methods (especially deep learning) have been introduced into the field of packet classification, known as "learned indexes." These methods (such as NuevoMatch and NeuTree) are typically based on the Recursive Model Indexing (RMI) framework. RMI treats the indexing problem as a regression problem, that is, learning a model (or a hierarchical structure of models) to predict the position of a key in an ordered dataset.

[0006] However, existing learning-based methods have the following main drawbacks when applied to packet classification: 1. Poor handling of overlapping rules: Network rule sets naturally have a large amount of overlap, and a single data packet can match multiple rules simultaneously (ultimately determined by priority). This conflicts with the RMI regression model's assumption that "one key corresponds to one unique location".

[0007] To circumvent this problem, oNuevoMatch isolates all overlapping rules into a "ReminderSet" and builds an RMI model only for non-overlapping rule subsets. This results in a significant drop in model coverage (below 60% in some rule sets). When a packet hits the ReminderSet, the system performance degrades to executing a slow, traditional algorithm, leading to highly unstable and unpredictable overall performance.

[0008] To achieve complete coverage, oNeuTree employs a "bucket partitioning" mechanism, allowing rules to be copied to all buckets they cover. While this solves the coverage problem, it reverts to the old "rule copying" approach of decision tree methods, leading to severe memory bloat and maintenance overhead, and also increasing the cost of searching within buckets.

[0009] 2. Limited GPU Integration: The recursive hierarchy of existing RMI methods leads to a severe "branch selection bottleneck." When processing a batch of data packets on a GPU, different data packets may require traversing completely different model branches in the RMI model. This results in thread divergence, making the GPU's SIMT (Single Instruction, Multithreaded) architecture inefficient and leaving a large number of computing units idle. Meanwhile, the floating-point computing power of CPUs is far inferior to that of GPUs, and hardware such as FPGAs lacks efficient floating-point computing support, all of which limit the throughput under large-scale rule sets.

[0010] Therefore, a new data packet classification method is urgently needed, which must simultaneously solve three key problems: 1) Efficiently handle overlapping rules to achieve complete coverage; 2) Avoid rule copying, reducing memory and update overhead; 3) The model design must be compatible with modern GPU architectures to fully utilize their powerful parallel floating-point computing capabilities. Summary of the Invention

[0011] To address the shortcomings of existing technologies, the purpose of this invention is to provide a tuple-structured and hardware-accelerated neural network data packet classification method and system.

[0012] A tuple-structured and hardware-accelerated neural network packet classification method according to the present invention includes: Step S1: Reconstruct the packet classification problem into a high-dimensional multi-class classification problem, and generate a unique tuple index for each packet through a rule priority mechanism; Step S2: Construct a semi-structured model based on the unique tuple index; The semi-structured model includes a neural network classification model and a tuple space search structure; Step S3: Perform classification on a CPU-GPU hybrid streaming framework. The GPU predicts tuple indices based on a neural network classification model, and the CPU uses these indices to call a tuple space search structure for precise lookup, outputting the packet classification results.

[0013] Preferably, each data packet has one and only one highest priority matching rule; the effective decision space of each rule is unique and does not overlap.

[0014] Preferably, the rule set is divided into multiple tuples according to the prefix length of each field of the rule; the tuple space search structure performs rule lookup within the tuples based on a hash table.

[0015] Preferably, the data packet classification process includes: The input data packet header fields are vectorized and fed into a single, unified neural network classification model, which performs a non-recursive forward inference and outputs the index of the tuple with the highest probability. After the CPU obtains the predicted tuple_idx, it accesses the tuple space to search for the corresponding tuple in the structure. The CPU truncates and hashes the corresponding fields of the data packet according to the field prefix length of the tuple, performs an O(1) time search in the hash table, traverses the matching rule chain, and returns the highest priority matching rule and its action.

[0016] Preferably, the CPU-GPU hybrid streaming framework uses multiple CUDA streams to asynchronously execute inference tasks on the GPU, and employs a multi-buffering mechanism between the CPU and GPU to achieve pipelined operations for computation and data transfer.

[0017] Preferably, the multi-buffering mechanism includes: At time T, the GPU writes the inference results of batch N to the first buffer; at the same time, the CPU reads the results of batch N-1 from the second buffer and performs the search. At time T+1, the GPU writes the inference results of batch N+1 to the second buffer; at the same time, the CPU reads the results of batch N from the first buffer and performs the search.

[0018] Preferably, when the CPU searches within the tuples predicted by the model, if no matching rule is found, post-verification is immediately triggered to perform an ordered search on the remaining tuples until the highest priority matching rule is found.

[0019] According to the present invention, a tuple-structured and hardware-accelerated neural network packet classification system includes: Module M1: Reconstructs the packet classification problem into a high-dimensional multi-class classification problem, and generates a unique tuple index for each packet through a rule priority mechanism; Module M2: Constructs a semi-structured model based on the unique tuple index; The semi-structured model includes a neural network classification model and a tuple space search structure; Module M3: Performs classification on a CPU-GPU hybrid streaming framework. The GPU predicts tuple indices based on a neural network classification model, and the CPU uses these indices to call a tuple space search structure for precise lookup, outputting the packet classification results.

[0020] Preferably, each data packet has one and only one highest priority matching rule; the effective decision space of each rule is unique and does not overlap.

[0021] Preferably, the rule set is divided into multiple tuples according to the prefix length of each field of the rule; the tuple space search structure performs rule lookup within the tuples based on a hash table.

[0022] Preferably, the data packet classification process includes: The input data packet header fields are vectorized and fed into a single, unified neural network classification model, which performs a non-recursive forward inference and outputs the index of the tuple with the highest probability. After the CPU obtains the predicted tuple_idx, it accesses the tuple space to search for the corresponding tuple in the structure. The CPU truncates and hashes the corresponding fields of the data packet according to the field prefix length of the tuple, performs an O(1) time search in the hash table, traverses the matching rule chain, and returns the highest priority matching rule and its action.

[0023] Preferably, the CPU-GPU hybrid streaming framework uses multiple CUDA streams to asynchronously execute inference tasks on the GPU, and employs a multi-buffering mechanism between the CPU and GPU to achieve pipelined operations for computation and data transfer.

[0024] Preferably, the multi-buffering mechanism includes: At time T, the GPU writes the inference results of batch N to the first buffer; at the same time, the CPU reads the results of batch N-1 from the second buffer and performs the search. At time T+1, the GPU writes the inference results of batch N+1 to the second buffer; at the same time, the CPU reads the results of batch N from the first buffer and performs the search.

[0025] Preferably, when the CPU searches within the tuples predicted by the model, if no matching rule is found, post-verification is immediately triggered to perform an ordered search on the remaining tuples until the highest priority matching rule is found.

[0026] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves complete coverage of large-scale rule sets through a single model without the need for rule duplication. It significantly reduces model complexity by utilizing TSS middleware and fully unleashes hardware potential through CPU-GPU pipeline co-design, thereby significantly improving classification throughput and stability.

[0027] 2. Compared with NuevoMatch, this invention achieves complete coverage of the rule set by refactoring it into a classification problem; compared with NeuTree, this invention completely eliminates rule duplication.

[0028] 3. The CPU-GPU hybrid framework and efficient model design of this invention fully unleash the computing potential of GPUs. Experiments show that (on a 512k rule set), the throughput of this invention is 12.19 times and 9.37 times that of NuevoMatch and NeuTree, respectively. More importantly, since the model is not affected by rule set skewness, its performance stability (less jitter) is improved by 98.84 times and 156.98 times, respectively, which is crucial for latency-sensitive telecom-grade applications.

[0029] 4. The single, non-recursive neural network model structure of this invention is well-suited for massively parallel processing (SIMT) on GPUs, avoiding the recursive branches and thread divergence bottlenecks of RMI. This maximizes the return on investment (ROI) of expensive GPU hardware and provides a smooth scaling path for performance improvements as GPU hardware iterates (such as the NVIDIA Blackwell architecture).

[0030] 5. By introducing TSS as middleware, the model only needs to predict tuples (hundreds of categories) instead of rules (hundreds of thousands of categories), significantly reducing model complexity and training difficulty. This makes model training faster, easier to converge, and more generalizable, and less prone to overfitting.

[0031] 6. Based on the TSS decoupling design, the "immediate update" strategy of this invention can handle rule changes without retraining the model, resulting in extremely low latency; the "delayed update" strategy achieves a good balance between performance and overhead through monitoring feedback and incremental training. This makes this invention truly applicable to real-world SDN and cloud environments where policies change frequently, avoiding the huge overhead of "retraining every time an update" in traditional ML methods. Attached Figure Description

[0032] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the classification and maintenance workflow of the present invention.

[0033] Figure 2 This is a schematic diagram of the CPU-GPU hybrid streaming classification framework of the present invention.

[0034] Figure 3 This is a schematic diagram of the neural network architecture used in this invention.

[0035] Figure 4 This is a flowchart of the method of the present invention. Detailed Implementation

[0036] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0037] This invention discloses a neural network packet classification method and system based on tuple structures and hardware acceleration, aiming to solve the problems of incomplete model coverage or severe rule duplication encountered by existing learning-based packet classification methods when handling overlapping rules, as well as the performance bottleneck caused by the limited integration of GPUs. The core of this invention lies in firstly, completely reconstructing the packet classification problem from traditional regression prediction (such as RMI) into a high-dimensional multi-class classification problem, utilizing rule priority to ensure that each packet has a unique optimal matching in the high-dimensional space. Secondly, an innovative semi-structured model is constructed, which organically combines a single, unified neural network classification model and an efficient Tuple Space Search (TSS) structure. In this design, the neural network model is trained to predict the "tuple" index to which a packet should belong, rather than directly predicting specific, large-scale rule IDs, while the TSS structure is responsible for fast and accurate searching within the predicted small range. This method performs classification on a specially designed CPU-GPU hybrid streaming framework, achieving a co-design of hardware: the GPU utilizes its powerful parallel computing capabilities to handle neural network inference, while the CPU handles lightweight, control-flow-intensive, fast hash search within tuples. This invention achieves complete coverage of large-scale rule sets through a single model without the need for rule duplication, significantly reduces model complexity by utilizing TSS middleware, and fully unleashes hardware potential through CPU-GPU pipeline co-design, thereby significantly improving classification throughput and stability.

[0038] This invention belongs to the field of computer network technology, specifically relating to network packet processing technology, and in particular to a high-performance packet classification method and system for software-defined networking (SDN), network function virtualization (NFV) and other modern network systems.

[0039] This invention can be widely applied to network devices and security components that require in-depth inspection and fine-grained control of data streams, including but not limited to: Network security gateways: such as firewalls, intrusion detection systems (IDS), and intrusion prevention systems (IPS).

[0040] Traffic engineering equipment: such as load balancers and WAN optimization controllers.

[0041] Quality of Service (QoS) Guarantee: Implement priority marking and resource allocation for different service flows (such as video, voice, and data) in routers and switches.

[0042] Large-scale network environments: particularly suitable for data center networks, cloud computing environments and edge computing nodes, which are typically characterized by large rule sets, frequent dynamic changes, and extremely high requirements for processing throughput and latency.

[0043] This invention primarily addresses the following problems existing in current learning-based packet classification techniques: 1. The fundamental contradiction between model coverage and rule duplication: When dealing with rule sets with a large number of overlapping rules, existing methods (such as NuevoMatch) sacrifice model coverage to avoid complexity, resulting in extremely unstable performance; or (such as NeuTree) introduce a large number of rule duplications to achieve full coverage, leading to a surge in memory and maintenance overhead and increased search costs. This invention aims to achieve both full coverage and zero rule duplication.

[0044] 2. Mismatch between Model Architecture and Hardware Parallelism: The model design of existing methods (such as the recursive structure of RMI) is not conducive to GPU parallel computing, resulting in the underutilization of GPU's floating-point computing power. This creates a performance bottleneck, preventing learning-based methods from truly realizing their potential and making it difficult to meet the line-rate processing requirements of modern high-speed networks (such as 400G / 800G). This invention aims to design a GPU-friendly model architecture to fully unleash the hardware's potential.

[0045] To address the aforementioned problems, this invention proposes a method and system called TaNG (TSS-assisted Neural Networks on GPUs).

[0046] like Figure 4As shown, a tuple-structured, hardware-accelerated neural network packet classification method includes: Step S1: Reconstruct the packet classification problem into a high-dimensional multi-class classification problem, and generate a unique tuple index for each packet through a rule priority mechanism; Step S2: Construct a semi-structured model based on the unique tuple index; The semi-structured model includes a neural network classification model and a tuple space search structure; Step S3: Perform classification on a CPU-GPU hybrid streaming framework. The GPU predicts tuple indices based on a neural network classification model, and the CPU uses these indices to call a tuple space search structure for precise lookup, outputting the packet classification results.

[0047] The specific content includes: (1) Reconstruction of High-Dimensional Multi-Class Classification Problem: This invention no longer treats packet classification as an RMI-style regression problem, but reconstructs it as a multi-class classification problem in a high-dimensional space. This invention utilizes the key characteristic of "priority" in network rules. Although the constraint spaces of multiple rules may overlap, due to the existence of priority, any point (i.e., a specific packet) has one and only one highest-priority matching rule. This means that the "effective decision space" of each rule (i.e., the set of packets that it can match and that no higher-priority rule can match) is unique and non-overlapping.

[0048] For example, rule A (high priority) and rule B (low priority) overlap. The effective space of rule A is the entire space it defines; while the effective space of rule B is the space it defines excluding the portion that overlaps with A. This invention utilizes this characteristic to transform the packet classification problem into a standard multi-class classification task: learning a model that can directly predict which "effective decision space" a packet falls into.

[0049] (2) Semi-structured model design based on TSS: Directly using the "effective decision space" of millions of rules as the output category of the classification model will result in an excessively large output layer, making it difficult to train and slow inference. To solve this problem, this invention introduces a semi-structured design, using the traditional Tuple Space Search (TSS) structure as middleware between the neural network model and the rule set.

[0050] Class compression: Rules are grouped into different "tuples" based on the prefix length of their fields (this is a many-to-one mapping). For example, a 512k rule set might only correspond to a few hundred (e.g., ~400) tuples. The task of the neural network model is not to predict specific rule IDs, but rather to predict the "tuple index" to which the data packet belongs. This class compression, ranging from hundreds of thousands to hundreds, makes machine learning problems highly feasible and efficient.

[0051] Model-rule decoupling: The TSS structure decouples the model from the implementation details of the rules. The model is only responsible for learning the macroscopic boundaries of the tuple space (coarse-grained prediction), while the hash table in the TSS structure is responsible for performing microscopic rule lookups within the tuples (fine-grained search). The biggest advantage of this decoupling is that when rules are added, deleted, or modified within the tuples (as long as the TSS structure remains unchanged), the model is completely unaffected, which lays the foundation for efficient updates.

[0052] (3) Two-stage classification workflow: The data packet classification process is divided into two efficient stages: Phase 1: Model Prediction (GPU Execution): The input packet header fields (e.g., 5-tuples) are vectorized and fed into a single, unified neural network classification model. The model (running on the GPU) performs a non-recursive forward inference and outputs the index of the tuple with the highest probability (tuple_idx). This process is highly parallelized; all packets in a batch undergo the same computation graph, perfectly suited to the GPU's SIMT architecture.

[0053] Phase Two: Tuple Search (CPU Execution): After obtaining the predicted tuple_idx, the CPU accesses the corresponding tuple in the TSS structure. This tuple contains a pre-built hash table. The CPU truncates the corresponding fields of the data packet according to the field prefix length of the tuple (e.g., SIP / 24, DIP / 16) (e.g., 192.168.1.5->192.168.1.0) and performs hash calculation. It then performs an O(1) (average) time search in the hash table, traverses the (very few) matching rule chains, and returns the highest priority matching rule and its action.

[0054] (4) GPU-accelerated hybrid streaming frameworks: such as Figure 2 As shown, in order to maximize the total system throughput, this invention designs and implements a CPU-GPU hybrid streaming classification framework, which is a carefully designed hardware collaborative system: Hardware Co-design: Fully leverage the strengths of different processors. GPUs are dedicated to their strengths in high-parallelism, floating-point intensive computation (neural network inference). CPUs, on the other hand, handle their strengths in latency-sensitive, control-flow intensive tasks (such as packet batching, hash lookups in TSS structures, branching, and post-verification logic).

[0055] Asynchronous Streams: Execute inference tasks asynchronously on the GPU using multiple CUDA streams. This allows multiple batches of data packets to "fly" and "queue" on the GPU, keeping the GPU's compute cores constantly busy and maximizing its utilization.

[0056] Double Buffering: A double (or multi-buffered) mechanism is used between the CPU and GPU to enable pipelined computation and data transfer.

[0057] At time T: The GPU writes the inference results of batch N to buffer A; at the same time, the CPU reads the results of batch N-1 from buffer B and performs the search.

[0058] At time T+1: The GPU writes the inference results of batch N+1 to buffer B; at the same time, the CPU reads the results of batch N from buffer A and performs the search.

[0059] This pipelined approach completely hides the latency of CPU search and data copying (assuming they are less than GPU inference time), so that the total throughput of the system is limited only to the slowest stage (usually GPU inference), rather than the sum of the latency of all stages.

[0060] TensorRT optimization: Optimize the trained model using NVIDIA TensorRT, including operator fusion (merging multiple computation layers into a single CUDA core), precision quantization (such as FP16 or INT8), and memory optimization, to further squeeze the inference performance of the GPU.

[0061] (5) Post-verification mechanism: Since neural network models are probabilistic, their predictions may be wrong (albeit with a very low probability). To ensure classification accuracy (or extremely high accuracy), this invention includes a post-verification mechanism: When the CPU searches within the tuples predicted by the model, if no matching rule is found (which is the most common error type), post-validation is triggered immediately.

[0062] The post-validation mechanism will perform an ordered search on the remaining tuples (e.g., sorting them according to priority, generality, or historical hit rate to optimize search efficiency) until the highest priority matching rule is found.

[0063] Because the model's base prediction accuracy is very high, the probability of validation after triggering is extremely low. Therefore, this fallback mechanism has a negligible impact on average throughput, but it can improve the final classification accuracy to over 99.99%, ensuring the system's reliability.

[0064] (6) Rule update mechanism based on semi-structured design: The decoupled design of the TSS middleware makes rule updates efficient and flexible. Immediate Update: • Modifications and deletions of rules can be performed directly in the tuple hash table corresponding to the TSS in O(1) operations.

[0065] For rule insertion, this invention employs a "restricted update strategy": if a new rule cannot perfectly match an existing tuple, it is inserted into the existing tuple with the closest and most compatible prefix signature.

[0066] This approach avoids changes to the TSS structure (i.e., the model's output category), thus eliminating the need to retrain the model and reducing update latency to the microsecond level.

[0067] Deferred Update: • When “immediate updates” accumulate to a certain extent (causing a large number of rules to be inserted into suboptimal tuples), the model’s prediction accuracy will decrease, which in turn will lead to a decrease in throughput (because post-validation is triggered more frequently).

[0068] The system automatically triggers "delayed updates" by monitoring throughput and suboptimal rule counters. This is a feedback loop.

[0069] • Incremental training: If the TSS structure has not changed significantly (the number of suboptimal rules is less than the threshold), the existing model is fine-tuned using only new data to quickly restore performance.

[0070] • Full retraining: If the rule set drifts too much (suboptimal rule count > threshold), the TSS structure is reconstructed and the model is trained from scratch to reach a new optimal state.

[0071] (7) Neural network model architecture: The neural network used in this invention is a fully connected network (FCN) with residual connections, which is designed to balance high fitting ability and high inference efficiency.

[0072] Input layer: Data packet header fields (such as SIP, DIP, SP, DP, PRO) are divided into 16-bit blocks and converted into 32-bit floating-point numbers to form a standardized input vector. For example, a 5-tuple (120 bits) can be divided into seven 16-bit blocks (two for SIP and two for DIP), resulting in an input dimension S=7. This segmentation method preserves sufficient information while standardizing the input format.

[0073] Hidden layers: composed of multiple stacked residual blocks. A typical, deep fully connected network (MLP) suffers from vanishing gradients during training. A typical, shallow, and wide network requires a massive number of parameters to fit complex high-dimensional boundaries. Residual blocks perfectly solve this problem: they allow the network to be built very deep (e.g., L=6), greatly enhancing the model's fitting capacity, while "skip connections" ensure smooth backpropagation of gradients, enabling training to converge.

[0074] Output layer: A fully connected layer with an output dimension C equal to the total number of tuples in the TSS structure (e.g., C=400). During inference, the Argmax function is used to select the tuple index with the highest confidence score and pass it to the CPU.

[0075] Example 1 This embodiment provides a tuple-structured, hardware-accelerated neural network packet classification system, including: 1. System architecture as follows Figure 1 As shown, the system of the present invention mainly consists of two decoupled workflows: a maintenance workflow (offline) and a classification workflow (online).

[0076] Workflow maintenance: responsible for model training and updates.

[0077] 1) TSS Structure Construction: Receives a rule set, iterates through all rules, and constructs a TSS structure based on the length of their field prefixes. This structure is a tuple table, where each tuple (defined by a unique prefix signature) contains a hash table.

[0078] 2) Training Data Generation: Using historical or generated data packets, determine the highest priority rule matching each data packet through a complete TSS lookup (i.e., potentially traversing multiple tuples). Record the tuple index to which this rule belongs. This ultimately generates a large number of (data packet vector, tuple index) training data pairs.

[0079] 3) Model Training: The training engine uses this data to train models such as... Figure 3The neural network model shown illustrates a fully connected network structure containing two residual blocks, including an input layer (S-dimensional), a hidden layer (N-dimensional), and an output layer (C-dimensional).

[0080] 4) Model Deployment: The trained model (after optimization by TensorRT) and the TSS structure are loaded into the classification workflow.

[0081] 5) Update processing: The update engine is responsible for handling dynamic changes in the rule set and performing immediate updates or delayed updates (incremental / full training).

[0082] Classification workflow: Responsible for real-time packet classification. It is deployed on a CPU-GPU hybrid framework.

[0083] 2. The specific classification process is as follows: Figure 2 As shown: 1) Batch processing (CPU): The CPU (host) rapidly captures data packets from the network interface (or memory) and extracts their header fields (such as 5-tuples). To efficiently utilize the GPU, the CPU combines multiple (e.g., 8192) packet headers into a batch and organizes them in contiguous memory.

[0084] 2) Data transfer (CPU -> GPU): The CPU asynchronously copies the batch data to the GPU (device) memory via the PCIe bus.

[0085] 3) Model Inference (GPU): A CUDA stream (e.g., stream 0) on the GPU is activated, and the TensorRT inference engine (e.g., context 0) on that stream begins parallel inference of the neural network model. The 8192 packets in the batch are processed simultaneously by thousands of CUDA cores on the GPU. The model is a... Figure 3 The diagram shows a fully connected network with residual blocks. The output of the inference is an array of the predicted tuple indices for each packet in the batch.

[0086] 4) Result backhaul (GPU -> CPU): The inference result (e.g., index 0) is asynchronously copied from the GPU device memory back to buffer A (e.g., buffer 0) on the CPU side.

[0087] 5) Pipelining: • While the GPU processes batch N and writes the results to buffer A: • The search engine on the CPU can simultaneously process the inference results of the previous batch (N-1) in buffer B (e.g., buffer 1).

[0088] This double-buffering and multi-stream parallel mechanism allows the CPU's search and the GPU's inference to completely overlap, with neither the CPU nor the GPU idling and waiting for the other, greatly improving the overall resource utilization and throughput of the system.

[0089] 6) Tuple search (CPU): The search engine on the CPU begins traversing the tuple index array in buffer A.

[0090] • Scenario A (Prediction correct, high probability): For a tuple index, the CPU accesses the corresponding tuple in the TSS structure. This tuple defines the prefix length required for hash calculation. The CPU truncates the corresponding field of the current packet (e.g., if the packet is SIP192.168.1.5 and the tuple requires / 24, it is truncated to 192.168.1.0) and hashes it, locating the hash table bucket within the tuple in O(1) time. The CPU traverses the (usually very few) rules in the bucket, performs an exact, full-field match, finds the highest priority rule, and returns its action.

[0091] • Scenario B (Prediction Error, Low Probability): If no matching rule is found within the predicted tuples (tuple index prediction error), a post-validation mechanism is triggered. The CPU will search other tuples in a predetermined order (e.g., sorted by tuple generality or historical hit rate) until a matching rule is found. Due to the extremely high model accuracy, this situation is rare and has minimal impact on average performance.

[0092] 7) Output Actions: The search engine outputs the final actions (action 0, action 1, ...) corresponding to all data packets in the batch.

[0093] 3. Neural network models, such as Figure 3 As shown: Input: A 5-tuple header (SIP 32bit, DIP 32bit, SP 16bit, DP 16bit, PRO 8bit) is split into 7 16-bit blocks (SIP and DIP each occupy 2 blocks, SP and DP each occupy 1 block, and PRO occupies 1 block with padding). Each block is converted to a 32-bit floating-point number, so the input dimension S=7 (or determined according to the actual splitting method).

[0094] The main body of the network consists of an initial fully connected layer, L residual blocks (e.g., L=6), and a final fully connected layer. The initial fully connected layer maps the input S dimensions to the hidden layer dimension N dimensions (e.g., N=512).

[0095] Residual Blocks: Each residual block contains two fully connected layers (N x N) and a ReLU activation function, along with a skip connection. This design allows the network to be built deep enough to learn complex tuple boundaries in high-dimensional space while maintaining training stability.

[0096] Output: The output dimension C of the final fully connected layer is equal to the total number of tuples in the TSS structure. This layer outputs C logits (raw scores), and the index of the tuple with the highest probability can be obtained through the Argmax operation.

[0097] 4. Training process 1) Data generation: Using historical data packets, the highest priority rule matching each data packet and its corresponding tuple index are determined through the TSS structure, and training samples <data packet vector, tuple index> are generated.

[0098] 2) Class Balance: Network traffic and rule distributions are often highly skewed (e.g., a few tuples may hit 90% of the traffic). If trained directly, the model will be heavily biased towards these common tuples, ignoring those that are uncommon but potentially extremely important (e.g., high-priority security rules). Therefore, this invention uses an oversampling method to oversample tuples with fewer than a threshold number of samples. The category (e.g., 1000) is replicated until its sample size reaches [a certain number]. This ensures that the model gives sufficient attention to all categories, especially rare categories, during training.

[0099] 3) Model Training: Use deep learning frameworks such as PyTorch, employing the Cross-Entropy Loss function and the Adam optimizer for training. Set a large batch size (e.g., 8192) to fully utilize the GPU. Use a learning rate decay strategy (e.g., initial 0.001, multiplied by 0.1 every 200 epochs) to enable the model to converge quickly in the early stages of training and fine-tune it in the later stages.

[0100] 5. Rule Update This invention provides two update strategies to cope with dynamic rule sets, which together constitute an adaptive feedback system: 1) Update immediately (default high-speed path): Deletion / Modification: Deleting or modifying rules directly in the hash table of TSS tuples in O(1) time.

[0101] Insertion: If the prefix signature of the new rule exactly matches an existing tuple T, insert it directly into T. Otherwise, search for all "compatible" tuples (i.e., the signatures of tuple T). ( Indicates the first The prefix length in each dimension is less than or equal to the signature of the new rule. (Prefix length for the corresponding dimension), and select the total prefix length. Insert the largest tuple.

[0102] Advantages: Extremely fast (microsecond level), no need for retraining, and almost no interruption to the system.

[0103] 2) Immediate Update + Delayed Update: Monitor: Deploy a lightweight monitor on the CPU-intensive search engine to continuously track classification throughput. ) and suboptimal rule counter ( (i.e., the number of rules inserted into incompletely matching tuples).

[0104] Trigger condition: When throughput drops by more than a threshold At that time, a "delayed update" is triggered.

[0105] Decision-making mechanism: Engine update check .

[0106] if ( If the threshold is set to 10000, it indicates that the macroscopic structure of the TSS is still reasonable, but the model's perception of the boundaries is outdated. In this case, incremental training (fine-tuning the model using new data) is performed, and this process is very fast (e.g., within 30 minutes).

[0107] if This indicates that the macroscopic structure of the TSS is severely out of place with the current rule set (for example, many new prefix signature combinations have emerged). In this case, a complete retraining is performed, which involves rebuilding the TSS structure, regenerating training data, and training the model from scratch.

[0108] Advantages: When continuous updates affect throughput, a delayed update is triggered, and the throughput can automatically recover to the initial high-performance level, maintaining the long-term stability and high performance of the system in dynamic environments.

[0109] The present invention also provides a tuple-structured and hardware-accelerated neural network packet classification system. The tuple-structured and hardware-accelerated neural network packet classification system can be implemented by executing the process steps of the tuple-structured and hardware-accelerated neural network packet classification method. That is, those skilled in the art can understand the tuple-structured and hardware-accelerated neural network packet classification method as a preferred embodiment of the tuple-structured and hardware-accelerated neural network packet classification system.

[0110] Specifically, a tuple-structured, hardware-accelerated neural network packet classification system includes: Module M1: Reconstructs the packet classification problem into a high-dimensional multi-class classification problem, and generates a unique tuple index for each packet through a rule priority mechanism; Module M2: Constructs a semi-structured model based on the unique tuple index; The semi-structured model includes a neural network classification model and a tuple space search structure; Module M3: Performs classification on a CPU-GPU hybrid streaming framework. The GPU predicts tuple indices based on a neural network classification model, and the CPU uses these indices to call a tuple space search structure for precise lookup, outputting the packet classification results.

[0111] Each data packet has one and only one highest priority matching rule; the effective decision space of each rule is unique and does not overlap.

[0112] The rule set is divided into multiple tuples based on the prefix length of each field in the rule; the tuple space search structure performs rule lookup within the tuples based on a hash table.

[0113] The data packet classification process includes: The input data packet header fields are vectorized and fed into a single, unified neural network classification model, which performs a non-recursive forward inference and outputs the index of the tuple with the highest probability. After the CPU obtains the predicted tuple_idx, it accesses the tuple space to search for the corresponding tuple in the structure. The CPU truncates and hashes the corresponding fields of the data packet according to the field prefix length of the tuple, performs an O(1) time search in the hash table, traverses the matching rule chain, and returns the highest priority matching rule and its action.

[0114] The CPU-GPU hybrid streaming framework uses multiple CUDA streams to asynchronously execute inference tasks on the GPU, and employs a multi-buffering mechanism between the CPU and GPU to achieve pipelined operations for computation and data transfer.

[0115] The multi-buffering mechanism includes: At time T, the GPU writes the inference results of batch N to the first buffer; at the same time, the CPU reads the results of batch N-1 from the second buffer and performs the search. At time T+1, the GPU writes the inference results of batch N+1 to the second buffer; at the same time, the CPU reads the results of batch N from the first buffer and performs the search.

[0116] When the CPU searches within the tuples predicted by the model, if no matching rule is found, post-validation is immediately triggered to perform an ordered search on the remaining tuples until the highest priority matching rule is found.

[0117] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0118] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A tuple-structured, hardware-accelerated neural network data packet classification method, characterized in that, include: Step S1: Reconstruct the packet classification problem into a high-dimensional multi-class classification problem, and generate a unique tuple index for each packet through a rule priority mechanism; Step S2: Construct a semi-structured model based on the unique tuple index; The semi-structured model includes a neural network classification model and a tuple space search structure; Step S3: Perform classification on a CPU-GPU hybrid streaming framework. The GPU predicts tuple indices based on a neural network classification model, and the CPU uses these indices to call a tuple space search structure for precise lookup, outputting the packet classification results.

2. The tuple structure and hardware-accelerated neural network packet classification method according to claim 1, characterized in that, Each data packet has one and only one highest priority matching rule; the effective decision space of each rule is unique and does not overlap.

3. The tuple structure and hardware-accelerated neural network packet classification method according to claim 1, characterized in that, The rule set is divided into multiple tuples based on the prefix length of each field in the rule; the tuple space search structure performs rule lookup within the tuples based on a hash table.

4. The tuple structure and hardware-accelerated neural network packet classification method according to claim 3, characterized in that, The data packet classification process includes: The input data packet header fields are vectorized and fed into a single, unified neural network classification model, which performs a non-recursive forward inference and outputs the index of the tuple with the highest probability. After the CPU obtains the predicted tuple_idx, it accesses the tuple space to search for the corresponding tuple in the structure. The CPU truncates and hashes the corresponding fields of the data packet according to the field prefix length of the tuple, performs an O(1) time search in the hash table, traverses the matching rule chain, and returns the highest priority matching rule and its action.

5. The tuple structure and hardware-accelerated neural network packet classification method according to claim 1, characterized in that, The CPU-GPU hybrid streaming framework uses multiple CUDA streams to asynchronously execute inference tasks on the GPU, and employs a multi-buffering mechanism between the CPU and GPU to achieve pipelined operations for computation and data transfer.

6. The tuple structure and hardware-accelerated neural network packet classification method according to claim 5, characterized in that, The multi-buffering mechanism includes: At time T, the GPU writes the inference results of batch N to the first buffer; at the same time, the CPU reads the results of batch N-1 from the second buffer and performs the search. At time T+1, the GPU writes the inference results of batch N+1 to the second buffer; at the same time, the CPU reads the results of batch N from the first buffer and performs the search.

7. The tuple structure and hardware-accelerated neural network packet classification method according to claim 1, characterized in that, When the CPU searches within the tuples predicted by the model, if no matching rule is found, post-validation is immediately triggered to perform an ordered search on the remaining tuples until the highest priority matching rule is found.

8. A tuple-structured, hardware-accelerated neural network data packet classification system, characterized in that, include: Step S1: Reconstruct the packet classification problem into a high-dimensional multi-class classification problem, and generate a unique tuple index for each packet through a rule priority mechanism; Step S2: Construct a semi-structured model based on the unique tuple index; The semi-structured model includes a neural network classification model and a tuple space search structure; Step S3: Perform classification on a CPU-GPU hybrid streaming framework. The GPU predicts tuple indices based on a neural network classification model, and the CPU uses these indices to call a tuple space search structure for precise lookup, outputting the packet classification results.

9. The tuple structure and hardware-accelerated neural network packet classification system according to claim 8, characterized in that, Each data packet has one and only one highest priority matching rule; the effective decision space of each rule is unique and does not overlap.

10. The tuple structure and hardware-accelerated neural network packet classification system according to claim 8, characterized in that, The rule set is divided into multiple tuples based on the prefix length of each field in the rule; the tuple space search structure performs rule lookup within the tuples based on a hash table.

Images