Network traffic classification method and device based on multi-modal feature fusion

Abstract

The invention discloses a network flow classification method and device based on multi-modal feature fusion, and the method comprises the steps: obtaining network flow data, extracting an IP address as a node, and constructing a communication graph; time modal features and event modal features are extracted for each node, and standardization processing is carried out on the time modal features and the event modal features; the time modal features and the event modal features are aligned; fusing the two types of aligned features; performing multi-layer neighbor sampling and hierarchical aggregation on the communication graph by using a graph neural network model to obtain structure-enhanced node features containing multi-hop neighbor information, inputting the node features into a linear classifier, outputting a category probability, selecting a category with the maximum probability as a prediction result, and performing iterative training to obtain a classification model; and mapping the classification model to a data plane of the programmable switch to realize online reasoning of the data packet. According to the method, high-precision identification of various network traffic types is realized by fusing the spatial modal, time and event modal characteristics.

Description

Technical Field

[0001] This invention relates to a network traffic classification method and apparatus based on multimodal feature fusion, belonging to the field of network traffic classification technology. Background Technology

[0002] Network traffic classification is a fundamental and core task in network security and management, and its results directly support key applications such as traffic monitoring, anomaly detection, Quality of Service (QoS) assurance, and network planning. However, with the continuous expansion of network scale, the surge in encrypted traffic, and the increasing diversification of network protocols, traditional traffic classification methods have gradually revealed significant limitations, as follows:

[0003] First, traditional port-based methods are ineffective. Dynamic port allocation, port reuse, and port masquerading reduce the accuracy of port number-based classification methods, making them unsuitable for modern network environments. Second, there are limitations to Deep Packet Inspection (DPI). DPI relies on protocol feature matching, performs poorly on encrypted traffic, has high computational overhead, struggles to meet real-time processing requirements, and raises privacy concerns. Third, machine learning methods based on statistical features are insufficient. Traditional methods (such as decision trees, SVM, and random forests) primarily focus on the statistical features of individual traffic streams, failing to fully utilize network topology information and inter-node relationships, making it difficult to capture complex network behavior patterns and limiting performance in encrypted traffic and complex protocol scenarios. Fourth, deep learning methods lack structural information. Existing deep learning methods (such as CNN, RNN, and LSTM) mainly focus on the temporal or spatial features of individual traffic streams, failing to model network traffic as a graph structure, and thus cannot utilize communication relationships between IP addresses and network topology information, limiting the improvement of classification performance. Fifth, graph neural network methods suffer from insufficient multimodal feature fusion. Existing graph neural network methods (such as GCN and GraphSAGE) typically use a single type of feature representation and do not effectively integrate multimodal features (such as basic traffic statistics features and transport layer features). Inconsistent feature spaces lead to poor fusion results and make it difficult to fully leverage the complementary advantages of different feature types. Summary of the Invention

[0004] In view of this, the present invention provides a network traffic classification method, apparatus, computer device and storage medium based on multimodal feature fusion. It achieves high-precision and real-time network traffic classification by constructing a communication graph to capture node topology relationships, designing heterogeneous feature alignment, mining the value of multimodal features through fusion mechanism, optimizing GraphSAGE sampling and aggregation strategy and deploying online line-speed inference collaboration.

[0005] The first objective of this invention is to provide a network traffic classification method based on multimodal feature fusion.

[0006] The second objective of this invention is to provide a network traffic classification device based on multimodal feature fusion.

[0007] A third objective of this invention is to provide a computer device.

[0008] A fourth objective of this invention is to provide a storage medium.

[0009] The first objective of this invention can be achieved by adopting the following technical solution:

[0010] A network traffic classification method based on multimodal feature fusion, the method comprising:

[0011] Obtain network traffic data;

[0012] Source host IP and destination host IP address are extracted from network traffic data as nodes to construct a communication graph, which serves as a spatial modal feature.

[0013] For each node, packet-level temporal features, flow-level statistical features, and TCP header flag features are extracted and standardized respectively. The packet-level temporal features and flow-level statistical features are used as time modal features, and the TCP header flag features are multiplied by the feature contribution weight to become event modal features.

[0014] Align the temporal modality features and the event modality features;

[0015] The aligned temporal modal features and event modal features are fused to output dual-modal fused features;

[0016] For the communication graph, perform multi-layer neighbor sampling to determine Q-layer neighbor nodes;

[0017] Each node is given a dual-modal fusion feature. A hierarchical aggregation is performed using a graph neural network model to obtain structurally enhanced node features containing multi-hop neighbor information. The classification model is then iteratively trained.

[0018] The trained classification model is structurally simplified and its weights are quantized. The classification model parameters are converted into a fixed-point format and decomposed into multi-level flow table matching actions, which are then mapped to the data plane pipeline of the programmable switch. This enables the programmable switch to process data packets passing through it, extract features, and input them into the classification model. The linear classifier outputs the class probability, and the class with the highest probability is selected as the prediction result, thus achieving linear-speed online inference of data packets.

[0019] Furthermore, the step of extracting source host IP and destination host IP addresses from network traffic data as nodes to construct a communication graph includes:

[0020] Extract the five-tuple from each batch of network traffic data, extract all unique source host IP addresses and destination host IP addresses from the current batch of data, establish a bidirectional mapping relationship between IP addresses and node IDs, and obtain a one-to-one corresponding set of nodes.

[0021] Iterate through each traffic record in the current batch. For the source host IP and destination host IP of each record, establish undirected edges between the corresponding nodes. Use an adjacency list to store the connection relationship of the graph, ensuring that each edge is recorded only once in the adjacency list, and obtain the edge set.

[0022] Calculate graph properties, including total number of nodes, total number of edges, average degree, and connectivity.

[0023] Furthermore, the packet-level timing features are extracted based on the timestamp information of the data packets; the flow-level statistical features are obtained by aggregating and calculating all data packets in the same network flow; the TCP header flag features include flag count and flag ratio.

[0024] Based on the pre-conducted feature importance ablation experiment, the contribution weight of each TCP header flag feature is calculated. The contribution weight vector is determined based on the difference in the impact of each flag bit on the classification accuracy. The flag bit proportion is weighted and encoded using the contribution weight, that is, the contribution weight vector is multiplied element by element by the flag bit proportion to generate an event modality feature vector that retains the difference in importance.

[0025] Furthermore, aligning the temporal modal features and the event modal features includes: mapping the temporal modal features and the weighted encoded event modal features to a unified feature space based on linear transformation projection, and performing normalization processing;

[0026] Furthermore, the fusion of the aligned temporal modal features and event modal features includes:

[0027] The aligned temporal modality features and aligned event modality features are concatenated along the feature dimension to form a concatenated feature with a dimension twice that of the alignment dimension. The concatenated feature is then mapped to the fused feature through a sequence network containing a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout pruning layer, outputting a dual-modality fused feature.

[0028] Alternatively, the aligned temporal modality features and the aligned event modality features can be added element-wise; the added features can then be processed through a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output dual-modal fusion features.

[0029] Alternatively, the aligned temporal modality features and the aligned event modality features can be multiplied element-wise; the multiplied features can then be processed through a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output dual-modal fusion features.

[0030] Alternatively, a multi-head attention mechanism can be used to calculate the attention weights of temporal modality features on event modality features, resulting in weighted event modality features; the temporal modality features The weighted event modal features are concatenated and mapped to the fused features through a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output the dual-modal fusion features.

[0031] Alternatively, for temporal modality features and event modality features, corresponding gating weights are generated by passing them through a gating network containing a linear transformation layer and a sigmoid activation function layer, respectively. The temporal modality features are multiplied by the corresponding gating weights to obtain the gated temporal modality features. The event modality features are multiplied by the corresponding gating weights to obtain the gated event modality features. The two types of gated features are concatenated and mapped to the fusion layer through a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output the dual-modality fused features.

[0032] Furthermore, the multi-layer neighbor sampling refers to determining the neighbor nodes of each node through the communication graph, which are called first-layer neighbor nodes, determining the neighbor nodes of the neighbor nodes, which are called second-layer neighbor nodes, repeating Q times to determine Q layers of neighbor nodes.

[0033] The hierarchical aggregation refers to fusing the features of each node and its neighboring nodes at different layers, including:

[0034] The vector representation of node v after multi-level sampling and aggregation at the k-th layer is as follows In this context, the bimodal fusion features of node v serve as the input vector for layer 0 of z, i.e. The calculation for stratified aggregation using mean aggregation is as follows:

[0035] First-level aggregation:

[0036] Second-level aggregation:

[0037] k-th level aggregation:

[0038] Where AGG represents the aggregation function, W represents the learnable weight matrix, and σ represents the activation function;

[0039] After k-level aggregation, we obtain Then, Input a linear classifier, output class probabilities, and select the class with the highest probability as the prediction result:

[0040]

[0041] Among them, W cls Let b represent the weight matrix that a linear classifier can learn. cls This indicates the bias of the linear classifier.

[0042] Furthermore, the process of simplifying the structure and quantizing the weights of the trained classification model, converting the classification model parameters into a fixed-point format, and decomposing them into multi-level flow table matching actions, which are then mapped to the data plane pipeline of the programmable switch, includes:

[0043] A register array is constructed in the data plane of the programmable switch to maintain and update the neighbor list and neighbor features of nodes in real time. The parameters of the fully connected layers in the trained classification model are quantized to fixed point, converting the 32-bit floating-point weights into a fixed-point format supported by the programmable switch registers. Based on the input feature dimension and the output dimension, the weight matrix of the linear classifier is decomposed into multiple dot product operators, and a pre-computed product lookup table is generated for each dot product operator. The product lookup table is stored in the matching action table of the programmable switch. The metadata of the data packet is used as the intermediate result carrier. The weight segment is accessed through the matching-action mechanism, and the multiplication operation between the feature vector and the weight matrix is ​​completed through the accumulation operation in the data plane pipeline. The predicted category is output through the comparison instruction.

[0044] The second objective of this invention can be achieved by adopting the following technical solution:

[0045] A network traffic classification device based on multimodal feature fusion, the device comprising:

[0046] The network traffic data acquisition module is used to acquire network traffic data.

[0047] A communication graph construction module is used to extract source host IP and destination host IP addresses from network traffic data as nodes to construct a communication graph, which serves as a spatial modal feature.

[0048] The dual-modal feature extraction module is used to extract packet-level temporal features, flow-level statistical features, and TCP header flag features for each node, and perform standardization processing on each. The packet-level temporal features and flow-level statistical features are used as temporal modal features, and the TCP header flag features are used as event modal features.

[0049] A dual-modal feature alignment module is used to align temporal modal features and event modal features;

[0050] The dual-modal feature fusion module is used to fuse aligned temporal modal features and event modal features;

[0051] The multi-layer neighbor sampling module is used to perform multi-layer neighbor sampling on the communication graph to determine the Q-layer neighbor nodes;

[0052] The hierarchical aggregation module is used to attach bimodal fusion features to each node. It uses a graph neural network model to perform hierarchical aggregation, obtains structurally enhanced node features that include multi-hop neighbor information, and iteratively trains to obtain a classification model.

[0053] The online inference module is used to simplify the structure and quantize the weights of the trained classification model, convert the classification model parameters into a fixed-point format, and decompose them into multi-level flow table matching actions. These actions are then mapped to the data plane pipeline of the programmable switch, enabling the programmable switch to process data packets passing through it, extract features, and input them into the classification model. The linear classifier outputs the class probability, and the class with the highest probability is selected as the prediction result, thus achieving linear-speed online inference for data packets.

[0054] The third objective of this invention can be achieved by adopting the following technical solution:

[0055] A computer device includes a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the above-described traffic classification method.

[0056] The fourth objective of this invention can be achieved by adopting the following technical solution:

[0057] A storage medium storing a program that, when executed by a processor, implements the traffic classification method described above.

[0058] The present invention has the following advantages over the prior art:

[0059] 1. This invention constructs a unified feature system for network multimodalities: network traffic features are divided into spatial modalities (communication graph), temporal modalities (packet-level time-series features and flow-level statistical features), and event modalities (TCP header flags). Temporal and event modalities are mapped to a unified space through feature alignment and fusion, achieving cross-modal fusion from temporal behavior patterns to event semantics. This fusion method effectively improves feature representation capabilities. The fused node features propagate through multiple hops in the graph structure, thereby obtaining cross-node interaction information and topology patterns.

[0060] 2. Interpretable analysis of feature contribution of the present invention: Ablation experiments were used to evaluate the contribution of each modality feature to classification performance, and the importance of key network event markers in event modalities was analyzed. Compared with the onehot encoding method that generally uses the same weight for binarization, the weighted encoding vector maintains the relative size relationship of the original TCP proportion information, and highlights the contribution of important markers through weight scaling. The calculation process is more intuitive and interpretable.

[0061] 3. This invention employs a multi-layer GraphSAGE sampling structure to enhance representation: Through multi-layer GraphSAGE sampling aggregation, a recursive fusion of "node self-features + neighbor behavior features + multi-hop structural semantics" is achieved, constructing a structurally enhanced representation with stronger expressive power. This method surpasses traditional single-node feature classification methods. This technology can capture local network structure, communication links, and multi-hop propagation features, significantly improving the expressive power of complex network behaviors.

[0062] 4. The classification model of this invention is deployed offline for training and online for inference in a coordinated manner: After the offline training and optimization of the model are completed in the control plane, the model is lightweighted and deployed on the data plane of the programmable switch through P4 program mapping; the pipeline structure of the switch is fully utilized to realize line-speed inference of network traffic, which solves the high latency problem of the traditional "data acquisition-reporting server-offline inference" mode and meets the real-time processing requirements. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0064] Figure 1 This is a framework diagram of the network traffic classification method based on multimodal feature fusion according to Embodiment 1 of the present invention.

[0065] Figure 2 This is a technical roadmap for the network traffic classification method based on multimodal feature fusion in Embodiment 1 of the present invention.

[0066] Figure 3 This is a flowchart of the network traffic classification method based on multimodal feature fusion according to Embodiment 1 of the present invention.

[0067] Figure 4 This is a block diagram illustrating the principle of feature alignment and feature fusion in Embodiment 1 of the present invention.

[0068] Figure 5This is a structural block diagram of the network traffic classification device based on multimodal feature fusion according to Embodiment 3 of the present invention.

[0069] Figure 6 This is a structural block diagram of the computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example 1:

[0072] This embodiment provides a network traffic classification method based on multimodal feature fusion to improve the accuracy and efficiency of network traffic classification. By treating feature alignment and feature fusion as independent modules, it can better handle heterogeneous features and improve classification performance. Figure 1 and Figure 2 As shown, the technical approach of this method is as follows: First, a communication graph is constructed based on the five-tuple information of the constructed network traffic data; second, multimodal features are extracted and feature alignment and fusion are completed; then, the aligned features are input into a graph neural network, and multi-classification of network traffic is achieved through multi-layer sampling and aggregation. Finally, after training is completed, it is deployed and inferred online on a switch.

[0073] like Figure 3 As shown, the network traffic classification method based on multimodal feature fusion in this embodiment includes the following steps:

[0074] S301, Obtain network traffic data.

[0075] This embodiment obtains network traffic data by constructing a dataset. In order to construct the dataset, the following features need to be extracted from the data packets:

[0076] First, for the network layer, 12 core fields are extracted from the IP protocol header, including protocol version (version), header length (ihl), type of service (tos), total length (len), identifier (id), flags (flags), fragment offset (frag), time to live (ttl), protocol type (proto), header checksum (chksum), source host IP address (src), and destination IP address (dst). These fields can reflect the basic communication characteristics and routing information of the network layer.

[0077] Secondly, for the transport layer, 10 basic fields were extracted, including source port (sport), destination port (dport), sequence number (seq), acknowledgment number (ack), data offset (dataofs), reserved field (reserved), window size (window), checksum (chksum), urgent pointer (urgptr), and TCP options (options). In particular, the system performed fine-grained parsing of TCP flags, extracting the eight standard flags (FIN, SYN, RST, PSH, ACK, URG, ECE, CWR) into independent binary features to capture protocol state information at different stages such as TCP connection establishment, data transmission, and connection termination.

[0078] Then, to further enhance the feature representation capability, some derived features were also statistically analyzed. Based on the original protocol fields, several derived features were calculated, including the total packet size (packet_size), IP header length (ip_header_len), TCP header length (tcp_header_len), and TCP payload length (tcp_payload_len). In addition, the timestamp information of the packets was retained to provide a basis for subsequent time-series feature analysis.

[0079] Finally, labels are added to each record and the data is balanced. An application category label is added to each extracted data packet record. To address the potential class imbalance problem in the dataset (such as attack traffic samples being far fewer than normal traffic samples), oversampling methods are used to augment the data of a few categories to ensure that the number of samples in each category is relatively balanced.

[0080] The final dataset is stored in CSV format. Each record corresponds to a complete feature vector of a TCP packet and the corresponding application category label. To adapt to the processing of large-scale datasets, this embodiment adopts a batch data loading mechanism, which divides the dataset into multiple batches according to a preset batch size. Network traffic data is loaded in batches. This method supports incremental processing of large-scale datasets and can avoid memory overflow problems caused by loading all data at once.

[0081] S302. Extract the source host IP and destination host IP addresses from the network traffic data as nodes to construct a communication graph.

[0082] This embodiment uses a communication graph as a spatial modal feature. Spatial modal features primarily reflect end-to-end communication between hosts. The basic idea behind constructing spatial modal features is to treat each unique IP address as a node, adding an undirected edge when communication occurs between them, thus forming a traffic-driven communication graph. This construction method allows nodes in the graph to represent logical entities of hosts or ports in the network, while edges represent actual transmission behaviors between two entities. This communication graph naturally reflects network interactions, enabling subsequent graph neural network models to capture communication relationship information between nodes.

[0083] The communication graph in this embodiment is represented as G = (V, E), where V is the set of nodes and E is the set of edges. The specific construction process is as follows:

[0084] First, establish a mapping between IP addresses and node IDs: The communication graph construction module extracts the 5-tuple <source IP, source port, destination IP, destination port, protocol> from the dataset built in the previous step for each batch of data; it then extracts all unique source host IP addresses and destination host IP addresses from the current batch of data, establishing a bidirectional mapping relationship between IP addresses and node IDs, meaning each unique IP address corresponds to a node in the graph. Ensuring a one-to-one correspondence between IP addresses and node IDs facilitates subsequent graph structure operations and feature associations, thus obtaining the node set V.

[0085] Next, construct the adjacency list: traverse each traffic record in the current batch, and for each record's source host IP and destination host IP, establish undirected edges between the corresponding nodes; use an adjacency list (dictionary type) to store the graph's connectivity, where the key is the node ID and the value is a list of all neighbor node IDs for that node; during construction, avoid adding duplicate edges to ensure that each edge is recorded only once in the adjacency list, thus obtaining the edge set E. At this point, the communication graph G = (V, E) is constructed.

[0086] Finally, the graph attributes are calculated, including the total number of nodes, the total number of edges, the average degree (the average degree of all nodes), and the connectivity (the proportion of connected nodes to the total number of nodes). These attributes are used for subsequent communication graph quality assessment and model parameter adjustment (such as the adaptation of the number of sampling layers and the number of samples). Thus, the spatial modal features have been constructed.

[0087] S303. Extract packet-level temporal features, flow-level statistical features, and TCP header flag features for each node, and perform standardization processing on each.

[0088] This embodiment uses packet-level temporal features and flow-level statistical features as temporal modal features. Temporal modal features are used to represent the temporal distribution patterns, transmission modes, and overall behavior of network traffic. The calculation method is as follows:

[0089] First, the packet-level timing features in the time modality are extracted based on the timestamp information of the data packets. These features are used to capture the temporal distribution patterns and transmission modes of network traffic. The extracted packet-level timing features mainly include: Inter-Arrival Time (IAT) and Packet Size, as explained below:

[0090] 1) The inter-packet arrival time interval is defined as the time difference between the arrival of two adjacent data packets in the same network flow. The calculation formula is: IAT(i) = timestamp i -timestamp i-1 timestamp i The timestamp represents the timestamp of the i-th data packet. i-1 This indicates the timestamp of the previous data packet.

[0091] 2) The packet size can be directly obtained from the packet_size field in the dataset.

[0092] Secondly, the flow-level statistical characteristics in the time modality are obtained by aggregating all data packets in the same network flow. These characteristics describe the overall behavior of the flow. A network flow is defined based on a 5-tuple (source IP address, destination IP address, source port, destination port, protocol type). Let a flow contain n data packets, and its time series be denoted as {p1, p2, ..., p...}. n Then we can obtain the following characteristics:

[0093] 1) The number of packets (packet_count) is defined as the total number of packets contained in the stream: packet_count = n.

[0094] 2) The total number of bytes in the stream is defined as the total number of bytes in all data packets in the stream. Let the length of the i-th data packet be b. i The total number of bytes in the stream is

[0095] 3) Flow duration is defined as the time span from the first packet to the last packet in the flow: flow_duration = timestamp n -timestamp1,timestamp n This represents the timestamp of the nth data packet, while timestamp1 represents the timestamp of the first data packet.

[0096] In this embodiment, the transport layer TCP header flags multiplied by the feature contribution weight are selected as the event modality feature because the transport layer header fields are discrete protocol-type or flag-type features. These information reflect the protocol layer and interaction behavior of the traffic. In different network environments, the distribution of transport layer features of different types of devices and traffic behaviors is significantly different.

[0097] The TCP header flags are denoted as tcp_flag, specifically including the FIN, SYN, RST, PSH, ACK, URG, ECE, and CWR flag fields. The eight standard TCP flags are statistically analyzed, including a flag count (flag_count) and a flag ratio (flag_ratio), calculated as follows:

[0098] First, let the number of packets in a stream be packet_count = n, and let the value of a certain TCP flag (such as SYN, ACK, FIN) corresponding to the i-th packet be denoted as tcp_flag. flag,i ∈{0,1}, where 0 indicates that the bit is not set and 1 indicates that the bit is set.

[0099] 1) Flag count (flag_count) is defined as the number of packets in the stream in which this flag is set.

[0100]

[0101] 2) The ratio of this flag bit (flag_ratio) is:

[0102]

[0103] Secondly, for each TCP flag (FIN, SYN, RST, PSH, ACK, URG, ECE, CWR), the proportion of the flag in each flow is read from the corresponding data column (flag_ratio). The proportion value is safely converted to a floating-point number using the safe_float function, handling null values, hexadecimal strings, and outliers. Then, the arithmetic mean of the proportion values ​​of the flag in all flow records of the IP address is calculated to obtain the average proportion of the flag on the node. Finally, a dictionary structure containing the proportions of 8 flags is formed, where the dictionary key is the flag name string and the value is the corresponding average proportion floating-point number.

[0104] Then, the importance weights of the flags are determined. This embodiment uses ablation experiments to evaluate the contribution of each feature to the classification performance. First, a baseline model using only temporal modality features is established. Then, by systematically removing or combining TCP flag features, the independent contribution and removal effect of each TCP feature are quantified. The importance score is calculated by the arithmetic mean of the removal effect and the individual contribution, where the removal effect is defined as the difference between the accuracy of the full feature set and the accuracy after removing the feature, and the individual contribution is defined as the accuracy improvement when using only the feature. The comprehensive importance score is calculated by combining the feature removal effect and the individual contribution, resulting in the importance weights of the eight TCP flags, with a total weight of 1.0. The TCP flag features are weighted and encoded. This weighting mechanism allows the encoded vector to better learn the differences in the discriminative ability of different TCP flags for traffic classification tasks during subsequent neural network training. At the same time, a mapping dictionary from TCP flag names to indices is established, mapping the eight flags sequentially to integer indices from 0 to 7 for subsequent vectorization operations.

[0105] Finally, after weighted encoding, the system standardizes the encoded vector. Standardization eliminates the scaling effect caused by differences in the weights of different flags, making the encoded vector more uniform in numerical distribution, which is beneficial for subsequent neural network training and feature fusion operations. The standardized encoded vector is converted into PyTorch tensor format. In addition, the previously extracted temporal modalities (packet-level temporal features + stream-level statistical features) are also standardized, so that the mean of the two types of features is 0 and the standard deviation is 1, eliminating the dimensional differences between different feature dimensions and providing a data foundation for subsequent feature alignment and fusion.

[0106] So far, a total of three traffic feature modal information have been obtained: spatial modal features (communication graph), temporal modal features (packet-level timing features + flow-level statistical features), and event modal features (TCP header flag statistical feature encoding). The difference from the BOS mechanism is that the modal features in the BOS mechanism only include packet-level temporal modal information, while the modal features in this embodiment include temporal modality, spatial modality, and event modality.

[0107] S304. Align the time modal features and the event modal features.

[0108] The design goal of feature alignment is to map heterogeneous traffic features (e.g., temporal mode A and event mode B) from different measurement domains and statistical distributions to a shared representation space (A*, B*), thereby eliminating scale and distribution differences between modes and facilitating subsequent fusion and communication graph aggregation. The feature alignment and feature fusion processes are as follows: Figure 4 As shown.

[0109] Feature alignment is based on linear transformation projection, which maps temporal modal features and weighted encoded event modal features to a unified feature space and performs normalization processing; specifically, it uses a learnable matrix W s W t Each modality is projected onto the same target dimension, and batch / layer normalization, Dropout regularization, nonlinearity, and L2 normalization are applied after projection to ensure numerical stability. The calculation formula is as follows:

[0110]

[0111] Among them, X s X t W represents the time modality feature matrix and the event modality feature matrix, respectively. s W t For the learnable weight matrix, b s ,b t The bias term, φ, can be omitted or replaced with an identity mapping, depending on the implementation details. f(X) represents the linear projection function. The vector is the embedding vector after projection; LayerNorm represents the layer normalization operation, L2. norm This represents the L2 norm normalization operation; A s A t This represents the final aligned feature vector. After alignment of different modal features by linear projection, layer normalization is first used to maintain their own distribution characteristics, and then L2 norm normalization is used to make the feature vectors at the same scale, ensuring that the features are comparable in a unified space.

[0112] S305. Fuse the aligned temporal modal features and event modal features.

[0113] The time modality feature vector and event modality feature vector at the i-th node after feature alignment are respectively represented as follows: The purpose of feature fusion is to map two modalities, after feature alignment, into a unified and discriminative high-dimensional representation. The fused features are called bimodal fusion features z. i It is the bimodal fusion feature of the i-th node. In order to achieve effective integration of cross-modal information, this embodiment adopts five independently selectable fusion strategies, covering concat, add, multiply, attention, and gating.

[0114] The first method is a concatenation and fusion approach: Aligned temporal modality features and aligned event modality features are concatenated along their feature dimensions, forming a concatenated feature with a dimension twice that of the alignment dimension. Then, a sequence network containing linear transformation layers, batch normalization layers, ReLU activation function layers, and Dropout pruning layers maps the concatenated feature to the fused feature, outputting a dual-modality fused feature z. i The calculation formula is as follows:

[0115]

[0116] Where [a||b] represents vector concatenation, W c For learnable weights, b c The bias term is φ(·), which is a combination of nonlinear activation (ReLU), batch normalization (BatchNorm), and dropout layers. The fused output vector is z. i .

[0117] The second method is the additive fusion method: the aligned temporal modality features and the aligned event modality features are added element-wise, requiring that the alignment dimension of the two types of features must be equal to the dimension of the fused output; the added features are processed through a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output the bimodal fused feature z. i The calculation formula is as follows:

[0118]

[0119] The third method is the multiplication fusion method: the aligned temporal modality features and the aligned event modality features are multiplied element-wise, requiring the alignment dimension to equal the output dimension; the multiplied features are then processed through batch normalization layers, ReLU activation function layers, and Dropout layers to output the dual-modal fusion feature z. i The calculation formula is as follows:

[0120]

[0121] The fourth method is attention fusion: using temporal modality features as the query and event modality features as the (Key / Value), a multi-head attention mechanism is used to calculate the attention weights of temporal modality features on event modality features, resulting in a weighted event modality feature representation Attn. i Then, the time modal features and weighted event modal features Attn i The data is concatenated and mapped through a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout layer to the fused data, outputting a bimodal fusion feature z. i The calculation formula is as follows:

[0122]

[0123] The fifth method is the gated fusion method: For temporal modality features and event modality features, corresponding gated weight vectors are generated through a gated network containing a linear transformation layer and a sigmoid activation function layer, respectively. Multiply the temporal modality features by their gating weights to obtain the gated temporal modality features. Multiply the event modality features by their gating weights to obtain the gated transport layer features. The two types of gated features are concatenated and mapped to the fused feature through a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout layer, outputting the bimodal fused feature z. i The calculation formula is as follows:

[0124]

[0125] S306. For the communication graph, perform multi-layer neighbor sampling to determine the Q-layer neighbor nodes.

[0126] S307. Assign bimodal fusion features to each node, perform hierarchical aggregation using a graph neural network model to obtain structurally enhanced node features containing multi-hop neighbor information, and iteratively train to obtain a classification model.

[0127] This embodiment utilizes a graph neural network model (e.g., GraphSAGE) to perform multi-layer neighbor sampling and hierarchical aggregation on the communication graph, enabling the fused node features to propagate multi-hop within the graph structure, thereby obtaining cross-node interaction information and local topology patterns. Ultimately, the output after sampling and aggregation simultaneously includes packet and flow-level temporal modes, transport layer header flag modes, and communication graph modalities. The neighbor sampling mechanism ensures the model's scalability and online inference capabilities in large-scale networks, while the aggregation mechanism enables recursive inference from raw traffic behavior to network structure behavior, ultimately achieving high-precision classification of network traffic.

[0128] 1) Multi-level neighbor sampling

[0129] The neighboring nodes of each node are determined by the generated communication graph G = (V, E). In the communication graph, each node v ∈ V corresponds to a unique network entity (IP address), and the edge set E = {(u, v) | there is traffic interaction between u and v}. Node u is a neighbor of node v. From the communication graph, the neighbor set of node v can be represented as N(v) = {u | (v, u) ∈ E}.

[0130] Multi-level neighbor sampling process: First, the first-level sampling of node v yields: S1 represents the number of sampling nodes in the first layer. The neighboring nodes of each node are determined and are called the first-layer neighboring nodes. Next, the neighbors of each u sampled in the first layer are sampled again to obtain the second-layer sampling. S2 represents the number of sampling nodes in the second layer. The neighboring nodes of the neighboring nodes are determined and called the second-layer neighboring nodes. Subsequent sampling in other layers follows the same pattern, repeating Q times to determine the Q-layer neighboring nodes.

[0131] 2) Layered aggregation

[0132] The purpose of hierarchical aggregation is to fuse the features of the node itself and its neighbors at different levels to form a node representation with enhanced structure. The new feature representation of the node is obtained by re-aggregating and updating the aggregation results of the neighbor features and its own features, and then by the activation function. Then, it is iterated in multiple layers to fuse the information of multiple layers of neighbors at the same time.

[0133] The vector representation of node v after multi-level sampling and aggregation at the k-th layer is as follows In this context, the bimodal fusion features of node v after fusion in step (4) are used as the input vector of layer 0 of z, that is, AGG represents the aggregation function, W represents the learnable weight matrix, and σ represents the activation function. Assuming GraphSAGE's default aggregation method (mean aggregation) is used, the calculation of hierarchical aggregation is as follows:

[0134] First-level aggregation:

[0135] Second-level aggregation:

[0136] k-th level aggregation:

[0137] After k-level aggregation, we obtain Then, Input a linear classifier, output class probabilities, and select the class with the highest probability as the prediction result:

[0138]

[0139] Among them, W cls Let b represent the weight matrix that the classifier can learn. cls This indicates the classifier bias.

[0140] Common metrics for evaluating the performance of classification models include accuracy, recall, precision, and F1 score, which are calculated as follows:

[0141]

[0142] Wherein, TP represents the number of cases where both the prediction and the actual value are positive; FN represents the number of cases where the prediction is negative but the actual value is positive; FP represents the number of cases where the prediction is positive but the actual value is negative; and TN represents the number of cases where both the prediction and the actual value are negative.

[0143] The steps S301-S307 above constitute the offline training phase, and the following step S308 includes the deployment phase and the online inference phase.

[0144] S308. The trained classification model is simplified in structure and weighted, the classification model parameters are converted into fixed-point format and decomposed into multi-level flow table matching actions, which are mapped to the data plane pipeline of the programmable switch so that the programmable switch processes the data packets passing through it, extracts features and inputs them into the classification model. The linear classifier outputs the class probability, and the class with the highest probability is selected as the prediction result, realizing linear online inference of data packets.

[0145] After the classification model is trained offline, to achieve real-time inference of network traffic, the trained model needs to be mapped to the data plane of a P4 programmable switch. The control plane of the P4 programmable switch needs to be developed to implement functions such as graph structure management, model weight distribution, and inference result caching. The P4Runtime API is used to communicate with the data plane to implement CRUD operations on table entries. The specific process is as follows:

[0146] First, the data plane performs real-time feature extraction and simplified neighbor feature aggregation. Data features are maintained and updated through P4's metadata and registers. In this embodiment, the neighbor list of each IP node is maintained in the register array, and the neighbor information is updated in real time by the data plane logic when a new flow or change in communication relationship is detected. The simplified neighbor aggregation process is fixed to the average value operation of the features of a limited number of neighbors. Register read and write and addition accumulation are used to achieve a local aggregation result that approximates GraphSAGE.

[0147] Secondly, model compression and structural simplification: Since the data plane of a programmable switch is limited by the number of pipeline stages, register capacity, and fixed-point arithmetic capabilities, this embodiment first performs structural and operator-level compression on the trained model. The original multi-layer GraphSAGE aggregation process is converged into a single-layer linear mapping that only depends on the embedding vector of the final node during the inference stage. That is, the graph structure propagation part is fixed as the offline computation result of the training stage, and only the final node representation is retained to participate in the data plane inference. This allows the data plane to only perform forward computation of a low-complexity linear classifier. At the same time, the parameters of the fully connected layer are weighted and quantized, and the 32-bit floating-point weights are converted into a fixed-point expression format supported by the switch registers to ensure that the parameters can be correctly parsed and manipulated within the data plane arithmetic unit.

[0148] Then, the weight matrix is ​​decomposed and a lookup table is constructed: After model quantization, this invention decomposes the weight matrix W based on the input feature dimension Q1 (determined by the fused time modality and event modality) and the output dimension Q2 (determined by the simplified linear classifier). cls It is decomposed into multiple dot product operators that can be executed pipeline by pipeline in the P4 data plane; specifically, the weight row corresponding to each output dimension is decomposed into several sub-weight segments, and for each sub-vector and sub-weight segment, a product lookup table entry for the entire input space is pre-generated in the control plane to form a corresponding lookup table, which is placed in the match-action table.

[0149] Finally, the weights are stored and mapped to the P4 pipeline. To ensure the model can execute at line speed in the P4 switch, this invention stores each sub-entry in the lookup table structure in a set of entries that can be maintained by the control plane, and the corresponding weight segments are accessed by the P4 program through a match-action mechanism. The quantized weights are divided into multiple register arrays and constant tables, and are loaded by the control plane when the switch starts. The data plane action units set multiple accumulation operations according to the number of output dimensions Q2. Each operation accesses the register segment and reads the lookup result in turn, multiplies and accumulates the segmented sub-weights into the intermediate value field in the metadata, and completes the evaluation of all output dimensions without increasing processing latency through a pipelined execution mechanism. Finally, all output scores will be compared with the maximum value argmax in the pipeline to obtain the classification label.

[0150] After deploying the model on the P4 programmable switch, run the switch. Network traffic is replayed from the pcap file using the tcpreplay tool on the host side, or a custom test package is generated using the scapy library. During runtime, the data plane processes the received data packets through a pipeline, performing a complete inference process including feature extraction, feature alignment, feature fusion, multi-level sampling aggregation, and final classification. The classification results are passed to the control plane by reading register storage, and the results are parsed and saved to a CSV file.

[0151] Performance testing measures the system's throughput, latency, and resource utilization. Throughput testing involves gradually increasing the transmission rate and observing the system's processing capacity. Latency testing measures the time difference between the arrival of a data packet at the switch and the output of the classification result. Resource utilization (such as TCAM and SRAM utilization) testing measures the number of lookup table entries, register usage, and memory usage metrics.

[0152] The network traffic classification method in this embodiment has the following technical advantages:

[0153] (1) Strong model expressive power: Through joint modeling of spatial modality, temporal modality and event modality, the model can deeply understand the overall pattern of traffic behavior, especially in multi-hop communication relationship and cooperative attack scenario.

[0154] (2) Make full use of graph structure information: GraphSAGE captures network topology and node relationships, and adopts a hierarchical sampling method, which makes the model more information-rich and better than the method that only uses a single flow feature.

[0155] (3) High scalability: It supports batch processing of datasets, which reduces computational complexity; the multi-layer neighbor sampling mechanism can avoid exponential expansion of the neighborhood, enabling the model to adapt to large-scale network data; the present invention supports five feature fusion mechanisms, which can adapt to more heterogeneous data.

[0156] (4) Good interpretability: The present invention evaluates the contribution of different modal characteristics in advance through the ablation experimental system, making the calculation process more intuitive and interpretable.

[0157] The network traffic classification method in this embodiment has the following performance advantages:

[0158] (1) High multi-class accuracy: The reasoning ability is excellent in categories such as VPN, Chat, and Streaming on the ISCXVPN2016 dataset, with an average accuracy significantly better than traditional methods.

[0159] (2) Good generalization performance: The generalization ability of the model of this invention is improved by feature alignment and fusion.

[0160] (3) High computational efficiency: The multi-layer sampling mechanism of graph neural networks reduces computational complexity and supports real-time processing.

[0161] (4) Strong real-time performance: By deploying the control plane and data plane in a coordinated manner, the high latency problem of the traditional network traffic analysis system of "data collection → reporting to the server → offline inference" is solved, enabling the model to perform traffic analysis in real time when the packet arrives.

[0162] It should be noted that although the method operations of the above embodiments are described in a specific order, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the described steps may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0163] Example 2:

[0164] This embodiment provides a network traffic classification method based on multimodal feature fusion. This method separates and extracts temporal modal features and transport layer flag features, employs feature alignment and fusion mechanisms, and combines multi-layer GraphSAGE for graph structure learning. It achieves accurate classification of multiple categories of network traffic in both offline and online modes. The specific implementation steps are as follows:

[0165] (1) Data Preprocessing: This case study uses the ISCXVPN2016 public dataset as the experimental data source. In the field of network traffic classification, the ISCXVPN2016 dataset is a common public dataset for network traffic. This dataset contains network traffic packet capture files (PCAP format) under VPN environments. The original dataset covers six typical network application scenarios, including Chat, VoIP, Streaming, File Transfer, Email, and P2P. To extract features usable for machine learning classification from the original network packets, a protocol header information extraction system was implemented based on the Scapy library in Python. The core function of this system is to parse and extract key network layer and transport layer fields from the PCAP file and save them as a CSV file. Load the ISCX-VPN dataset CSV file with a batch size of 300 records; read network flow data from the CSV file, with the data format including source IP address, destination IP address, number of packets, flow duration, total bytes, average packet size, average packet interval, and the proportion of 8 TCP flags (FIN, SYN, RST, PSH, ACK, URG, ECE, CWR); for each data field, use the safe_float function for safe type conversion, handle hexadecimal strings, null values, and outliers to ensure data quality.

[0166] (2) Communication graph construction: Use the NetworkGraphBuilder function to construct an undirected IP communication graph; map each unique IP address in each batch of data to a graph node, with node IDs numbered consecutively starting from 0; for each flow record, establish an undirected edge between the source IP node and the destination IP node, and store the neighbor list of each node through the adjacency list adj_lists to avoid duplicate edges; calculate the graph size parameters, including the total number of nodes, the total number of edges, the average degree value (2× number of edges / number of nodes), and the proportion of connected nodes, to evaluate the complexity and connectivity of the graph structure.

[0167] (3) Feature Extraction: Basic features are extracted, including degree, number of packets, stream duration, total number of bytes, average packet size, and average packet interval. The count and proportion features of eight transport layer flags (FIN, SYN, RST, PSH, ACK, URG, ECE, CWR) are extracted and standardized respectively. Feature extraction is divided into two categories: temporal modality features and event modality features, stored as 6-dimensional and 8-dimensional feature vectors respectively. The basic features and TCP features are standardized using the StandardScaler function and converted to PyTorch tensor format, while maintaining the same dimensionality.

[0168] (4) Feature Alignment: The FeatureAlignment module is used to align temporal features and transport layer features into a unified semantic space. The alignment method is explicit alignment, implemented through two independent neural network sequences: the basic feature aligner includes a linear transformation layer (input dimension 6, output dimension 32), a batch normalization layer, a ReLU activation function, and a Dropout layer (Dropout rate 0.1), while the TCP feature aligner has the same structure (input dimension 8, output dimension 32). The aligned features are normalized by a LayerNorm layer, followed by L2 normalization to ensure that the two modalities are in a unified space. The aligned features are then fused using FeatureFusionLayer.

[0169] (5) Feature fusion: The aligned features are fused using the Concat method. The aligned 32-dimensional temporal features and the aligned 32-dimensional transport layer features are concatenated in the feature dimension to form a 64-dimensional fused feature vector. The 64-dimensional features are mapped to the output dimension of 128 dimensions through a linear transformation layer. This transformation layer includes batch normalization, ReLU activation and Dropout regularization.

[0170] (6) GraphSAGE training:

[0171] First, perform two layers of neighbor sampling for the training nodes: use the MultiLayerNeighborSampler module to implement multi-layer neighbor sampling; the sampling configuration is two layers, with each layer sampling 5 neighbors, i.e., sample_sizes = [5,5]; for each target node, the first layer randomly samples 5 nodes from its direct neighbors, and the second layer samples 5 more nodes from the neighbors of the nodes sampled in the first layer.

[0172] Secondly, a mean aggregator is constructed: the target node's own features are mapped to the output dimension through a linear transformation layer, the average of the sampled neighbor features is calculated on the neighbor dimension to obtain the neighbor aggregator features, and then mapped to the output dimension through a linear transformation layer. The transformed own features and neighbor features are concatenated on the feature dimension, and the concatenated features are mapped to the output dimension through a combined linear transformation layer. Dropout regularization is applied (Dropout rate 0.1).

[0173] Then, a multi-layer GraphSAGE model is constructed: the input dimension is the feature dimension of the previous feature fusion, which is 128 dimensions, the hidden layer dimension is set to [128, 32], and the output dimension is equal to the number of classification categories, which is 6. The model contains two GraphSAGE layers and a linear classifier. Each GraphSAGE layer contains a mean aggregator and a ReLU activation function.

[0174] Finally, the training parameters were set as follows: the Adam optimizer was used with a learning rate of 0.01 and a weight decay of 1e-4; the weighted cross-entropy loss function was adopted, and the class weights were calculated by inverse frequency weighting, that is, the weight of each class is the reciprocal of the number of samples in that class (adding 1e-6 to avoid division by zero), and then normalized so that the sum of the weights equals the number of classes; the number of training epochs was set to 100; stratified sampling was used to divide the training set, validation set and test set, with the test set accounting for 30% and the validation set accounting for 20%, to ensure that the distribution of samples in each class is balanced.

[0175] (7) Performance Evaluation: Evaluate classification performance on the test set. Calculate accuracy, precision, recall, and F1 score. Generate a confusion matrix and performance reports for each class.

[0176] Experiments on the ISCX-VPN dataset in this embodiment demonstrate that the proposed method achieves significant performance in six traffic classification tasks, with an average accuracy of 91.56%. It performs particularly well in VoIP, FTP, and Streaming categories, achieving 91.57%, 91.39%, and 95.38%, respectively. Furthermore, the contribution of transport layer features to classification performance was analyzed, revealing that FIN, SYN, and PSH contribute significantly to the results, enhancing the model's interpretability. Experimental results prove that the proposed enhanced GraphSAGE method has application value in network traffic classification, network security monitoring, and traffic anomaly detection.

[0177] Example 3:

[0178] like Figure 5As shown, this embodiment provides a network traffic classification device based on multimodal feature fusion. The device includes a network traffic data acquisition module 501, a communication graph construction module 502, a bimodal feature extraction module 503, a bimodal feature alignment module 504, a bimodal feature fusion module 505, a multi-layer neighbor sampling module 506, a hierarchical aggregation module 507, and an online inference module 508. Specific descriptions of each module are as follows:

[0179] The network traffic data acquisition module 501 is used to acquire network traffic data;

[0180] The communication graph construction module 502 is used to extract source host IP and destination host IP addresses from network traffic data as nodes to construct a communication graph, which serves as a spatial modal feature.

[0181] The dual-modal feature extraction module 503 is used to extract packet-level temporal features, flow-level statistical features, and TCP header flag features for each node, and perform standardization processing on them respectively. The packet-level temporal features and flow-level statistical features are used as temporal modal features, and the TCP header flag features are used as event modal features.

[0182] The dual-modal feature alignment module 504 is used to align temporal modal features and event modal features;

[0183] The dual-modal feature fusion module 505 is used to fuse the aligned temporal modal features and event modal features;

[0184] The multi-layer neighbor sampling module 506 is used to perform multi-layer neighbor sampling on the communication graph to determine the Q-layer neighbor nodes;

[0185] The hierarchical aggregation module 507 is used to attach dual-modal fusion features to each node, perform hierarchical aggregation using a graph neural network model, obtain structurally enhanced node features containing multi-hop neighbor information, and iteratively train to obtain a classification model.

[0186] The online inference module 508 is used to simplify the structure and quantize the weights of the trained classification model, convert the classification model parameters into a fixed-point format, and decompose them into multi-level flow table matching actions, which are then mapped to the data plane pipeline of the programmable switch. This enables the programmable switch to process the data packets passing through it, extract features, and input them into the classification model. The linear classifier outputs the class probability, and the class with the highest probability is selected as the prediction result, thus realizing linear-speed online inference of data packets.

[0187] The specific implementation of each module in this embodiment can be found in Embodiment 1 above, and will not be repeated here. It should be noted that the device provided in this embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0188] Example 4:

[0189] This embodiment provides a computer device, such as... Figure 6 As shown, it includes a processor 602, a memory, and a network interface 603 connected via a system bus 601. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium 604 and internal memory 605. The non-volatile storage medium 604 stores an operating system, computer programs, and a database. The internal memory 605 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the processor 602 executes the computer programs stored in the memory, it implements the network traffic classification method of Embodiment 1 described above, as follows:

[0190] The process involves: acquiring network traffic data; extracting source and destination host IP addresses as nodes from the network traffic data to construct a communication graph, which serves as the spatial modal feature; extracting packet-level temporal features, flow-level statistical features, and TCP header flag features for each node, and standardizing them accordingly; aligning the temporal and event modal features; fusing the aligned temporal and event modal features; performing multi-layer neighbor sampling and hierarchical aggregation on the communication graph using a graph neural network model to obtain node features containing multi-hop neighbor information; inputting these node features into a linear classifier to output class probabilities; selecting the class with the highest probability as the prediction result; iteratively training the model to obtain a classification model; and mapping the trained classification model onto the data plane of a programmable switch to enable online inference of data packets using the classification model.

[0191] Example 5:

[0192] This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the program is executed by a processor, the processor executes the computer program stored in the memory to implement the network traffic classification method of Embodiment 1 above, as follows:

[0193] The process involves: acquiring network traffic data; extracting source and destination host IP addresses as nodes from the network traffic data to construct a communication graph, which serves as the spatial modal feature; extracting packet-level temporal features, flow-level statistical features, and TCP header flag features for each node, and standardizing them accordingly; aligning the temporal and event modal features; fusing the aligned temporal and event modal features; performing multi-layer neighbor sampling and hierarchical aggregation on the communication graph using a graph neural network model to obtain node features containing multi-hop neighbor information; inputting these node features into a linear classifier to output class probabilities; selecting the class with the highest probability as the prediction result; iteratively training the model to obtain a classification model; and mapping the trained classification model onto the data plane of a programmable switch to enable online inference of data packets using the classification model.

[0194] It should be noted that the computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0195] In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used or combined with an instruction execution device, apparatus, or device. In this embodiment, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use or combined with an instruction execution device, apparatus, or device. The computer program contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0196] The computer-readable storage medium described above can be used to write computer programs for executing this embodiment in one or more programming languages ​​or combinations thereof. These programming languages ​​include object-oriented programming languages—such as Java, Python, and C++—and conventional procedural programming languages—such as C or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0197] In summary, this invention achieves high-precision, real-time network traffic classification by constructing a communication graph to capture node topology relationships, designing heterogeneous feature alignment, mining multimodal feature value through a fusion mechanism, optimizing the GraphSAGE sampling and aggregation strategy, and deploying online line-speed inference collaboration.

[0198] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A network traffic classification method based on multimodal feature fusion, characterized in that, The method includes: Obtain network traffic data; Source host IP and destination host IP address are extracted from network traffic data as nodes to construct a communication graph, which serves as a spatial modal feature. For each node, packet-level temporal features, flow-level statistical features, and TCP header flag features are extracted and standardized respectively. The packet-level temporal features and flow-level statistical features are used as time modal features, and the TCP header flag features are multiplied by the feature contribution weight to become event modal features. Align the temporal modality features and the event modality features; The aligned temporal modal features and event modal features are fused to output dual-modal fused features; For the communication graph, perform multi-layer neighbor sampling to determine Q-layer neighbor nodes; Each node is given a dual-modal fusion feature. A hierarchical aggregation is performed using a graph neural network model to obtain structurally enhanced node features containing multi-hop neighbor information. The classification model is then iteratively trained. The trained classification model is structurally simplified and its weights are quantized. The classification model parameters are converted into a fixed-point format and decomposed into multi-level flow table matching actions, which are then mapped to the data plane pipeline of the programmable switch. This enables the programmable switch to process data packets passing through it, extract features, and input them into the classification model. The linear classifier outputs the class probability, and the class with the highest probability is selected as the prediction result, thus achieving linear-speed online inference of data packets.

2. The network traffic classification method according to claim 1, characterized in that, The step of extracting source host IP and destination host IP addresses from network traffic data as nodes to construct a communication graph includes: Extract the five-tuple from each batch of network traffic data, extract all unique source host IP addresses and destination host IP addresses from the current batch of data, establish a bidirectional mapping relationship between IP addresses and node IDs, and obtain a one-to-one corresponding set of nodes. Iterate through each traffic record in the current batch. For the source host IP and destination host IP of each record, establish undirected edges between the corresponding nodes. Use an adjacency list to store the connection relationship of the graph, ensuring that each edge is recorded only once in the adjacency list, and obtain the edge set. Calculate graph properties, including total number of nodes, total number of edges, average degree, and connectivity.

3. The network traffic classification method according to claim 1, characterized in that, The packet-level timing features are extracted based on the timestamp information of the data packets; the flow-level statistical features are obtained by aggregating all data packets in the same network flow; the TCP header flag features include flag count and flag ratio. Based on the pre-conducted feature importance ablation experiment, the contribution weight of each TCP header flag feature is calculated. The contribution weight vector is determined based on the difference in the impact of each flag bit on the classification accuracy. The flag bit proportion is weighted and encoded using the contribution weight, that is, the contribution weight vector is multiplied element by element by the flag bit proportion to generate an event modality feature vector that retains the difference in importance.

4. The network traffic classification method according to claim 1, characterized in that, Aligning the temporal modal features and event modal features includes: mapping the temporal modal features and the weighted encoded event modal features to a unified feature space based on linear transformation projection, and performing normalization processing.

5. The network traffic classification method according to claim 1, characterized in that, The process of fusing the aligned temporal modality features and event modality features includes: The aligned temporal modality features and aligned event modality features are concatenated along the feature dimension to form a concatenated feature with a dimension twice that of the alignment dimension. The concatenated feature is then mapped to the fused feature through a sequence network containing a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout pruning layer, outputting a dual-modality fused feature. Alternatively, the aligned temporal modality features and the aligned event modality features can be added element-wise; the added features can then be processed through a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output dual-modal fusion features. Alternatively, the aligned temporal modality features and the aligned event modality features can be multiplied element-wise; the multiplied features can then be processed through a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output dual-modal fusion features. Alternatively, a multi-head attention mechanism can be used to calculate the attention weights of temporal modality features on event modality features, resulting in weighted event modality features; the temporal modality features The weighted event modal features are concatenated and mapped to the fused features through a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output the dual-modal fusion features. Alternatively, for temporal modality features and event modality features, corresponding gating weights are generated by passing them through a gating network containing a linear transformation layer and a sigmoid activation function layer, respectively. The temporal modality features are multiplied by the corresponding gating weights to obtain the gated temporal modality features. The event modality features are multiplied by the corresponding gating weights to obtain the gated event modality features. The two types of gated features are concatenated and mapped to the fusion layer through a linear transformation layer, a batch normalization layer, a ReLU activation function layer, and a Dropout layer to output the dual-modality fused features.

6. The network traffic classification method according to claim 1, characterized in that, The multi-layer neighbor sampling refers to determining the neighbor nodes of each node through the communication graph, which are called first-layer neighbor nodes. The neighbor nodes of the neighbor nodes are called second-layer neighbor nodes. This process is repeated Q times to determine Q layers of neighbor nodes. The hierarchical aggregation refers to fusing the features of each node and its neighboring nodes at different layers, including: The vector representation of node v after multi-level sampling and aggregation at the k-th layer is as follows In this context, the bimodal fusion features of node v serve as the input vector for layer 0 of z, i.e. The calculation for stratified aggregation using mean aggregation is as follows: First-level aggregation: Second-level aggregation: k-th level aggregation: Where AGG represents the aggregation function, W represents the learnable weight matrix, and σ represents the activation function; After k-level aggregation, we obtain Then, Input a linear classifier, output class probabilities, and select the class with the highest probability as the prediction result: Among them, W cls Let b represent the weight matrix that a linear classifier can learn. cls This indicates the bias of the linear classifier.

7. The network traffic classification method according to claim 1, characterized in that, The process of simplifying the structure and quantizing the weights of the trained classification model, converting the classification model parameters into a fixed-point format, and decomposing them into multi-level flow table matching actions, which are then mapped to the data plane pipeline of the programmable switch, includes: A register array is constructed in the data plane of the programmable switch to maintain and update the neighbor list and neighbor features of nodes in real time. The parameters of the fully connected layers in the trained classification model are quantized to fixed point, converting the 32-bit floating-point weights into a fixed-point format supported by the programmable switch registers. Based on the input feature dimension and the output dimension, the weight matrix of the linear classifier is decomposed into multiple dot product operators, and a pre-computed product lookup table is generated for each dot product operator. The product lookup table is stored in the matching action table of the programmable switch. The metadata of the data packet is used as the intermediate result carrier. The weight segment is accessed through the matching-action mechanism, and the multiplication operation between the feature vector and the weight matrix is ​​completed through the accumulation operation in the data plane pipeline. The predicted category is output through the comparison instruction.

8. A network traffic classification device based on multimodal feature fusion, characterized in that, The device includes: The network traffic data acquisition module is used to acquire network traffic data. A communication graph construction module is used to extract source host IP and destination host IP addresses from network traffic data as nodes to construct a communication graph, which serves as a spatial modal feature. The dual-modal feature extraction module is used to extract packet-level temporal features, flow-level statistical features, and TCP header flag features for each node, and perform standardization processing on each. The packet-level temporal features and flow-level statistical features are used as temporal modal features, and the TCP header flag features are used as event modal features. A dual-modal feature alignment module is used to align temporal modal features and event modal features; The dual-modal feature fusion module is used to fuse aligned temporal modal features and event modal features; The multi-layer neighbor sampling module is used to perform multi-layer neighbor sampling on the communication graph to determine the Q-layer neighbor nodes; The hierarchical aggregation module is used to attach bimodal fusion features to each node. It uses a graph neural network model to perform hierarchical aggregation, obtains structurally enhanced node features that include multi-hop neighbor information, and iteratively trains to obtain a classification model. The online inference module is used to simplify the structure and quantize the weights of the trained classification model, convert the classification model parameters into a fixed-point format, and decompose them into multi-level flow table matching actions. These actions are then mapped to the data plane pipeline of the programmable switch, enabling the programmable switch to process data packets passing through it, extract features, and input them into the classification model. The linear classifier outputs the class probability, and the class with the highest probability is selected as the prediction result, thus achieving linear-speed online inference for data packets.

9. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the network traffic classification method according to any one of claims 1-7.

10. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the network traffic classification method according to any one of claims 1-7.

Images