Lightweight malicious network traffic detection method based on heterogeneous modal feature fusion

Abstract

The invention discloses a lightweight malicious network traffic detection method based on heterogeneous modal feature fusion, and mainly solves the problems of low feature extraction efficiency and insufficient single modal feature representation of the existing method. Comprising the following steps: acquiring and optimizing a network flow data set, preprocessing the network flow data set, and extracting and generating spatial feature, time sequence feature and behavior pattern feature vectors; an improved self-attention mechanism network is constructed, a lightweight heterogeneous modal feature fusion model LMF is designed, multi-modal feature deep fusion is performed through dynamic weight distribution, a lightweight classification model is trained, and the model is utilized to detect network malicious traffic. Through the lightweight heterogeneous modal feature fusion model LMF and in combination with a dynamic weight distribution mechanism, spatial distribution, time sequence dependence and behavior semantic information of network traffic are deeply mined, data processing efficiency, malicious traffic detection accuracy and system robustness are improved, and the method is suitable for efficient malicious traffic identification and defense in the field of network security.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology and further relates to malicious network traffic detection. Specifically, it is a lightweight malicious network traffic detection method based on heterogeneous modal feature fusion, which is used for identifying and defending against network attack behaviors in the field of network security. Background Art

[0002] In the current cybersecurity landscape, with the rapid development of the internet, the frequency and sophistication of cyberattacks are increasing. Malicious network traffic poses a serious threat to individuals, businesses, and even national security. Traditional malicious traffic detection methods primarily rely on rule-based signature matching or simple statistical analysis techniques. While effective in detecting known attack patterns, they often perform poorly against emerging attacks, such as zero-day attacks and advanced persistent threats (APTs), due to a lack of adaptability to complex attack patterns. According to statistics, the economic losses caused by cyberattacks have been rising annually in recent years, reaching hundreds of billions of dollars globally in 2023 alone. These figures reflect the urgent need for accurate, real-time, and resource-efficient malicious traffic detection technology.

[0003] In recent years, deep learning technology has been widely used in the field of malicious traffic detection, significantly improving detection speed and accuracy compared to traditional methods. Current mainstream detection methods include dynamic analysis based on network traffic and static analysis based on logs or protocols. Dynamic analysis methods effectively capture runtime characteristics of attacks by monitoring network traffic and system behavior in real time. However, they are computationally complex and resource-intensive, making them particularly difficult to deploy on edge devices or in resource-constrained environments. Static analysis methods, by analyzing static data such as network protocols and system logs, can efficiently process large amounts of data. However, they often rely on a single feature and fail to fully characterize the diverse behavioral patterns of network attacks, resulting in high false negative rates. Although some methods attempt to improve detection performance by combining multiple features, such as traffic statistics and protocol features, this simple feature combination is prone to the curse of dimensionality and information redundancy, increasing the computational burden and potentially reducing the model's generalization and interpretability. Therefore, despite the widespread application of deep learning technology in the field of malicious traffic detection, its high computational complexity, high resource consumption, and difficulty in deployment, particularly on edge devices or in resource-constrained environments, persist. Summary of the Invention

[0004] The purpose of the present invention is to address the shortcomings of the above-mentioned existing malicious traffic detection technologies in terms of accuracy, adaptability and resource efficiency, and to propose a lightweight malicious network traffic detection method based on heterogeneous modal feature fusion, which mainly solves the problems of low feature extraction efficiency and insufficient single modal feature representation of existing methods, and aims to achieve higher detection accuracy, stronger attack mode adaptability and lower computing resource consumption. By optimizing the single modal feature extraction method and constructing a lightweight heterogeneous modal feature fusion model LMF, combined with a dynamic weight allocation mechanism, the spatial distribution, temporal dependency and behavioral semantic information of network traffic are deeply explored to achieve efficient fusion of multi-dimensional features. The present invention can effectively improve data processing efficiency, detection accuracy and system robustness, and is suitable for efficient malicious traffic identification and defense in the field of network security.

[0005] To achieve the above object, the technical solution of the present invention includes the following steps:

[0006] (1) Obtain the NSL-KDD network traffic dataset containing normal traffic samples and malicious traffic samples. Based on attack behavior ontology analysis, the 41 fine-grained attack types in the dataset are mapped into five semantic categories, including denial of service attacks, probing attacks, remote-to-local attacks, user-to-root attacks, and normal traffic. Then, the dataset is preprocessed to obtain the initial dataset D.

[0007] (2) Using the conditional generative adversarial network (GAN) to generate traffic samples that conform to the distribution of various categories, and combining it with the synthetic minority class oversampling technology (SMOTE) to achieve data enhancement and category balance, generating a balanced enhanced dataset E;

[0008] (3) Use convolutional neural networks to extract spatial features from the enhanced dataset E, capture local patterns in the traffic data through convolution operations, and generate optimized spatial traffic feature vectors O;

[0009] (4) Divide the enhanced dataset E by time window, extract the time series features and input them into the bidirectional gated recurrent unit for time series modeling. The bidirectional gated recurrent unit captures the long-term dependencies in the time series and generates a time series feature vector P;

[0010] (5) A behavior pattern graph is constructed for the enhanced dataset E, which reflects the communication relationship of its traffic samples. Each node represents a communication entity, and each edge represents the communication relationship between entities. A multi-head self-attention mechanism is used to process the behavior pattern graph. The global dependency features between nodes are extracted through the self-attention mechanism, and the behavior pattern feature vector Q is generated by combining node embedding and edge embedding.

[0011] (6) Construct a deep fusion model, which realizes feature fusion through multi-scale convolution, temporal modeling, dynamic weight allocation and global attention mechanism, including the following steps:

[0012] (6.1) The spatial flow feature vector O is passed through a multi-channel convolutional Inception module to perform multi-scale feature extraction on the input features, capture local spatial patterns, and generate a spatial feature representation; a one-dimensional convolutional layer is then used to perform nonlinear transformation on the spatial features, and the adaptive maximum pooling layer is used to compress the dimensions to generate a spatial enhancement feature vector O';

[0013] (6.2) Input the time series feature vector P into the bidirectional gated recurrent unit for temporal context enhancement to generate a time series enhanced feature vector P';

[0014] (6.3) Input the behavior pattern feature vector Q into the multi-head self-attention mechanism to enhance the graph relationship and generate the behavior enhancement feature vector Q';

[0015] (6.4) Construct a dynamic weight allocation module (DWAM) to receive the enhanced feature vectors O', P', and Q'; calculate the statistical characteristics of each modality through a lightweight multi-layer perceptron (MLP) to generate an initial weight vector; normalize it through the Softmax function and output the dynamic weight coefficients w for the spatial, temporal, and behavioral modalities. O 、w P and w Q , and it satisfies w O +w P +w Q =1;

[0016] (6.5) Each dynamic weight coefficient is applied to its corresponding enhanced feature vector to generate a weighted fusion feature F, and F is converted into a compact feature vector R suitable for classification tasks through a fully connected layer. This is used for the classification task of network malicious traffic detection;

[0017] (7) Using the teacher-student model framework, the deep fusion model is trained end-to-end, and the cross-entropy loss function is used to optimize the model parameters until convergence to obtain the deep fusion teacher model;

[0018] (8) The fused feature F is used as the input of the teacher model to generate a comprehensive prediction result; the comprehensive prediction result is then transferred to the student model through the knowledge distillation technology. During the distillation process, the student model inherits the feature pattern of the teacher model and is trained until convergence, thereby obtaining a lightweight detection model;

[0019] (9) For the network traffic sample to be detected, the corresponding spatial traffic feature vector, time series feature vector and behavior pattern feature vector are generated in sequence according to steps (3)-(5), and they are input into the deep fusion teacher model for heterogeneous modal feature fusion to obtain a compact feature fusion vector; finally, the compact feature fusion vector is classified by the lightweight detection model, and the category to which the sample belongs is output to complete the detection of malicious network traffic.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] First, higher detection accuracy:

[0022] By deeply integrating the spatial local characteristics, time series characteristics and behavioral pattern characteristics of network traffic, the present invention can comprehensively capture the spatial patterns, temporal dynamics and interactive relationships of malicious traffic. Compared with single feature analysis or simple feature splicing methods, it can more effectively identify complex attack behaviors and significantly reduce false alarm and missed alarm rates.

[0023] Second, stronger attack mode adaptability:

[0024] This paper uses conditional generative adversarial networks (GANs) and synthetic minority oversampling technology (SMOTE) for data enhancement to generate traffic samples that conform to the distribution of each category, significantly improving the model's ability to detect rare attack types. At the same time, it extracts global features of the behavior pattern graph through a multi-head self-attention mechanism, enhancing the model's generalization ability for unknown attack patterns.

[0025] Third, lower resource consumption:

[0026] This invention uses the teacher-student model framework and knowledge distillation technology to transfer complex teacher model knowledge to a lightweight student model, significantly reducing the model's computational complexity and resource usage, making it suitable for edge devices and real-time detection scenarios; the dynamic modal weight fusion mechanism and lightweight fully connected layer design in the deep fusion model further reduce the number of parameters and computational burden.

[0027] Fourth, lower false positive and false negative rates:

[0028] Through the deep fusion of heterogeneous modal features and a dynamic modal weight fusion mechanism, this method comprehensively characterizes malicious traffic behavior, enhances feature complementarity, and avoids the high dimensionality and information redundancy problems caused by feature splicing. Dimensionality reduction and nonlinear transformation operations in the fusion layer further enhance the expressive power of features, significantly reducing false positives and false negatives. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flowchart of the implementation of the method of the present invention;

[0030] Figure 2 A schematic diagram of the overall framework for malicious traffic detection using the present invention;

[0031] Figure 3 Schematic diagram of the architecture of the lightweight heterogeneous modal feature fusion model LMF in the present invention;

[0032] Figure 4 This is a bar chart comparing the accuracy and F1 value of different feature combinations in the present invention; Figure 5 The figure shows the accuracy and loss change curves of the present invention and the existing model during the training process. DETAILED DESCRIPTION

[0033] To more clearly describe the purpose, method and beneficial effects of the present invention, the present invention is further described below with reference to the accompanying drawings.

[0034] Example 1: Refer to the attached Figure 1 This paper uses the idea of ​​heterogeneous modal feature fusion. After analysis, research and experiments, it proposes a lightweight malicious network traffic detection method based on heterogeneous modal feature fusion, which specifically includes the following steps:

[0035] Step 1. Obtain an NSL-KDD network traffic dataset containing normal traffic samples and malicious traffic samples, and based on attack behavior ontology analysis, map the 41 fine-grained attack types in the dataset into 5 semantic categories, including denial of service attacks, probing attacks, remote-to-local attacks, user-to-root attacks, and normal traffic; then preprocess it to obtain an initial dataset D; the preprocessing described in this embodiment is to first use a label encoder to convert the category label into a numerical form, then use discrete feature encoding, apply one-hot encoding to the discrete features to generate a sparse matrix, and finally perform standardization on the continuous features to make them conform to the zero mean and unit variance distribution.

[0036] Step 2. Use the Conditional Generative Adversarial Network (GAN) to generate traffic samples that conform to the distribution of various categories. At the same time, combine the synthetic minority oversampling technique (SMOTE) to achieve data enhancement and category balance, and generate a balanced enhanced dataset E.

[0037] Step 3. Use a convolutional neural network to extract spatial features from the enhanced dataset E. The convolution operation captures the local patterns in the traffic data and generates an optimized spatial traffic feature vector O, which includes discrete features after one-hot encoding and continuous features after normalization.

[0038] Step 4. Divide the enhanced dataset E by time windows, extract time series features and input them into the bidirectional gated recurrent unit for time series modeling. The bidirectional gated recurrent unit captures the long-term dependencies in the time series and generates a time series feature vector P; this vector includes traffic rate, packet interval time, and burstiness indicators.

[0039] Step 5. Construct a behavior pattern graph for the enhanced dataset E, reflecting the communication relationships of its traffic samples. Each node represents a communication entity, and each edge represents the communication relationship between entities. A multi-head self-attention mechanism is used to process the behavior pattern graph. The self-attention mechanism extracts the global dependency features between nodes, and the behavior pattern feature vector Q is generated by combining node embedding and edge embedding.

[0040] Step 6. Build a deep fusion model that achieves feature fusion through multi-scale convolution, temporal modeling, dynamic weight allocation, and global attention mechanism. The model includes the following steps:

[0041] (6.1) The spatial flow feature vector O is passed through a multi-channel convolutional Inception module to perform multi-scale feature extraction on the input features, capture local spatial patterns, and generate a spatial feature representation; then a one-dimensional convolutional layer is used to perform nonlinear transformation on the spatial features, and the dimension is compressed through an adaptive maximum pooling layer to generate a spatial enhanced feature vector O'.

[0042] In this embodiment, the multi-channel convolution Inception module contains four parallel branches: the first branch uses 1×1 convolution with 64 output channels; the second branch first uses 1×1 convolution with 128 output channels, followed by 3×3 convolution with 128 output channels; the third branch first uses 1×1 convolution with 32 output channels, followed by 5×5 convolution with 32 output channels; the fourth branch uses 3×3 maximum pooling followed by 1×1 convolution with 32 output channels; and the convolution operations of all branches are maintained by the same padding. The sequence length remains unchanged, and the ReLU activation function is applied after each convolution layer. The number of output channels of each branch is adjusted to 32 through 1×1 convolution and then added to generate a spatial feature representation; the one-dimensional convolution layer is used to perform nonlinear transformation on the spatial features, wherein the number of input channels of the convolution layer is 32, the number of output channels is 128, and the convolution kernel size is 64, and the nonlinear features are extracted through the ReLU activation function; and the dimension is compressed through the adaptive maximum pooling layer, specifically, the sequence dimension is compressed to a length of 1 through the adaptive maximum pooling layer to generate a spatial enhanced feature vector O'.

[0043] (6.2) The time series feature vector P is input into a bidirectional gated recurrent unit for temporal context enhancement, generating an enhanced time series feature vector P'. In this step, the time series feature vector P is converted into a three-dimensional tensor and then input into a BiGRU bidirectional gated recurrent unit for temporal modeling. The enhanced feature vector P' is captured through forward and backward processing, where the BiGRU has an input dimension of 128 and a hidden layer dimension of 256.

[0044] (6.3) The behavior pattern feature vector Q is input into the multi-head self-attention mechanism for graph relationship enhancement, generating the enhanced behavior feature vector Q'. The multi-head attention mechanism used in this embodiment has an embedding dimension of 512 and contains 8 attention heads. First, the time dimension of the behavior pattern feature vector Q is compressed to a length of 1 through an adaptive average pooling layer, forming a feature vector with a dimension of batch size × 512. Then, the correlation between features is calculated and weighted summed to generate the enhanced behavior feature vector Q'.

[0045] (6.4) Construct a dynamic weight allocation module (DWAM) to receive the enhanced feature vectors O', P', and Q'; calculate the statistical characteristics of each modality through a lightweight multi-layer perceptron (MLP) to generate an initial weight vector; normalize it through the Softmax function and output the dynamic weight coefficients w for the spatial, temporal, and behavioral modalities. O 、w P and w Q , and it satisfies w O +w P +w Q =1;

[0046] In this embodiment, the above-mentioned spatial, temporal sequence and behavioral dynamic weight coefficients w O 、w P and w Q , specifically according to the following steps:

[0047] (6.4.1) Reduce the dimension of P' and Q' to 128 through 1×1 convolution or fully connected layer respectively, and align them with O';

[0048] (6.4.2) Concatenate O', P', and Q' into the input tensor X m , the dimension is batch size × 3 × 128;

[0049] (6.4.3) Process X through a lightweight multi-layer perceptron MLP m The MLP consists of two fully connected layers. The first layer has an input dimension of 32 and an output dimension of 16, and uses the ReLU activation function. The second layer has an input dimension of 16 and an output dimension of 3, generating an initial weight vector V = [v O ,v P ,vQ ];

[0050] (6.4.4) Apply the Softmax function to V and calculate the dynamic weight coefficient according to the following formula:

[0051]

[0052] Among them, i∈{O,P,Q}, represents the spatial, temporal and behavioral modes; v i Represents the element corresponding to the i-th mode in the initial weight vector V, that is, the initial weight value; exp(v i ) is v i The exponential function value of is used to map the initial weight to the positive value range; is the sum of the initial weight index values ​​of the three modes.

[0053] (6.5) Each dynamic weight coefficient is applied to its corresponding enhanced feature vector to generate a weighted fusion feature F. This is then converted through a fully connected layer into a compact feature vector R suitable for classification tasks. This is used for the classification task of detecting malicious network traffic. In this embodiment, the weighted fusion feature F is obtained according to the following formula:

[0054] F=w o ·O′ f +w p ·P′ f +w q ·Q′ f

[0055] Among them, the dimension of F is batch size × 128, which represents the comprehensive representation after the fusion of three modal features; O' f is the enhanced spatial feature vector; P' f is the enhanced time series feature vector, with a dimension of batch size × 128; Q' f is the enhanced behavioral feature vector;

[0056] Furthermore, the above converts F into a compact feature vector R suitable for classification tasks through a fully connected layer, where the input dimension of the fully connected layer is 512 and the output dimension is 5, corresponding to 5 categories of labels: DoS, Probe, R2L, U2R and Normal; the Dropout operation is applied after the fully connected layer, and the compact feature vector R is output as the input of the subsequent classification task.

[0057] Step 7. Use the teacher-student model framework to perform end-to-end training on the deep fusion model, and use the cross-entropy loss function to optimize the model parameters until convergence to obtain the deep fusion teacher model;

[0058] Step 8. Use the fused features F as input to the teacher model to generate a comprehensive prediction result. Then, use knowledge distillation technology to transfer the comprehensive prediction result to the student model. During the distillation process, the student model inherits the characteristic patterns of the teacher model and is trained until convergence, resulting in a lightweight detection model. The training process in this step of this embodiment includes the following steps:

[0059] (8.1) Pre-training the teacher model: Use the fused feature F as the input of the teacher model, use the cross entropy loss function and the Adam optimizer to pre-train the teacher model, iteratively optimize the parameters, and save the model weights to a file after pre-training is completed;

[0060] (8.2) Construct and initialize the student model: This model consists of a fully connected layer using the nonlinear activation function ReLU. A dropout layer follows the fully connected layer to reduce the dimensionality and perform nonlinear transformations on the high-dimensional feature vector output by the fusion layer, generating a compact feature vector suitable for the classification task.

[0061] (8.3) Define the multimodal knowledge distillation loss and perform training: Load the pre-trained teacher model weights and set it to evaluation mode, transfer the comprehensive prediction results to the student model, and input the feature fusion vector F into the teacher model and student model respectively; define the knowledge distillation loss function:

[0062]

[0063] where y teacher Output classification for the teacher model, Output classification for the student model; for The cross entropy loss with the true label y, is the divergence loss between the student soft label and the teacher soft label, T is the temperature parameter, and α is the weight parameter. Use the Adam optimizer to train the student model, calculate the loss and update the parameters, evaluate the accuracy on the validation set, and save the model weight with the best performance on the validation set.

[0064] (8.4) Verify and output the student model detection category: Perform multiple rounds of training based on the enhanced dataset E. In each round, input batch data to generate feature fusion vector F. The teacher model generates soft labels to guide the student model learning. The student model updates its parameters in each round of training so that Gradually approaching y teacher , while maintaining the ability to classify the true label y; after training, the student model takes F as input and obtains a compact feature vector R suitable for the classification task, and then directly outputs the classification label as the detection category through the fully connected layer to complete malicious traffic detection.

[0065] Step 9. For the network traffic sample to be detected, generate the corresponding spatial traffic feature vector, time series feature vector, and behavior pattern feature vector in sequence according to steps 3-5, input them into the deep fusion teacher model for heterogeneous modal feature fusion, and obtain a compact feature fusion vector; finally, classify the compact feature fusion vector through the lightweight detection model, output the category to which the sample belongs, and complete the detection of malicious network traffic.

[0066] Example 2: The overall implementation steps of the lightweight malicious network traffic detection method proposed in this example are the same as those in Example 1. Figure 1-4 , give specific examples to further describe the implementation process of the present invention in detail:

[0067] Step 1) Obtain and preprocess the network traffic dataset: In order to obtain a network traffic dataset containing malicious traffic samples and normal traffic samples, this embodiment preferably obtains the NSL-KDD network traffic dataset from the official website as the data source, which contains normal traffic samples and intrusion traffic samples, which can effectively represent the diversity of network attack behaviors. Each sample in the NSL-KDD dataset contains 41-dimensional features, including basic features (such as duration, protocol type, service type, etc.); content features (such as error flags, number of login attempts) and time-based statistical features (such as connection rate, error rate, etc.); and is accompanied by category labels, which are divided into normal traffic, namely Normal, and 4 attack types, namely DoS, Probe, R2L, and U2R. To ensure data quality, the dataset is preprocessed using Python's pandas library to delete duplicate samples, missing value samples, or invalid samples to ensure that all samples in the dataset are valid. The dataset is stored in TXT format and is divided into two parts: a training set and a test set. Based on attack behavior ontology analysis, the 41 fine-grained attack types in the dataset are mapped into five semantic categories: Denial of Service (DoS), Probe, Remote-to-Local (R2L), User-to-Root (U2R), and Normal. Data preprocessing involves first converting category labels such as DoS and Probe into numerical form using a label encoder. In this embodiment, DoS is preferably mapped to 0, Probe to 1, and so on. Discrete feature encoding is then used to apply One-Hot Encoding to the protocol type, service type, and flag fields within the discrete features to generate a sparse matrix. Finally, continuous feature normalization is performed to normalize the continuous features to conform to a zero-mean, unit-variance distribution. The formula is as follows:

[0068]

[0069] Where x is the original eigenvalue, μ is the eigenvalue mean, and σ is the eigenvalue standard deviation. This ultimately forms the optimized initial dataset D, which has a dimension of N×M, where N is the number of samples and M is the feature dimension.

[0070] Step 2) Since the distribution of samples of each category in the NSL-KDD dataset is uneven, for example, the number of U2R attack samples is extremely small, directly using the original dataset may lead to insufficient detection capabilities of the model for minority categories. To this end, the conditional generative adversarial network Conditional GAN ​​is used to generate traffic samples that conform to the distribution of each category. The generator of Conditional GAN ​​uses the category label as the conditional input. The generator network structure includes 3 layers of fully connected layers, each layer uses the LeakyReLU activation function, and the output layer uses the Tanh activation function; the discriminator network structure includes 3 layers of fully connected layers, using the LeakyReLU activation function and the Dropout layer, and the output layer uses the Sigmoid activation function. When training Conditional GAN, the cross entropy loss function is used, the optimizer is Adam, and the learning rate is 0.0002. After generating synthetic samples that conform to the distribution of each category through Conditional GAN, the synthetic minority class oversampling technology SMOTE is combined to further balance the dataset. The specific steps are as follows:

[0071] 2a) Apply the SMOTE algorithm to the minority class U2R and R2L samples and generate new synthetic samples through interpolation. The SMOTE algorithm interpolates between the k nearest neighbors of the minority class samples to generate new samples. The formula is as follows:

[0072] x new =x i +λ·(x j -x i )

[0073] Among them, x i Minority class samples, x j is the enhanced dataset E of its k nearest neighbor samples.

[0074] 2b) Merge the samples generated by Conditional GAN ​​with those generated by SMOTE to generate a balanced enhanced dataset E. The enhanced dataset E has a uniform distribution of categories and a significantly increased number of samples, which can effectively improve the model's detection ability for rare attack types.

[0075] Step 3) Extracting Spatial Features of Network Traffic: Extract features from the augmented dataset E, including discrete features after one-hot encoding and continuous features after normalization. Use a convolutional neural network (CNN) to extract spatial features from the augmented dataset E, capturing local patterns in the traffic data through convolution operations. The specific operations are as follows:

[0076] 3a) Input layer: Each sample of the enhanced dataset E is represented as a vector with a dimension of M.

[0077] 3b) Convolutional Layer: A one-dimensional convolutional neural network is constructed, consisting of two convolutional layers. Each layer uses 32 convolution kernels with a kernel size of 3, a stride of 1, and a Reluctant Unified Unit (ReLU) activation function. The convolution operation can capture local patterns in traffic data, such as the correlation between protocol type and byte count.

[0078] 3c) Pooling layer: A max pooling layer is added after each convolutional layer with a pooling window size of 2 and a stride of 2 to reduce the dimensionality and extract the main features.

[0079] 3d) Flatten layer: Flattens the output of the pooling layer into a vector form.

[0080] 3e) Fully Connected Layer: This fully connected layer maps the flattened vector into a 64-dimensional feature vector using the Reluctant Unified Unit (ReLU) activation function. A Dropout layer is added to prevent overfitting. This ultimately generates the optimized spatial flow feature vector O, which has dimensions N × 64, where N is the number of samples.

[0081] Step 4) Extract time series features: Divide the enhanced dataset E into time windows and extract time series features, including traffic rate, packet interval time, and burst indicators. The specific operations are as follows:

[0082] 4a) Time Window Partitioning: Divide the traffic samples in the enhanced dataset E into fixed time windows and calculate the statistical features within each time window, including the traffic rate, which is the number of bytes transmitted per unit time; and the packet interval, which is the time difference between adjacent data packets.

[0083] 4b) Time series modeling: The extracted time series features are input into the bidirectional gated recurrent unit for time series modeling. The BiGRU network structure consists of two layers of bidirectional GRU, each layer contains 128 hidden units, the activation function is Tanh, and the dropout rate is 0.3. BiGRU can capture long-term dependencies in time series and generate time series feature vectors. The time series features are represented in matrix form with a dimension of T×F, where T is the number of time windows and F is the feature dimension of each time window. The time series data is then processed through the BiGRU layer to capture forward and backward time series information. Finally, the output of the last time step of the BiGRU is taken as the time series feature vector P with a dimension of N×128, where N is the number of samples.

[0084] Step 5) Extracting Behavior Pattern Features: The communication relationships of traffic samples in the enhanced dataset E are constructed as a behavior pattern graph, where each node represents a communicating entity and each edge represents the communication relationship between entities. A multi-head self-attention mechanism is used to process the behavior pattern graph. This mechanism extracts global dependency features between nodes and combines node and edge embedding to generate a behavior pattern feature vector. The specific steps are as follows:

[0085] 5a) Graph Construction: Based on the traffic data in the augmented dataset E, a behavior pattern graph G = (V, E) is constructed, where V is the set of communication entities and E is the set of communication relationships. Each edge has a weight, representing the communication frequency or number of bytes.

[0086] 5b) Node embedding: For each node v∈V, an initial node embedding vector with a dimension of 32 is generated based on its traffic statistical characteristics.

[0087] 5c) Edge embedding: For each edge e∈E, an edge embedding vector with dimension 16 is generated according to its weight.

[0088] 5d) Multi-head self-attention mechanism: A multi-head self-attention mechanism is used to process the behavior pattern graph and capture the global dependency features between nodes. The calculation formula of the attention mechanism is as follows:

[0089]

[0090] Among them, Q, K, V are query matrix, key matrix and value matrix respectively, d k is the dimension of the key vector. The multi-head attention mechanism consists of four attention heads. The output of each head is concatenated and linearly transformed to generate the final node feature representation. Global feature extraction: Graph node features are aggregated through global pooling operations (such as max pooling) to generate graph-level feature representations. The resulting behavior pattern feature vector Q has a dimension of N × 64, where N is the number of samples.

[0091] Step 6) Construct a lightweight heterogeneous modal feature fusion model LMF and perform feature fusion: The constructed lightweight heterogeneous modal feature fusion model LMF achieves feature fusion through multi-scale convolution, time series modeling, dynamic weight allocation, and global attention mechanism, receiving the network traffic spatial feature vector O, the time series feature vector P, and the behavior pattern feature vector Q. The specific operations are as follows:

[0092] 6a) Feature enhancement and dimensionality unification: The feature vectors O, P, and Q are enhanced separately through a fully connected layer to generate high-dimensional vectors O', P', and Q'. The mapping dimension is 128, the activation function is ReLU, and a Dropout layer is added.

[0093] 6b) Construct a dynamic weight allocation module, using a dynamic modal weight fusion mechanism and a multi-level fully connected structure to perform weighted fusion, dimensionality reduction, nonlinear transformation, and multimodal fusion on O', P', and Q'. The calculation formula of the dynamic modal weight fusion mechanism is as follows:

[0094]

[0095] Among them, v i is an element in the initial weight vector V, representing the initial weight value of mode i∈{o,p,q}; exp(v i ) is v i The exponential function value of is used to map the initial weight to the positive value range; is the sum of the initial weight index values ​​of the three modes, used as the normalized denominator; w i is the dynamic weight coefficient, which represents the normalized weight of the modality in fusion and satisfies w o +w p +w q =1.

[0096] 6b) Calculate the weighted fusion input using the following formula:

[0097] F=w o ·O′ f +w p ·P′ f +w q ·Q′ f

[0098] Among them, F is the feature vector after weighted fusion, the dimension is batch size × 128, which represents the comprehensive representation after the fusion of three modal features; w o 、w p 、w q is the dynamic weight coefficient, which is calculated by the Softmax function and represents the weight ratio of modal O, P, and Q in the fusion, and satisfies w o +w p +w q =1;O f ' is the enhanced spatial feature vector O; P f ' is the enhanced time series feature vector P, with a dimension of batch size × 128; Q f ' is the enhanced behavioral feature vector Q;

[0099] Step 7) Use the teacher-student model framework for model training. Using the teacher-student model framework, the teacher model obtains high predictive ability through pre-training, and then transfers its knowledge to the student model through knowledge distillation technology. The specific operation is as follows:

[0100] 7a) Teacher model training: The teacher model receives the comprehensive feature tensor of the enhanced dataset E. It first extracts spatial features through the Inception module to generate O; then captures time series features through BiGRU to generate P; then constructs a behavior pattern graph and uses a multi-head self-attention mechanism to generate Q. O, P, and Q are then enhanced to generate O', P', and Q'. O', P', and Q' are fused through DWAM to generate a feature vector F. Finally, the classification logitsy is output through multiple fully connected layers. teacher The training was performed using the cross entropy loss function, the Adam optimizer, the learning rate was 0.001, the batch size was 128, and the number of training epochs was 50.

[0101] 7b) Student model training: The student model is a lightweight neural network that receives the same input and generates O through the lightweight Inception module, P through the BiGRU, and Q through the multi-head self-attention mechanism. It then performs feature enhancement on O, P, and Q to generate O', P', and Q', which are then fused into F through DWAM. Finally, the classified logits are output through a two-layer fully connected layer. The number of parameters is about 1 / 5 of the teacher model.

[0102] Step 7.c) Knowledge Distillation: The generated feature fusion vector F is input into the teacher model and the student model, and the teacher model outputs the soft label y teacher , the student model output The knowledge distillation loss function is as follows:

[0103]

[0104] where y teacher Output classification for the teacher model, Output classification for the student model; for The cross entropy loss with the true label y, is the divergence loss between the student soft label and the teacher soft label, T is the temperature parameter, and α is the weight parameter; the Adam optimizer is used to train the student model, calculate the loss and update the parameters, evaluate the accuracy through the validation set, save the model weight with the best performance on the validation set, and finally generate a lightweight detection model, which can significantly reduce the computational complexity and resource usage while maintaining high accuracy.

[0105] Step 8) Detecting Malicious Network Traffic For the network traffic sample to be detected, perform steps 3) to 5) in sequence to generate corresponding feature vectors O, P, and Q; input them into the trained lightweight heterogeneous modal feature fusion model LMF for heterogeneous modal feature fusion to obtain a feature fusion vector F; finally, classify F through the lightweight student model and output the sample's category DoS, Probe, R2L, U2R, and Normal to complete the detection of malicious network traffic. The present invention is an overall technical solution. In response to the shortcomings of existing malicious traffic detection technology in terms of accuracy, adaptability, and resource efficiency, it optimizes the single-modal feature extraction method and adopts the lightweight heterogeneous modal feature fusion model LMF to achieve efficient fusion of multi-dimensional features such as network traffic features, time series features, and behavioral pattern features. The use of Conditional GAN ​​and SMOTE for data augmentation significantly improves the model's ability to detect rare attack types. A multi-head self-attention mechanism extracts global features from the behavioral pattern graph, enhancing the model's generalization ability for unknown attack patterns. The teacher-student model framework and knowledge distillation techniques significantly reduce the model's computational complexity and resource usage. This invention not only effectively adapts to and identifies new attacks, but also provides an accurate, efficient, and lightweight method for detecting malicious traffic in the field of network security. The implementation steps are as follows:

[0106] 8.1) Construct and pre-train the teacher model: Build a teacher model and take the enhanced dataset E as input; extract multi-scale features through the Inception module to capture local spatial patterns. After the features are transformed by the one-dimensional convolution layer, the dimensions are compressed by the adaptive maximum pooling layer, and then regularized by the batch normalization layer to generate an intermediate spatial feature vector. Next, the features are input into the bidirectional gated recurrent unit to capture the temporal features, and the intermediate time series feature vector is generated through forward and backward processing. Subsequently, the global features are fused through the multi-head attention mechanism to generate an intermediate global feature vector. On this basis, a dynamic weight allocation module DWAM is constructed to receive the intermediate spatial feature vector, the time series feature vector and the global feature vector. The statistical characteristics of each modality are calculated by the lightweight multi-layer perceptron and the initial weight vector is generated. The dynamic weight coefficient is normalized by the Softmax function to generate weighted fusion features. Finally, the weighted fusion features are converted into classification logits through the fully connected layer; the cross entropy loss function and the Adam optimizer are used to pre-train the teacher model for several cycles, and the parameters are iteratively optimized based on the training set of the enhanced dataset E. After the pre-training is completed, the model weights are saved to a file;

[0107] 8.2) Construct and initialize the student model: This model consists of a fully connected layer using the nonlinear activation function ReLU. A dropout layer follows the fully connected layer to reduce the risk of overfitting. This layer performs dimensionality reduction and nonlinear transformation on the high-dimensional feature vector output by the fusion layer to generate a compact feature vector. Finally, the output is a feature vector suitable for the classification task.

[0108] 8.3) Define the multimodal knowledge distillation loss and train it: Load the pre-trained teacher model weights and set it to evaluation mode. Input the feature fusion vector F into the teacher model and the student model respectively; the teacher model outputs the classification logitsy teacher , the student model outputs classification logits Define the knowledge distillation loss function, combining hard label loss and soft label loss, and the distillation loss formula is:

[0109]

[0110] where y teacher Output classification for the teacher model, Output classification for the student model; for The cross entropy loss with the true label y, is the divergence loss between the student soft label and the teacher soft label, T is the temperature parameter, and α is the weight parameter. Use the Adam optimizer to train the student model, calculate the loss and update the parameters, evaluate the accuracy on the validation set, and save the model weight with the best performance on the validation set.

[0111] 8.4) Verify and output the student model detection category: Perform multiple rounds of training based on the enhanced dataset E. In each round, input batch data to generate feature fusion vector F, and generate soft labels y through the teacher model. teacher Guide the student model to learn; the student model updates its parameters in each round of training, so that Gradually approaching y teacher , while maintaining the classification ability of the true label y; after training, the student model takes F as input and directly outputs the classification logits as the detection category through the fully connected layer for malicious traffic detection.

[0112] Example 3: The overall implementation steps of the lightweight malicious network traffic detection method proposed in this embodiment are the same as those of Example 1. The lightweight heterogeneous modal feature fusion model LMF constructed and formed in step 6 utilizes a multi-level fully connected layer to fuse features of different dimensions through a layer-by-layer progressive fusion strategy. Figure 3 , the construction process of the lightweight heterogeneous modal feature fusion model LMF in the present invention is further described in detail:

[0113] Step 6.1) Feature enhancement and dimension unification: The feature transformation module consists of multiple fully connected layers, which is responsible for receiving the spatial traffic feature vector O, the time series feature vector P, and the behavior pattern feature vector Q, and enhancing them from their respective original feature spaces and mapping them into a unified high-dimensional space to form high-dimensional feature vectors O', P', and Q'.

[0114] The specific operations are as follows:

[0115] Step 6.1.a) Input Processing: The spatial feature vector O has a dimension of N × 64, the time series feature vector P has a dimension of N × 128, and the behavioral pattern feature vector Q has a dimension of N × 64, where N is the number of samples. To unify the feature dimensions, the mapping layer processes each feature vector through a separate fully connected layer.

[0116] Step 6.1.b) Fully connected layer design: Each modal feature vector is mapped to a high-dimensional space through a fully connected layer, with a mapping dimension of 128. The mapping formula is as follows:

[0117] O'=ReLU(W O O+b O )

[0118] P'=ReLU(W P P+b P )

[0119] Q'=ReLU(W Q Q+b Q )

[0120] Among them, W O 、W P 、W Q are the weight matrices of each modal feature, b O 、b P 、b Q is the bias vector, and ReLU is a nonlinear activation function used to enhance the model's ability to learn nonlinear relationships between features.

[0121] Step 6.1.c) Regularization: Add a Dropout layer after each fully connected layer to randomly discard some neuron connections to reduce the risk of model overfitting, thereby improving the robustness and generalization ability of the model.

[0122] Step 6.1.d) Output: The mapping layer outputs high-dimensional feature vectors O', P', and Q', all of dimension N × 128, providing unified input for the subsequent fusion layer.

[0123] Step 6.2) Feature Integration Module: The feature integration module consists of multiple fully connected layers. It uses a dynamic modal weight fusion mechanism and a multi-level feature dimensionality reduction strategy to perform weighted fusion, nonlinear transformation, and multimodal feature fusion on the high-dimensional feature vectors O', P', and Q' to generate a comprehensive feature fusion vector F. The specific operations are as follows:

[0124] Step 6.2.a) Dynamic modality weight fusion: To fully utilize the contribution of each modality feature and enhance the complementarity between features, the fusion layer first performs weighted fusion of O', P' and Q' through the dynamic modality weight fusion mechanism. The weight score a of each modality is calculated through a fully connected layer and the Softmax function. i (i∈O',P',Q'), the formula is as follows:

[0125]

[0126] in, and is the weight matrix and bias vector, Flatten means flattening the eigenvector into a vector form. The normalized modal weight w i The calculation formula is as follows:

[0127]

[0128] Among them, v i is an element in the initial weight vector V, representing the initial weight value of mode i∈{o,p,q}; exp(v i ) is v i The exponential function value of is used to map the initial weight to the positive value range; is the sum of the initial weight index values ​​of the three modes, used as the normalized denominator; w i is the dynamic weight coefficient, which represents the normalized weight of the modality in fusion and satisfies w o +w p +w q =1. Then according to the calculated modal weight w i , perform weighted fusion of O', P' and Q' to generate the initial fusion feature vector F, the formula is as follows:

[0129] F=w o ·O′ f +w p ·P′ f +w q ·Q′ f

[0130] Among them, F is the feature vector after weighted fusion, the dimension is batch size × 128, which represents the comprehensive representation after the fusion of three modal features; w o 、w p、w q is the dynamic weight coefficient, which is calculated by the Softmax function and represents the weight ratio of modal O, P, and Q in the fusion, and satisfies w o +w p +w q =1;O f ' is the enhanced spatial feature vector O; P f ' is the enhanced time series feature vector P, with a dimension of batch size × 128; Q f ' is the enhanced behavioral feature vector Q.

[0131] Step 6.2.b) Multi-level fully connected structure: The fusion layer contains two fully connected modules, each module consists of a fully connected layer, a ReLU activation function and a Dropout layer for further dimensionality reduction and nonlinear transformation. The first fully connected module: init Perform nonlinear transformation and preliminary dimensionality reduction to generate the intermediate feature vector F mid , with a dimension of N×64. The specific operations include: reducing the dimension from 128 to 64 through the fully connected layer, using the ReLU activation function to enhance the nonlinear expression ability, and adding the Dropout layer to prevent overfitting. The second fully connected module: mid Further feature fusion and dimensionality reduction are performed to generate the final feature fusion vector F, with a dimension of N × 64. Feature integration is performed through a fully connected layer, and the ReLU activation function is used to enhance the nonlinear relationship between features. A dropout layer (with a dropout rate of 0.3) is added to improve model robustness. The final fusion output, the feature fusion vector F, has a dimension of N × 64 and integrates information from network traffic, time series, and behavioral patterns. It can comprehensively characterize the complex patterns of network intrusion behavior. Dimensionality reduction also removes redundant information and extracts more critical and representative features.

[0132] Step 6.3) Build the output module: Build the output module to further reduce the dimension and perform nonlinear transformation on the feature fusion vector F output by the fusion layer to generate a compact feature vector R suitable for classification tasks. The specific operations are as follows:

[0133] Step 6.3.a) Fully connected layer design: Use a fully connected layer to reduce the dimension of the feature fusion vector F from 64 to 16, generating a compact feature vector R. The formula is as follows:

[0134] R=ReLU(W F ·F+b F )

[0135] Among them, W F is the weight matrix, b F is the bias vector and ReLU is the activation function.

[0136] Step 6.3.b) Regularization: Add a Dropout layer after the fully connected layer to further reduce the risk of overfitting.

[0137] Step 6.3.c) Construct output: The output layer outputs a compact feature vector R with a dimension of N×16, which is suitable for subsequent classification tasks and provides an efficient and compact feature representation for the detection of malicious network traffic.

[0138] This paper demonstrates significant advantages in feature extraction and malicious network traffic detection by constructing a lightweight heterogeneous modal feature fusion model LMF, including:

[0139] Efficient feature mapping: The mapping layer effectively maps input features of different modalities into a high-dimensional space through fully connected layers and ReLU activation functions, enhancing the model's ability to capture complex feature relationships. Furthermore, the introduction of the Dropout layer reduces the risk of overfitting and improves the model's generalization capabilities.

[0140] Dynamic feature fusion: The fusion layer uses a dynamic modal weight fusion mechanism to adaptively assign weights based on the importance of each modal feature, enhancing the complementarity between features. The multi-level fully connected structure further integrates features through dimensionality reduction and nonlinear transformations, strengthening the model's understanding of complex network attack patterns.

[0141] Compact feature representation: The output layer generates a compact feature vector R through dimensionality reduction and nonlinear transformation, which significantly reduces the computational complexity while retaining key information, providing accurate and efficient input for subsequent classification tasks.

[0142] The design of the lightweight heterogeneous modal feature fusion model (LMF) deeply explores and integrates network traffic features, time series features, and behavioral pattern features, significantly improving detection accuracy. Through the dynamic modal weight fusion mechanism and multi-level dimensionality reduction operations, the model improves computational efficiency while retaining key information. In addition, the introduction of the Dropout layer and the application of the teacher-student model framework further reduce the computational complexity and resource consumption of the model, making it suitable for edge devices and real-time detection scenarios. Overall, the lightweight heterogeneous modal feature fusion model (LMF) not only improves the detection accuracy of malicious network traffic, but also reduces manual intervention through automated feature processing, enhancing the model's generalization and robustness.

[0143] Example 4: The overall implementation steps of the lightweight malicious network traffic detection method proposed in this example are the same as those in Example 1. The teacher-student model framework in step 7 is to form an efficient and accurate lightweight detection model by pre-training a complex teacher model and using knowledge distillation technology to transfer knowledge to a lightweight student model. Figure 2The training process of the model training using the teacher-student model framework in the present invention is further described in detail:

[0144] Step 7.1) Build and pre-train the teacher model: The teacher model is a complex deep neural network designed to achieve high predictive power through pre-training, providing high-quality soft labels for subsequent knowledge distillation. The teacher model uses a multi-layer, fully connected neural network architecture, consisting of five fully connected layers. Residual connections are introduced to enhance training stability and feature representation. Specifically, the first layer is the input layer, receiving the weighted fused features F. The second layer is the hidden layer, consisting of four fully connected layers with dimensions in the order of 256 → 512 → 256 → 128 → 64. Each layer uses the ReLU activation function to enhance nonlinear representation, and a Dropout layer is added after each layer to prevent overfitting. Residual connections are introduced between the second and fourth layers to mitigate the vanishing gradient problem and improve model training stability. The third layer is the output layer. Through fully connected layers, the feature vector is mapped to the number of classification categories (five categories: DoS, Probe, R2L, U2R, and Normal). The output class prediction probabilities are N × 5, and the Softmax activation function is used to generate the probability distribution. The teacher model is trained using the cross entropy loss function, and the loss function formula is as follows:

[0145]

[0146] Among them, y i,c is the one-hot encoding of the true label, is the predicted probability output by the teacher model, and N is the number of samples. The optimizer is Adam, the learning rate is 0.001, the batch size is 128, and the number of training epochs is 50. The training data is the enhanced dataset E. Through pre-training, the teacher model can fully learn the complex patterns in the feature vectors O, P, and Q, providing high-precision category predictions.

[0147] Step 7.2) Construct a lightweight student model: The student model is a lightweight neural network designed to inherit the predictive capabilities of the teacher model while significantly reducing computational complexity and resource consumption to adapt to edge devices and real-time detection scenarios. The student model uses a simplified fully connected neural network architecture, consisting of two fully connected layers and approximately one-fifth the number of parameters of the teacher model. The first layer is the input layer, identical to the teacher model, receiving a comprehensive feature vector with a dimension of N × 256. The second layer is the hidden layer, consisting of one fully connected layer with a dimension of 256→64, using the ReLU activation function, and followed by a dropout layer to prevent overfitting. The third layer is the output layer, which maps the feature vector to the number of categories for the classification task through a fully connected layer and outputs the predicted category probabilities with a dimension of N × 5, using the Softmax activation function to generate a probability distribution. By reducing the number and dimension of fully connected layers, the student model has significantly lower parameter count and computational complexity than the teacher model. Furthermore, through knowledge distillation techniques, it inherits the characteristic patterns of the teacher model, ensuring minimal loss in detection performance.

[0148] Step 7.3) Use knowledge distillation technology to transfer knowledge: Through knowledge distillation technology, the knowledge of the teacher model is transferred to the student model, so that the student model can maintain high accuracy while significantly reducing computational complexity and resource usage. The teacher model generates soft labels through pre-training, that is, softening the category prediction probability of the input sample. The softening formula is as follows:

[0149]

[0150] Among them, z c where is the logits output by the teacher model, and T is the temperature parameter, which controls the smoothness of the soft labels. A higher T value smoothes the probability distribution, helping the student model learn the generalization knowledge of the teacher model. The training loss function of the student model consists of two parts: classification loss and distillation loss. The classification loss is the cross entropy loss between the student model's predicted probability and the true label, and is formulated as follows:

[0151]

[0152] Among them, y i,c is the one-hot encoding of the true label, is the predicted probability output by the student model. In addition, the distillation loss is the KL divergence loss between the predicted probability of the student model and the soft label of the teacher model, and the formula is as follows:

[0153]

[0154] Among them, T 2The weight used to amplify the distillation loss to balance the contribution of classification loss and distillation loss. The total loss function is the weighted sum of classification loss and distillation loss, and the formula is as follows:

[0155] L=α·L CE +(1-α)·L KL

[0156] Here, α is a weight parameter, initially 0.1, to balance the contributions of the two losses. The training parameter optimizer is Adam, with a learning rate of 0.001, a batch size of 128, and 30 training epochs. The training data is the augmented dataset E. Through knowledge distillation, the student model effectively inherits the characteristic patterns of the teacher model while maintaining its lightweight nature.

[0157] This invention, by adopting a teacher-student model framework, demonstrates significant advantages in model training and malicious network traffic detection. Specific advantages are as follows:

[0158] High-precision teacher model: Through complex network structure and pre-training, the teacher model fully mines the deep patterns of network traffic characteristics, time series characteristics, and behavioral pattern characteristics, provides high-precision category predictions, and provides high-quality soft labels for knowledge distillation.

[0159] Lightweight student model: The student model significantly reduces the number of parameters and computational complexity through a simplified network structure, making it suitable for edge devices and real-time detection scenarios. At the same time, it inherits the predictive ability of the teacher model through knowledge distillation technology, ensuring that the loss of detection performance is minimized.

[0160] Efficient knowledge transfer: Knowledge distillation technology uses soft labels and KL divergence loss to enable the student model to learn not only the hard classification information of the true labels, but also the generalization knowledge of the teacher model, enhancing the model's adaptability and robustness to complex attack patterns.

[0161] The design of the teacher-student model framework significantly reduces the computational complexity and resource consumption of the detection model while maintaining high accuracy by pre-training a complex teacher model and transferring knowledge using knowledge distillation techniques. The complex structure and residual connection design of the teacher model enhance the model's ability to deeply mine features, while the lightweight design of the student model and the application of knowledge distillation techniques enable it to operate efficiently in resource-constrained environments. Overall, the teacher-student model framework not only improves the detection efficiency of malicious network traffic, but also reduces the complexity of model optimization through automated knowledge transfer, enhancing the model's practicality and adaptability.

[0162] The lightweight heterogeneous modal feature fusion model (LMF) constructed in the present invention integrates network traffic features, time series features, and behavioral pattern features. It can not only deeply integrate these three features, but also integrate features from other existing malicious network traffic detection technologies, such as protocol analysis features, traffic statistics features, system log features, and user behavior features. At the same time, the present invention has the ability to adapt to new features that may emerge in the future, such as traffic semantic reasoning features based on large language models, which extract hidden semantic patterns by performing natural language processing and contextual analysis on traffic content; adversarial traffic generation features based on generative AI, which simulate new attack behaviors through generative adversarial networks to enhance the model's robustness against unknown attacks; and distributed behavioral pattern features based on federated learning, which capture global attack trends through cross-device collaborative learning to achieve more comprehensive and efficient malicious network traffic detection. By deeply integrating network traffic, time series, and behavioral pattern features, the present invention demonstrates its high versatility and scalability in detection technology. This forward-looking and flexible approach enables the detection mechanism to cope with evolving network attack technologies while maintaining high accuracy and robustness. By combining network traffic characteristics, time series characteristics, and behavioral pattern characteristics, the detection accuracy of known malicious traffic is significantly improved, while the adaptability to new malicious traffic is greatly enhanced. The design of the lightweight heterogeneous modal feature fusion model (LMF) cleverly applies deep learning technology. Through multi-level fusion and nonlinear transformation, it effectively integrates different types of data features, significantly improving the accuracy and adaptability of malicious traffic identification, enhancing the level of detection automation, and the efficiency of processing large-scale data.

[0163] The effects of the present invention will be further described below in conjunction with simulation experiments.

[0164] 1. Experimental Conditions

[0165] Hardware environment: Linux operating system, equipped with Intel Core i7-8550U CPU, 16GB memory, and NVIDIA A10 graphics card.

[0166] Software environment: Programming language is Python 3.8, PyTorch 1.3.0 deep learning framework, experimental code is developed and run in the Jupyter Notebook environment

[0167] 2. Experimental content

[0168] By obtaining network traffic samples from the NSL-KDD dataset and performing data cleaning and preprocessing to ensure the dataset's validity and integrity, we generated a reliable dataset containing both malicious and normal traffic samples. The training set, validation set, and test set ratio was 6:2:2, with 76,424 malicious traffic samples and 77,663 normal traffic samples. The dataset was preprocessed to obtain spatial traffic features, time series features, and behavioral pattern features, which were then vectorized to obtain the final feature vectors of different dimensions.

[0169] Experiment 1: Combined with the limitations of the experimental environment, experiments are conducted on whether to add GAN and SMOTE data enhancement methods to determine the best data processing method under this experimental environment. The present invention uses a lightweight heterogeneous modal feature fusion model LMF to perform multiple rounds of training and malicious traffic detection on three different feature vectors: spatial traffic feature vector, time series feature vector and behavior pattern feature vector, and saves the detection results of each round. Then, these three features are directly spliced ​​in pairs, and multiple rounds of training and malicious traffic detection are performed through the LMF model, and the optimal results are also saved. Secondly, the present invention also uses the feature deep fusion model LMF to perform a deeper fusion processing on these three features and directly splice the three features, and multiple rounds of training and malicious traffic detection are performed, and the detection results of each round are saved.

[0170] Experiment 2: The proposed LMF model utilizes a deep fusion architecture of Inception-BiGRU-Attention, aiming to achieve efficient network traffic classification through multi-layer feature extraction and fusion. To comprehensively evaluate the performance of the LMF model, a benchmark comparison experiment was designed. In this benchmark comparison experiment, the LMF model was compared with classic network traffic classification models: CNN-BiLSTM, CNN-GRU, and CNN-LSTM. The performance was evaluated in terms of classification accuracy, loss variance, and other aspects.

[0171] 3. Experimental Results and Analysis

[0172] Experiment 1: This experiment tested two different data processing methods to determine the most appropriate data augmentation settings in the current environment and evaluate the impact of different settings on the model classification accuracy and F1 value.

[0173] Experiments were conducted on spatial traffic feature vectors, time series feature vectors, and behavior pattern feature vectors, pairwise splicing fusion, direct splicing fusion of three features, and fusion using the feature deep fusion model LMF to evaluate the effects of different feature combinations and different fusion methods on classification accuracy. The experimental results are shown in the figure. Figure 4As shown in the figure, the bar chart shows in detail the accuracy and F1 value of malicious traffic detection using spatial traffic feature vectors, time series feature vectors, behavior pattern feature vectors, pairwise splicing fusion, direct splicing fusion of three features, and LMF fusion, a deep feature fusion model. Figure 4 Although a single feature vector can provide basic classification capabilities, its accuracy is relatively low, all below 88%. The combination of two feature vectors significantly improves the classification effect by utilizing the complementary information between features, with the highest accuracy reaching 92.0%. The direct concatenation of the three feature vectors improves the performance to a certain extent, but the most significant improvement comes from the use of the feature deep fusion model LMF of the present invention, with an accuracy of 95.8%. This not only reflects the effectiveness of this method in improving classification accuracy, but also verifies through experimental data that the present invention is superior to the classification methods of a single feature and a combination of two features in terms of accuracy, and solves the problems of high dimensionality and information redundancy caused by the direct concatenation of multiple features. The experimental results are shown in Table 1, which shows the accuracy and F1 value of the fusion of GAN and SMOTE with and without the fusion of GAN and SMOTE.

[0174] Table 1: Effect of different maximum timestamp settings on classification accuracy and F1 value

[0175]

[0176] It can be seen from the experimental results that the lowest accuracy of the present invention is 90.5% when GAN and SMOTE are not added. This shows that under this data processing method, there is a class imbalance problem in the data processed by the model, and the characteristics of the minority class samples are not fully captured, thereby affecting the accuracy of the classification. The highest accuracy is when GAN and SMOTE are added, and the accuracy reaches 95.8%. This shows that the present invention generates synthetic samples that conform to the distribution of each category through the conditional generative adversarial network Conditional GAN, and combines the synthetic minority class oversampling technology SMOTE to interpolate the minority class samples to generate new synthetic samples, which can significantly increase the number and diversity of minority class samples, generate a balanced enhanced data set, and thus significantly improve the accuracy of classification. However, the choice of whether to add GAN and SMOTE also needs to consider the resource limitations and computational costs of the experimental environment.

[0177] Experiment 2: To further evaluate the performance of the LMF model, this experiment compares it with the classic network traffic classification models CNN-BiLSTM, CNN-GRU, and CNN-LSTM. The experimental results are as follows Figure 5The figure shows the accuracy and loss trends of each model during training. The top graph shows the accuracy curve, and the bottom graph shows the loss curve. The top graph shows that the accuracy of the LMF model (CNN-BiGRU-Transformer, blue curve) increases rapidly early in training and stabilizes after epoch 20, ultimately reaching approximately 95.46% on the test set, significantly outperforming other baseline models. In contrast, the accuracy of the CNN-BiLSTM (green curve), CNN-GRU (orange curve), and CNN-LSTM (purple curve) stabilizes at approximately 95.33%, 95.21%, and 95.22%, respectively, all approximately 0.2% lower than the LMF model. This demonstrates that the LMF model, through the multi-scale feature extraction of the Inception module, the time series modeling of the BiGRU module, and the global feature enhancement of the Transformer, achieves stronger feature representation, thereby improving classification performance. The bottom graph shows the loss changes of each model during training. As can be seen from the figure, the loss of the LMF model decreases the fastest, falling below 0.06 by the fifth epoch and stabilizing at around 0.03 after the 15th epoch, demonstrating good convergence. In contrast, the loss of CNN-BiLSTM, CNN-GRU, and CNN-LSTM decreases more slowly and still fluctuates somewhat in the later stages of training. The final loss values ​​stabilize at around 0.0187, 0.0179, and 0.0182, respectively, all higher than those of the LMF model. This demonstrates that the LMF model has greater stability during training, enabling more effective optimization of model parameters and reducing the risk of overfitting.

[0178] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention. The lightweight malicious traffic detection method based on heterogeneous modal feature fusion proposed in the present invention effectively solves the problems of incomplete coverage, low feature utilization, and high dimensionality and information redundancy caused by the direct splicing of multiple features faced by traditional static methods using a single feature to detect malicious network traffic. The present invention significantly improves the accuracy and adaptability of malicious traffic detection. This processing method based on deep fusion of multiple features not only deeply mines the semantics and behavioral information of malicious traffic, but also greatly optimizes the data processing efficiency, thereby greatly improving the comprehensiveness and response speed of the detection system. The present invention not only provides an efficient and reliable malicious traffic detection solution, but also provides strong technical support for modern network security protection. It can be used for the identification and prevention of malicious traffic in the field of network security, as well as network security defense.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0180] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0181] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A lightweight malicious network traffic detection method based on heterogeneous modal feature fusion, characterized by: The steps include: (1) Obtain the NSL-KDD network traffic dataset containing normal traffic samples and malicious traffic samples. Based on attack behavior ontology analysis, the 41 fine-grained attack types in the dataset are mapped into five semantic categories, including denial of service attacks, probing attacks, remote-to-local attacks, user-to-root attacks, and normal traffic. Then, the dataset is preprocessed to obtain the initial dataset D. (2) Using the conditional generative adversarial network (GAN) to generate traffic samples that conform to the distribution of various categories, and combining it with the synthetic minority class oversampling technology (SMOTE) to achieve data enhancement and category balance, generating a balanced enhanced dataset E; (3) Use convolutional neural networks to extract spatial features from the enhanced dataset E, capture local patterns in the traffic data through convolution operations, and generate optimized spatial traffic feature vectors O; (4) Divide the enhanced dataset E by time window, extract the time series features and input them into the bidirectional gated recurrent unit for time series modeling. The bidirectional gated recurrent unit captures the long-term dependencies in the time series and generates a time series feature vector P; (5) A behavior pattern graph is constructed for the enhanced dataset E, which reflects the communication relationship of its traffic samples. Each node represents a communication entity, and each edge represents the communication relationship between entities. A multi-head self-attention mechanism is used to process the behavior pattern graph. The global dependency features between nodes are extracted through the self-attention mechanism, and the behavior pattern feature vector Q is generated by combining node embedding and edge embedding. (6) Construct a deep fusion model, which realizes feature fusion through multi-scale convolution, temporal modeling, dynamic weight allocation and global attention mechanism, including the following steps: (6.1) The spatial flow feature vector O is passed through a multi-channel convolutional Inception module to perform multi-scale feature extraction on the input features, capture local spatial patterns, and generate a spatial feature representation; a one-dimensional convolutional layer is then used to perform nonlinear transformation on the spatial features, and the adaptive maximum pooling layer is used to compress the dimensions to generate a spatial enhancement feature vector O'; (6.2) Input the time series feature vector P into the bidirectional gated recurrent unit for temporal context enhancement to generate a time series enhanced feature vector P'; (6.3) Input the behavior pattern feature vector Q into the multi-head self-attention mechanism to enhance the graph relationship and generate the behavior enhancement feature vector Q'; (6.4) Construct a dynamic weight allocation module (DWAM) to receive the enhanced feature vectors O', P', and Q'; calculate the statistical characteristics of each modality through a lightweight multi-layer perceptron (MLP) to generate an initial weight vector; normalize it through the Softmax function and output the dynamic weight coefficients w for the spatial, temporal, and behavioral modalities. O 、w P and w Q , and it satisfies w O +w P +w Q =1; (6.5) Each dynamic weight coefficient is applied to its corresponding enhanced feature vector to generate a weighted fusion feature F, and F is converted into a compact feature vector R suitable for classification tasks through a fully connected layer. This is used for the classification task of network malicious traffic detection; (7) Using the teacher-student model framework, the deep fusion model is trained end-to-end, and the cross-entropy loss function is used to optimize the model parameters until convergence to obtain the deep fusion teacher model; (8) The fused feature F is used as the input of the teacher model to generate a comprehensive prediction result; the comprehensive prediction result is then transferred to the student model through the knowledge distillation technology. During the distillation process, the student model inherits the feature pattern of the teacher model and is trained until convergence, thereby obtaining a lightweight detection model; (9) For the network traffic sample to be detected, the corresponding spatial traffic feature vector, time series feature vector and behavior pattern feature vector are generated in sequence according to steps (3)-(5), and they are input into the deep fusion teacher model for heterogeneous modal feature fusion to obtain a compact feature fusion vector; finally, the compact feature fusion vector is classified by the lightweight detection model, and the category to which the sample belongs is output to complete the detection of malicious network traffic.

2. The method according to claim 1, wherein: The preprocessing in step (1) is to first convert the category label into a numerical form using a label encoder, then use discrete feature encoding, apply one-hot encoding to the discrete features to generate a sparse matrix, and finally perform standardization on the continuous features.

3. The method according to claim 1, wherein: The spatial flow feature vector O in step (3) includes discrete features after one-hot encoding and continuous features after standardization.

4. The method according to claim 1, wherein: The time series feature vector P in step (4) includes traffic rate, packet interval time and burst index.

5. The method according to claim 1, wherein: The multi-channel convolution Inception module described in step (6.1) contains four parallel branches: the first branch uses 1×1 convolution with 64 output channels; the second branch first uses 1×1 convolution with 128 output channels, followed by 3×3 convolution with 128 output channels; the third branch first uses 1×1 convolution with 32 output channels, followed by 5×5 convolution with 32 output channels; the fourth branch uses 3×3 maximum pooling followed by 1×1 convolution with 32 output channels; and the convolution operations of all branches are performed with the same padding. Keeping the sequence length unchanged, applying the ReLU activation function after each convolution layer, adjusting the number of output channels of each branch to 32 through 1×1 convolution and then adding them together to generate a spatial feature representation; using a one-dimensional convolution layer to perform nonlinear transformation on the spatial features, wherein the number of input channels of the convolution layer is 32, the number of output channels is 128, and the convolution kernel size is 64, and the nonlinear features are extracted through the ReLU activation function; and compressing the dimension through the adaptive maximum pooling layer, specifically, compressing the sequence dimension to a length of 1 through the adaptive maximum pooling layer to generate a spatial enhanced feature vector O'.

6. The method according to claim 1, wherein: The time series enhanced feature vector P' described in step (6.2) is obtained by converting the time series feature vector P into a three-dimensional tensor, inputting it into a BiGRU bidirectional gated recurrent unit for time series modeling, and capturing long-term dependencies through forward and backward processing; The input dimension of BiGRU is 128 and the hidden layer dimension is 256.

7. The method according to claim 1, wherein: The multi-head attention mechanism described in step (6.3) has an embedding dimension of 512 and contains 8 attention heads. The time dimension of the behavior pattern feature vector Q is compressed to a length of 1 through an adaptive average pooling layer, forming a feature vector of dimension batch size × 512; Calculate the correlation between features and perform weighted summation to generate the behavior enhancement feature vector Q'.

8. The method according to claim 1, wherein: The spatial, temporal sequence and behavioral dynamic weight coefficient w described in step (6.4) O 、w P and w Q , follow the steps below to obtain: (6.4.1) Reduce the dimension of P' and Q' to 128 through 1×1 convolution or fully connected layer respectively, and align them with O'; (6.4.2) Concatenate O', P', and Q' into the input tensor X m , the dimension is batch size × 3 × 128; (6.4.3) Process X through a lightweight multi-layer perceptron MLP m The MLP consists of two fully connected layers. The first layer has an input dimension of 32 and an output dimension of 16, and uses the ReLU activation function. The second layer has an input dimension of 16 and an output dimension of 3, generating an initial weight vector V = [v O ,v P ,v Q ]; (6.4.4) Apply the Softmax function to V and calculate the dynamic weight coefficient according to the following formula: Among them, i∈{O,P,Q}, represents the spatial, temporal and behavioral modes; v i Represents the element corresponding to the i-th mode in the initial weight vector V, that is, the initial weight value; exp(v i ) is v i The exponential function value of is used to map the initial weight to the positive value range; is the sum of the initial weight index values ​​of the three modes.

9. The method according to claim 1, wherein: The weighted fusion feature F in step (6.5) is obtained according to the following formula: F=w o ·O' f +w p ·P' f +w q ·Q' f Among them, the dimension of F is batch size × 128, which represents the comprehensive representation after the fusion of three modal features; O' f is the enhanced spatial feature vector; P' f is the enhanced time series feature vector, with a dimension of batch size × 128; Q' f is the enhanced behavioral feature vector; The method converts F into a compact feature vector R suitable for classification tasks through a fully connected layer, where the input dimension of the fully connected layer is 512 and the output dimension is 5, corresponding to five types of labels: DoS, Probe, R2L, U2R, and Normal; the Dropout operation is applied after the fully connected layer, and the compact feature vector R is output as the input of the subsequent classification task.

10. The method according to claim 1, wherein: The training process of the lightweight detection model in step (8) includes the following steps: (8.1) Pre-training the teacher model: The fused feature F is used as the input of the teacher model. The teacher model is pre-trained using the cross entropy loss function and the Adam optimizer. The parameters are iteratively optimized. After pre-training, the model weights are saved to a file to generate a comprehensive prediction result. (8.2) Construct and initialize the student model: This model consists of a fully connected layer using the nonlinear activation function ReLU. A dropout layer follows the fully connected layer to reduce the dimensionality and perform nonlinear transformations on the high-dimensional feature vector output by the fusion layer, generating a compact feature vector suitable for the classification task. (8.3) Define the multimodal knowledge distillation loss and perform training: Load the pre-trained teacher model weights and set it to evaluation mode, transfer the comprehensive prediction results to the student model, and input the feature fusion vector F into the teacher model and student model respectively; define the knowledge distillation loss function: where y teacher Output classification for the teacher model, Output classification for the student model; for The cross entropy loss with the true label y, is the divergence loss between the student soft label and the teacher soft label, T is the temperature parameter, and α is the weight parameter. Use the Adam optimizer to train the student model, calculate the loss and update the parameters, evaluate the accuracy on the validation set, and save the model weight with the best performance on the validation set. (8.4) Verify and output the student model detection category: Perform multiple rounds of training based on the enhanced dataset E, input batch data in each round to generate feature fusion vector F, and use the teacher model to generate soft labels to guide the student model learning; The model updates its parameters in each round of training so that Gradually approaching y teacher , while maintaining the ability to classify the true label y; after training, the student model takes F as input and obtains a compact feature vector R suitable for the classification task, and then directly outputs the classification label as the detection category through the fully connected layer to complete malicious traffic detection.

Images