A method, system, and medium for detecting abnormal network traffic based on a lightweight contrast-enhanced attention detection model.

Abstract

This invention relates to the technical field and discloses a method, system, and medium for detecting abnormal network traffic based on a lightweight contrast-enhanced attention detection model. The method first extracts core features such as packet length, direction, and arrival interval from network traffic to construct a SAM matrix. After normalization and PCA dimensionality reduction, a time window sample is constructed, and a normal feature library is built based on normal samples. Then, local spatiotemporal features are extracted through the iRMB backbone, and global dependencies are fused using a thin multi-head attention model. The model is trained with a classification + contrastive joint loss and finally deployed in the cloud / edge, supporting real-time detection and incremental updates. This invention reduces the computational complexity of the abnormal network traffic detection model while maintaining its advantages of resistance to overfitting and adaptability to large samples.

Description

Technical Field

[0001] This invention relates to the field of network anomaly detection technology, and more specifically to a method for detecting network anomaly traffic based on a lightweight contrast-enhanced attention detection model (LCADM). Background Technology

[0002] Anomaly detection in network traffic is one of the key means of preventing and controlling network attacks (traditional network attack detection includes analysis and identification of links, traffic, and topology dimensions). It plays an important role in network security, which is conducive to early detection of network attack behavior and early warning and prevention, reducing the security risks of network attacks and reducing the impact of business interruption.

[0003] In existing technologies, network anomaly detection models based on machine learning have been used to identify network attacks, especially by identifying abnormal traffic in network transmission. These models mainly include detection models based on random forest / logistic regression, convolutional detection models based on convolutional networks such as CNN or multi-resolution CNN, Transformer-based detection models based on multi-head attention mechanisms, and lightweight detection models based on pruned network structures such as MobileNet / VGG. These models are deployed after sample collection, feature engineering, and deep learning to detect abnormal network traffic.

[0004] Traditional machine learning methods, such as random forests and logistic regression, transform network traffic into structured data through traffic statistics and protocol features (packet count, byte count, etc.) with the penetration and large-scale application of 5G and 5G+ networks. However, these traditional feature engineering methods struggle to reflect the real-time characteristics of traffic and are weak in processing high-dimensional traffic data (such as encrypted traffic and multi-protocol mixed traffic), easily leading to low detection accuracy and high false alarm rates due to feature redundancy. Convolutional detection models based on CNNs or multi-resolution CNNs mainly utilize the local convolution and weight sharing characteristics of CNNs to capture local spatiotemporal features of traffic data at different scales, transforming traffic data into sequential data in chronological order (such as the feature composition sequence of each data packet). They extract local correlation features (such as the combination pattern of "short packet + long packet + short packet") through the sliding window of the convolutional kernel. However, in practical applications, they cannot model the global dependencies of long-sequence traffic (such as traffic pattern changes across time windows), resulting in excellent performance on the training set but a significant drop in accuracy on the test set. They also lack global feature constraints and have a high risk of overfitting.

[0005] To address this, existing technologies propose network anomaly detection methods based on Transformer-based multi-head attention mechanisms. These methods calculate the correlation weights between the features of each data packet and the features of other data packets in the sequence, capturing the global dependencies of traffic. This overcomes the limitations of local convolution in CNNs and models the correlations between long-term, non-adjacent data packets. While the model performs more accurately than CNNs, its computational complexity is increased. Furthermore, it focuses excessively on global dependencies and lacks detailed capture of local adjacent data packets (such as sudden changes in the direction of a single data packet or fluctuations in packet length over a short period). These local features are often key signals of early anomalies (such as the initial data packet features of a DDoS attack), leading to a high false negative rate for fine-grained anomalies.

[0006] Lightweight detection methods, such as the simplified version of MobileNet, single-branch lightweight CNN, and ShuffleNet-based models, reduce the total number of parameters and computational cost while maintaining a certain level of detection performance by simplifying the network structure, adopting efficient convolutional operations, and simplifying the classifier. However, in pursuit of lightweightness, the network structure and number of channels are excessively pruned, resulting in models that can only capture shallow local features and cannot learn complex traffic patterns (such as the implicit features of multi-protocol mixed traffic and encrypted traffic). The detection accuracy is generally low, the ability to subdivide anomaly types is poor, and the feature representation ability is insufficient. Moreover, they rely only on local convolution and simple pooling and cannot model the global dependencies of traffic. When faced with anomalies that require global judgment (such as traffic peak anomalies across time windows), the false negative rate is high. Summary of the Invention

[0007] In view of the defects and shortcomings of existing network anomaly traffic detection technologies, the purpose of this invention is to propose a lightweight contrast-enhanced attention detection model for network anomaly traffic detection. This method constructs an anomaly traffic detection model based on inverted residual moving blocks (iRMB) and a thin multi-head attention mechanism. The inverted residual moving block (iRMB) is used as the core structure. Through the modular design of depthwise separable convolutions and meta-moving blocks, the total number of parameters is reduced while capturing local spatiotemporal features, thus lightweighting the backbone network. Simultaneously, the thin multi-head attention mechanism retains the ability to capture global dependencies through parallel interactive computation, maintaining the advantages of overfitting resistance and large sample adaptability while reducing model computation and complexity.

[0008] In a further objective, this invention proposes a lightweight contrastive-enhanced attention detection model for network anomaly traffic detection. This method utilizes contrastive learning pre-training based on unlabeled traffic data to construct a normal traffic feature library. During model training, classification loss and contrastive loss are integrated, allowing the model to learn the consistency between local behavior and global interaction of benign traffic. This improves generalization ability in large-sample scenarios and enhances the robustness and accuracy of detection under multi-modal data transmission and unknown anomaly traffic.

[0009] According to a first aspect of the present invention, a method for detecting abnormal network traffic based on a lightweight contrast-enhanced attention detection model is provided, comprising: Step 1: Obtain a dataset covering normal and abnormal traffic, remove empty packets, duplicate packets, and erroneous data packets that fail verification, and retain only valid data packets containing the IP layer corresponding to the TCP / UDP protocol; Step 2: For each valid data packet, extract three types of core semantic features, construct a semantic attribute matrix (SAM), and perform normalization processing to obtain the normalized data matrix (SAM). norm The three core semantic features include: data packet length, transmission direction, and arrival interval. Step 3: Process the data matrix SAM norm Principal component analysis is performed to reduce dimensionality, eliminating redundant information between features, and obtaining the dimensionality-reduced feature matrix X. pca ; Step 4: Based on the dimensionality-reduced feature matrix X pca Based on the arrival time of data packets, the data is divided into segments using the time window method and labeled with sample tags to obtain the sample set {(X1,y1),(X2,y2),...,(X... M ,y M )}, where M represents the total number of samples, X k ∈R n_max×d k=1,2,3,…,M represents the feature matrix of the k-th sample; n_max represents the maximum number of data packets contained in each sample; y represents the sample label. If the data segment contains at least one abnormal data packet, then the sample label y=1; otherwise, y=0. Step 5: Construct an abnormal traffic detection model based on inverted residual moving blocks and thin multi-head attention mechanism, including an input layer, a feature upscaling layer, an inverted residual moving block backbone network, a thin multi-head attention fusion layer, a feature aggregation layer, and a two-layer classification output layer; Step 6: Use the training set based on the proportional division of the sample set to train the abnormal traffic detection model. During the training process, iterative training is performed by fusing the joint loss function of classification loss and contrastive loss and the AdamW optimizer to obtain the trained detection model. Step 7: Deploy the trained detection model on the cloud platform and / or edge device, capture packets in real time through the network interface, perform feature processing, and generate samples in real time according to the time window. There is no need to wait for the full amount of data. After the window length is reached, input the data into the model to detect abnormal traffic and output the normal / abnormal classification results.

[0010] According to a second aspect of the present invention, a computer system is provided, comprising: One or more processors; and Memory stores instructions that can be operated. When the instruction is executed by one or more processors, it causes the aforementioned one or more processors to perform operations, including the process of executing the aforementioned network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model.

[0011] In a third aspect of the invention, a computer-readable storage medium is provided for storing one or more programs, said one or more programs comprising instructions or instruction sets executable by one or more processors; When the instructions or instruction set are executed by one or more processors, they perform the aforementioned process of network anomaly traffic detection method based on lightweight contrast-enhanced attention detection model.

[0012] The network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model, as described in the above embodiments of the present invention, firstly extracts core features such as packet length, direction, and arrival interval from the network traffic dataset to construct a SAM matrix. After normalization and PCA dimensionality reduction, time window samples are constructed. Prior to training, a traffic feature library is built based on normal samples in the dataset to guide the model in learning the feature distribution of normal traffic during training, strengthening the model's memory of normal traffic features and improving generalization ability. Then, a multi-iRMB cascaded backbone network is used to extract multi-scale local spatiotemporal features using depthwise separable convolution. Combined with thin multi-head attention to fuse global dependencies, a classification + contrastive joint loss is used to train the model for detecting anomaly traffic in network transmission, enabling early identification of network attacks. Real-time detection and incremental updates are achieved through cloud / edge deployment.

[0013] In the network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model of this invention, a two-layer backbone structure based on an iRMB module empowered by depthwise separable convolution and a simplified multi-head attention mechanism reduces computational complexity while retaining the ability to capture global dependencies. Through local feature extraction by iRMB and global fusion of simplified attention, both local traffic details and global interaction patterns are taken into account. Furthermore, the feature aggregation layer compresses the sequence dimension, further reducing the computational cost of attention and improving the fine-grained anomaly detection score (F1 score).

[0014] It should be understood that all combinations of the foregoing concepts and the additional concepts described in more detail below may be considered part of the inventive subject matter of this disclosure, provided that such concepts do not contradict each other. Furthermore, all combinations of the claimed subject matter are considered part of the inventive subject matter of this disclosure.

[0015] The foregoing and other aspects, embodiments, and features of the teachings of the present invention will be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the invention, such as features and / or beneficial effects of exemplary embodiments, will become apparent from the following description or may be learned through practice of specific embodiments according to the teachings of the present invention. Attached Figure Description

[0016] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in the various figures may be denoted by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the invention will now be described by way of example and with reference to the accompanying drawings.

[0017] Figure 1 This is a flowchart illustrating a network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of constructing a sample set based on the time window method according to an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the process for constructing a normal traffic feature prototype library according to an embodiment of the present invention. Detailed Implementation

[0020] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0021] Various aspects of the invention are described in this disclosure with reference to the accompanying drawings, which illustrate numerous illustrative embodiments. The embodiments of this disclosure are not necessarily intended to encompass all aspects of the invention. It should be understood that the various concepts and embodiments described above, as well as those described in more detail below, can be implemented in any of many ways, because the concepts and embodiments disclosed herein are not limited to any particular implementation. Furthermore, some aspects of the invention disclosed may be used alone or in any suitable combination with other aspects of the invention disclosed.

[0022] {Example 1} Combination Figure 1 As shown, the network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to an embodiment of the present invention includes the following process: Step 1: Obtain a dataset covering normal and abnormal traffic, remove empty packets, duplicate packets, and erroneous data packets that fail verification, and retain only valid data packets containing the IP layer corresponding to the TCP / UDP protocol; Step 2: For each valid data packet, extract three types of core semantic features, construct a semantic attribute matrix (SAM), and perform normalization processing to obtain the normalized data matrix (SAM). norm The three core semantic features include: data packet length, transmission direction, and arrival interval. Step 3: Process the data matrix SAM norm Principal component analysis is performed to reduce dimensionality, eliminating redundant information between features, and obtaining the dimensionality-reduced feature matrix X. pca ; Step 4: Based on the dimensionality-reduced feature matrix X pca Based on the arrival time of data packets, the data is divided into segments using the time window method and labeled with sample tags to obtain the sample set {(X1,y1),(X2,y2),...,(X... M ,y M )}, where M represents the total number of samples, X k ∈R n_max×d k=1,2,3,…,M represents the feature matrix of the k-th sample; n_max represents the maximum number of data packets contained in each sample; y represents the sample label. If the data segment contains at least one abnormal data packet, then the sample label y=1; otherwise, y=0. Step 5: Construct an abnormal traffic detection model based on inverted residual moving blocks and thin multi-head attention mechanism, including an input layer, a feature upscaling layer, an inverted residual moving block backbone network, a thin multi-head attention fusion layer, a feature aggregation layer, and a two-layer classification output layer; Step 6: Use the training set based on the proportional division of the sample set to train the abnormal traffic detection model. During the training process, iterative training is performed by fusing the joint loss function of classification loss and contrastive loss and the AdamW optimizer to obtain the trained detection model. Step 7: Deploy the trained detection model on the cloud platform and / or edge device, capture packets in real time through the network interface, perform feature processing, and generate samples in real time according to the time window. There is no need to wait for the full amount of data. After the window length is reached, input the data into the model to detect abnormal traffic and output the normal / abnormal classification results.

[0023] As an optional implementation, in step 1, the dataset covering normal traffic and abnormal traffic is one or more of the public datasets CSE-CIC-IDS2018, UNSW-NB15, and KDDCup99, or a dataset constructed using historical PCAP format packet capture data. The proportion of abnormal traffic in the dataset (including DDoS attacks, port scans, and abnormal encrypted traffic) is controlled between 10% and 30%. Preferably, the total number of samples in a single dataset is no less than 100,000.

[0024] As an optional implementation, step 2 includes the extraction of the three types of core semantic features: The data packet length, measured in bytes, is obtained by extracting the total number of bytes in the data packet, denoted as . l i ,in i Indicates the sequence number of the data packet; For the transmission direction, binary encoding is used, denoted as . d i Define uplink d i =1, downlink d i =0; The arrival time interval, in milliseconds (ms), is denoted as... t i By calculating the first i The data packet and the first i -1 data packet time difference is obtained; if it is the first data packet, then t 1 = 0; Construct a semantic attribute matrix (SAM) based on packet length, transmission direction, and arrival time interval: SAM=[L,D,T]∈R n×3 , where n represents the number of data packets contained in a single sample; Where L=[ l 1, l 2, l 3…, l n ] T D=[ d 1, d 2, d 3…, d n ] T T=[ t 1, t 2, t 3…, t n ] T These represent the length vector, direction vector, and time interval vector, respectively.

[0025] In an embodiment of the present invention, the Min-Max standard method is used to normalize the semantic attribute matrix SAM, normalize the non-binary features (data packet length, arrival interval T) and map the feature values ​​to the [0,1] interval to eliminate dimensional differences.

[0026] Since the directional features are already binary data, no normalization is required.

[0027] The normalized data matrix is denoted as SAM norm .

[0028] As an optional implementation, in step 3, perform principal component analysis dimensionality reduction on the data matrix SAM norm to eliminate redundant information between features and obtain the dimensionality-reduced feature matrix X pca , including: For the data matrix SAM norm , by using the principal component analysis method, solve the eigenvalues and eigenvectors of the feature covariance matrix, and select the principal components with a cumulative variance contribution rate ≥ 95% as the final input features to obtain the dimensionality-reduced feature matrix X pca , X pca ∈R n×d : The feature matrix X pca =SAM norm ⋅W pca ; where, W pca represents the projection matrix of PCA, which is composed of the eigenvectors corresponding to the first <--snip-->[ d largest eigenvalues; d represents the dimensionality of the features after dimensionality reduction.

[0029] In the embodiments of the present invention, through principal component analysis PCA dimensionality reduction, redundant information between features is eliminated, and the input dimension of the model is further reduced.

[0030] As an optional embodiment, the dimensionality of the features after dimensionality reduction d takes the value of 3. In another embodiment, the dimensionality of the features after dimensionality reduction d is adjustable. For example, if the feature redundancy is high, it can be adjusted to 2.

[0031] As an optional embodiment, in step 4, based on the dimensionality-reduced feature matrix X pca , in the order of the arrival time of data packets, use the time window method to divide data segments and perform sample label annotation to obtain the sample set {(X1, y1), (X2, y2),..., (X / / Note: The ellipsis and the following part seem to be incomplete in the original. Here's the best translation based on what's provided M / / The ellipsis and the following part seem to be incomplete in the original. Here's the best translation based on what's provided , y M / / The ellipsis and the following part seem to be incomplete in the original. Here's the best translation based on what's provided )}, including: For the dimensionality-reduced feature matrix X pca , in the order of the arrival time of data packets, divide data segments at intervals of the time window length T window , and each data segment corresponds to 1 sample; If the number of data packets n in the data segment < n_max, then fill it with 0 vectors to n_max data packets, i.e.: Xpad = pad(Xpca, (0, n_max - n), 0);<000023If the number of data packets n in the data segment is greater than or equal to n_max, then the first n_max data packets are truncated as samples. Label the samples. If a data segment contains at least one abnormal data packet, the sample label y=1; otherwise, y=0. Construct the complete sample set {(X1,y1),(X2,y2),...,(X...} M ,y M )}.

[0032] For example, the sample is divided using a fixed time window method, with a set time window length T. window =10s (can be adjusted to 5s or 15s, etc., depending on network traffic density). Considering sample representativeness and model complexity, the maximum number of data packets in each sample is set to n_max = 150. Furthermore, combined with... Figure 2 As shown, the process of dividing and labeling data segments using the time window method is as follows: Step 4-1: Divide the data packets into segments at 10-second intervals according to their arrival time, with each data segment corresponding to one sample; Step 4-2: If the number of data packets in the data segment is less than 150, then fill it with a 0 vector to make it 150 data packets; If the number of data packets in a data segment is more than 150, then the first 150 data packets are truncated as samples. Step 4-3: Label the samples. If the data segment contains at least one abnormal data packet, the sample label y=1 (indicating abnormal traffic); otherwise, y=0 (indicating normal traffic). Step 4-4: Integrate the labeled samples to construct a complete sample set, represented as: {(X1,y1),(X2,y2),...,(X M ,y M )}; Where M represents the total number of samples, X k ∈R n_max×d , k=1,2,3,…,M, represents the feature matrix of the k-th sample; n_max represents the maximum number of data packets contained in each sample.

[0033] Furthermore, the sample set can be divided according to a certain ratio. For example, the constructed sample set can be divided into a training set, a validation set, and a test set in a ratio of 7:2:1. The training set is used for parameter learning during the model training phase, the validation set is used for hyperparameter tuning and overfitting monitoring during the training process, and the test set is used for the final model performance evaluation.

[0034] As an alternative, the partitioning process can employ stratified sampling to ensure that the proportion of normal / abnormal samples in each set is consistent with that in the original sample set, thus avoiding data distribution bias.

[0035] Furthermore, prior to training, a prototype library of normal traffic features is constructed based on unlabeled normal samples (y=0) in the training set. This provides a reference benchmark for comparative learning, improves the model's feature recognition and generalization ability for normal traffic, and converts it into a normal traffic feature library T in tensor format. tensor (Such as tensor types supported by deep learning frameworks (such as PyTorch and TensorFlow), used for subsequent operations, such as supporting automatic differentiation, GPU acceleration and other necessary operations for model training. The format conversion is usually completed by the framework functions of the deep learning framework, especially following the construction steps of the feature library.

[0036] As an optional implementation, the aforementioned normal traffic feature library T tensor It is constructed based on normal samples with y=0 in the training set, combined with Figure 3 As shown, it includes the following processes: Select all normal samples with y=0 from the training set and construct a normal sample subset containing N normal samples; For each normal sample, the mean vector of its feature matrix is ​​calculated as a local prototype of a single normal flow. Set the number of prototypes in the normal traffic feature prototype library to m_proto, and divide the normal sample subset into m_proto subsets evenly according to the sample distribution. Each subset contains ⌊N / m_proto⌋ samples. Calculate the mean of the local prototypes within each of the m_proto subsets, and use this as the final global prototype. P j Obtain the normal traffic feature library P, P={ P 1, P 2, P 3,…, P m_proto}; Before model training, the framework function is called to perform tensor format conversion of the normal traffic feature library P, obtaining the normal traffic feature library T in tensor format. tensor .

[0037] In optional embodiments, the number of prototypes m_proto typically ranges from 8 to 15. In this example, a value of 10 is used for illustration.

[0038] As an optional implementation, in step 5, an abnormal traffic detection model based on inverted residual moving blocks and thin multi-head attention mechanism is constructed, including an input layer, a feature upscaling layer, an inverted residual moving block backbone network, a thin multi-head attention fusion layer, a feature aggregation layer, and a two-layer classification output layer.

[0039] The feature upscaling layer is linked to the input layer to map the PCA-reduced input features X, which are input through the input layer, to the hidden layer dimension of the model, so as to adapt to the feature extraction requirements of the subsequent iRMB module.

[0040] The feature enhancement layer is implemented using a fully connected layer, with the following parameter settings: Input Dimensions d The output dimension (i.e., the number of hidden layer channels) is c hidden The default value is 16, which can be adjusted to 8-32 dimensions to meet lightweight requirements. The activation function used is ReLU. The mathematical expression is: X proj = s (W proj ⋅X+b proj ); Among them, W proj Let b be the weight matrix. proj For bias vectors, s () represents the activation function, X proj This represents the feature matrix after dimensionality increase.

[0041] As an optional implementation, the inverted residual moving block backbone network is composed of multiple inverted residual moving blocks connected in series, which is used to capture the multi-scale local spatiotemporal features of traffic data, while reducing computational complexity through depthwise separable convolution.

[0042] Each inverted residual shift block includes a 1×1 upscaling convolution, a depthwise separable convolution with 3×1 kernels, a meta-shift calibration layer, a 1×1 downscaling convolution, and a residual connection layer. The input of a single inverted residual shift block is added to the downscaling features via residual connections, and then transposed back to the sequence before outputting in the first-order format. The output of the concatenated three inverted residual shift blocks is X. irmb This is used to accumulate and capture multi-scale local features ranging from fine-grained to coarse-grained.

[0043] As a specific example, the design structure and operation of each inverted residual moving block are as follows: 1) 1×1 upscaling convolution: increases the number of channels in the input features from... c hidden Expand to c mid : c mid =⌊ c hidden * c ⌋, c This represents the expansion factor, with a value of 1.5; 2) Depthwise separable convolution: Uses a 3×1 convolution kernel (suitable for sequential data), with the number of groups equal to... cmid Convolution is performed only on features within a single channel, reducing computational cost. The formula is as follows: X depth =DepthConv(X expand W depth ); Among them, X expand W depth These represent the output of a 1×1 up-dimensional convolution and the weights of a depthwise convolution, respectively. DepthConv() represents the depthwise convolution operation. 3) Meta-shift calibration: This enhances the robustness of feature representation by adjusting the feature distribution using adaptive coefficients. The formula is: X calib =X depth + α BN(X depth ); in, α This represents the learnable calibration coefficient (initialized to 1), and BN() represents the batch normalization operation. 4) 1×1 dimensionality reduction convolution: reduces the number of channels from... c mid Restore to c hidden Complete the dimension matching of residual connections; 5) Residual connection: Add the module input to the dimensionality-reduced features, using the following formula: X out =X proj +X in T ; Among them, X in This indicates the module input (i.e., the output of the previous iRMB module or the output of the feature upscaling layer). Then, by transposing the sequence back to the previous format X... out T This is the final output of a single iRMB module.

[0044] In embodiments of the present invention, a simplified multi-head attention mechanism is employed, namely, a thin multi-head attention fusion layer. h Group attention head count h Less than or equal to 4. And the dimension of each attention group. d k for c hidden / h , c hidden This represents the number of channels in the input features; multi-head fusion is then used to... h The attention outputs are concatenated and mapped to a unified dimension through a fully connected layer.

[0045] As an alternative example, the output X of the three inverted residual moving blocks concatenated by three independent fully connected layers is... irmb (As input to the thin multi-head attention fusion layer) it is mapped as a query, key, and value matrix.

[0046] Then, through multi-head splitting, Q, K, and V are split according to the number of attention heads to obtain the local Q / K / V of each attention group; then, through single-head attention calculation, the attention weights and outputs are calculated for each Q / K / V group; finally, multi-head fusion is performed. h The attention outputs are concatenated and then mapped to a unified dimension through a fully connected layer.

[0047] In the embodiments of the present invention, the aforementioned head number splitting of Q, K, V, single-head attention weight calculation and multi-head fusion processing can be implemented based on existing technologies, and will not be described in detail here.

[0048] As an optional embodiment, the feature aggregation layer uses adaptive average pooling to aggregate the sequence features after attention fusion, compressing the variable-length sequence of length n_max into a fixed-dimensional global feature vector X. pool The processing includes: ; Among them, Attn T This represents the transposed feature matrix, and the output after pooling is the global feature vector X. pool That is, the dimension is c hidden The global feature vector.

[0049] As an optional embodiment, the two-layer classification output layer uses two fully connected layers to construct the classifier: The input dimension of the first layer is the number of channels of the input features. c hidden The output dimension is 32, the activation function is ReLU, and the dropout probability is 0.2 to suppress overfitting; The second layer has an input dimension of 32 and an output dimension of 2, i.e., binary classification: normal / abnormal. There is no activation function, and the probability is calculated by Softmax.

[0050] As an optional embodiment, during the training process in step 6, the joint loss function that fuses the classification loss and the contrastive loss is expressed as: L n = L cls +λL cont ; In the formula, Lcls This represents the classification loss, used to guide the optimization of normal / abnormal classification accuracy during training; L cont This represents the contrastive loss, used to guide the optimization of normal / abnormal classification accuracy during training; l This represents the weight of the comparative loss, with a value ranging from 0.05 to 0.2.

[0051] As a concrete example, classification loss L cls The expression is as follows: ; in, y k Let y be the one-hot encoded label of the sample, where y=[1,0] for normal samples and y=[0,1] for abnormal samples. Softmax(logits) k For the first k The predicted probability of the class; logits is the classification logits value.

[0052] As a concrete example, consider the contrast loss. L cont The expression is as follows: ; Where S(,) represents the cosine similarity operation. t This represents the temperature coefficient used to control the smoothness of the similarity distribution; in this example, it is set to 0.7. P j To be with X pool The most similar normal prototype is obtained by solving argmax.

[0053] As an optional implementation, the parameters of the AdamW optimizer are set as follows during training: Initial learning rate or =1e-4, which can be adjusted to 2e-4 for large sample scenarios; Weight decay l delay =1e-5, used to suppress model overfitting; Momentum parameters β 1 = 0.9 β 2 = 0.999; Epsilon value: 1e-8, to avoid a denominator of 0.

[0054] As an optional embodiment, the training parameters are set as follows during the training process: Batch size: 32~128; Training epochs: 30 (the learning rate is fixed for the first 20 epochs, and decays to 1e-6 using cosine annealing for the last 10 epochs); Gradient clipping: Set the maximum norm of the gradient to 1.0 to avoid gradient explosion; Early stopping strategy: If the accuracy on the validation set does not improve for 5 consecutive rounds, stop training and save the current best model.

[0055] Furthermore, the model training results are validated using a validation set.

[0056] In an optional embodiment, model performance is evaluated by accuracy (Accuracy, Acc), F1 score (F1-Score), and false alarm rate (False Alarm Rate, FAR, the proportion of normal traffic that is misclassified as abnormal).

[0057] The model is evaluated using the validation set after each round of training, and three metrics are recorded: Acc, F1, and FAR. After training, the best model (with the highest Acc in the validation set) is evaluated using the test set to ensure the model's generalization ability.

[0058] Furthermore, deployment is carried out based on the optimal model obtained through training, including deployment on cloud platforms and edge devices (such as routers and gateways).

[0059] For example, on the cloud platform: GPU inference computing is used to process traffic data in batches, the inference batch size is set to 256, and the single sample inference latency is ≤28ms; Edge devices (taking routers as an example): CPU inference (ARM Cortex-A53 / A72) is used to compress the model with INT8 quantization (quantization-aware training QAT), the total number of parameters is compressed to ≤1M, and the single-sample inference latency is ≤50ms, which meets the real-time detection requirements.

[0060] The real-time detection process mainly includes the following steps: 1. Packet capture: Capture packets in real time through the network interface (using the libpcap library), with the capture rate adapted to the network bandwidth (10ms packet capture interval for speeds below 100Mbps, and 1ms interval for speeds above 1Gbps). 2. Real-time preprocessing: The captured data packets are subjected to feature extraction, normalization, and PCA dimensionality reduction according to the process described in the previous embodiment (using the Min-Max scaling parameters and PCA projection matrix saved during the training phase). 3. Sample generation: Samples are generated in real time according to the aforementioned time window (e.g., 10 seconds), without waiting for the full amount of data. Once the window length is reached, the samples are immediately input into the model. 4. Model inference output: Call the deployed LCADM model, output the classification result (normal / abnormal), and feed it back to the network management system in real time; 5. Anomaly Alarm: If an abnormal sample is detected, record the time of occurrence, data packet characteristics, and anomaly probability, triggering the alarm mechanism and subsequent control and protection measures.

[0061] In embodiments of the present invention, incremental updates can also be performed periodically. For example, new network traffic data (including new anomaly types) can be collected for each period (e.g., 30 days), and an incremental sample set can be constructed according to the aforementioned process. Existing model weights can be loaded for incremental training (10 training rounds, learning rate 5e-5) to avoid the resource consumption of retraining.

[0062] In an optional embodiment, the model performance is monitored by real-time statistical model detection metrics (Acc, F1, FAR). If FAR exceeds 5% for 7 consecutive days or Acc is below 85%, the threshold label can be configured to trigger the model update process.

[0063] In optional embodiments, the feature library can also be incrementally updated periodically. The normal traffic feature library can be rebuilt according to each cycle (such as configurable cycles of 30 days, 60 days, 90 days, etc.), and new normal traffic patterns can be incorporated to improve the model's adaptability to changes in the network environment.

[0064] {Example 2} According to embodiments of the network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model described above, a computer system is also disclosed according to the present invention, comprising: One or more processors; and Memory stores instructions that can be operated.

[0065] When the instructions are executed by one or more processors, they cause the aforementioned one or more processors to perform operations, including the process of executing the network anomaly traffic detection method based on the lightweight contrast-enhanced attention detection model of the aforementioned embodiments.

[0066] {Example 3} Based on the embodiments of the network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model described above, the present invention also discloses a computer-readable storage medium for storing one or more programs. The one or more programs include instructions or instruction sets executable by one or more processors.

[0067] When the instructions or instruction set are executed by one or more processors, they perform the process of the network abnormal traffic detection method based on the lightweight contrast-enhanced attention detection model described in the foregoing embodiments.

[0068] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for detecting abnormal network traffic based on a lightweight contrast-enhanced attention detection model, characterized in that, include: Step 1: Obtain a dataset covering normal and abnormal traffic, remove empty packets, duplicate packets, and erroneous data packets that fail verification, and retain only valid data packets containing the IP layer corresponding to the TCP / UDP protocol; Step 2: For each valid data packet, extract three types of core semantic features, construct a semantic attribute matrix (SAM), and perform normalization processing to obtain the normalized data matrix (SAM). norm The three core semantic features include: data packet length, transmission direction, and arrival interval. Step 3: Process the data matrix SAM norm Principal component analysis is performed to reduce dimensionality, eliminating redundant information between features, and obtaining the dimensionality-reduced feature matrix X. pca ; Step 4: Based on the dimensionality-reduced feature matrix X pca Based on the arrival time of data packets, the data is divided into segments using the time window method and labeled with sample tags to obtain the sample set {(X1,y1),(X2,y2),...,(X... M ,y M )}, where M represents the total number of samples, X k ∈R n_max×d k=1,2,3,…,M represents the feature matrix of the k-th sample; n_max represents the maximum number of data packets contained in each sample; y represents the sample label. If the data segment contains at least one abnormal data packet, then the sample label y=1; otherwise, y=0. Step 5: Construct an abnormal traffic detection model based on inverted residual moving blocks and thin multi-head attention mechanism, including an input layer, a feature upscaling layer, an inverted residual moving block backbone network, a thin multi-head attention fusion layer, a feature aggregation layer, and a two-layer classification output layer; Step 6: Use the training set based on the proportional division of the sample set to train the abnormal traffic detection model. During the training process, iterative training is performed by fusing the joint loss function of classification loss and contrastive loss and the AdamW optimizer to obtain the trained detection model. Step 7: Deploy the trained detection model on the cloud platform and / or edge devices, capture packets in real time through the network interface, perform feature processing, and generate samples in real time according to the time window. There is no need to wait for the full amount of data. After the window length is reached, input the data into the model to detect abnormal traffic and output the normal / abnormal classification results. In step 5, an abnormal traffic detection model based on inverted residual moving blocks and a thin multi-head attention mechanism is constructed, wherein: The inverted residual moving block backbone network consists of multiple inverted residual moving blocks connected in series, used to capture multi-scale local spatiotemporal features of traffic data. Each inverted residual moving block includes a 1×1 up-dimensional convolution, a depthwise separable convolution with 3×1 kernels, a meta-movement calibration layer, a 1×1 down-dimensional convolution, and a residual connection layer. The residual connection layer adds the input of a single inverted residual moving block to the down-dimensional features, transposes it back to the sequence-first format, and outputs the result. The output of three inverted residual moving blocks connected in series is denoted as X. irmb This is used to accumulate and capture multi-scale local features ranging from fine-grained to coarse-grained. The thin multi-head attention fusion layer adopts h Group attention head count h Less than or equal to 4, and the dimension of each attention group is . c hidden / h , c hidden This represents the number of channels in the input features; multi-head fusion is then used to... h The attention outputs of the groups are concatenated and mapped to a unified dimension through a fully connected layer. The feature aggregation layer uses adaptive average pooling to aggregate the sequence features after attention fusion, compressing the variable-length sequence of length n_max into a global feature vector of fixed dimensions. The two-layer classification output layer uses two fully connected layers to construct the classifier, where the input dimension of the first layer is the number of channels of the input features. c hidden The first layer has an output dimension of 32, an activation function of ReLU, and a dropout probability of 0.

2. The second layer has an input dimension of 32 and an output dimension of 2, which is a binary classification: normal / abnormal. There is no activation function, and the probability is calculated by Softmax.

2. The network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to claim 1, characterized in that, In step 1, the dataset covering normal and abnormal traffic is one or more of the public datasets CSE-CIC-IDS2018, UNSW-NB15, and KDDCup99, or a dataset constructed using historical PCAP format packet capture data. Among them, the proportion of abnormal traffic in the data is controlled between 10% and 30%.

3. The network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to claim 1, characterized in that, Step 2 involves the extraction of three types of core semantic features, including: The data packet length, measured in bytes, is obtained by extracting the total number of bytes in the data packet, denoted as . l i ,in i Indicates the sequence number of the data packet; For the transmission direction, binary encoding is used, denoted as . d i Define uplink d i =1, downlink d i =0; The arrival time interval, in milliseconds, is denoted as... t i By calculating the first i The data packet and the first i -1 data packet time difference is obtained; if it is the first data packet, then t 1 = 0; Construct a semantic attribute matrix (SAM) based on packet length, transmission direction, and arrival time interval: SAM=[L,D,T]∈R n×3 , where n represents the number of data packets contained in a single sample; Where L=[ l 1, l 2, l 3…, l n ] T D=[ d 1, d 2, d 3…, d n ] T T=[ t 1, t 2, t 3…, t n ] T These represent the length vector, direction vector, and time interval vector, respectively.

4. The network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to claim 1, characterized in that, In step 3, the data matrix SAM norm Principal component analysis is performed to reduce dimensionality, eliminating redundant information between features, and obtaining the dimensionality-reduced feature matrix X. pca ,include: For the data matrix SAM norm By using principal component analysis, the eigenvalues ​​and eigenvectors of the feature covariance matrix are solved. Principal components with a cumulative variance contribution rate ≥ 95% are selected as the final input features to obtain the dimensionality-reduced feature matrix X. pca X pca ∈R n×d : Characteristic matrix X pca =SAM norm ⋅W pca ; Among them, W pca The projection matrix of PCA is composed of the eigenvectors corresponding to the first d largest eigenvalues; d represents the feature dimension after dimensionality reduction, and n represents the number of data packets contained in a single sample.

5. The network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to claim 1, characterized in that, In step 4, based on the dimensionality-reduced feature matrix X pca Based on the arrival time of data packets, the data is divided into segments using the time window method and labeled with sample tags to obtain the sample set {(X1,y1),(X2,y2),...,(X... M ,y M )},include: For the dimensionality-reduced feature matrix X pca According to the arrival time of data packets, in order of time window length T window The data is divided into segments at intervals, with each segment corresponding to one sample; If the number of data packets n in the data segment is less than n_max, then fill it with 0 vectors to n_max data packets, that is: Xpad = pad(X pca ,(0, n_max - n), 0); If the number of data packets n in the data segment is greater than or equal to n_max, then the first n_max data packets are truncated as samples. Label the samples. If a data segment contains at least one abnormal data packet, the sample label y=1; otherwise, y=0. Construct the complete sample set {(X1,y1),(X2,y2),...,(X...} M ,y M )}.

6. The network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to claim 1, characterized in that, During the training process in step 6, the joint loss function that fuses the classification loss and the contrastive loss is expressed as: L n = L cls + λL cont ; In the formula, L cls This represents the classification loss, used to guide the optimization of normal / abnormal classification accuracy during training; L cont This represents the contrastive loss, used to guide the optimization of normal / abnormal classification accuracy during training; λ This represents the weight of the comparative loss, with a value ranging from 0.05 to 0.

2.

7. The network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model according to claim 6, characterized in that, Normal flow feature library T tensor The process of constructing the system based on normal samples where y=0 in the training set includes the following steps: Select all normal samples with y=0 from the training set and construct a normal sample subset containing N normal samples; For each normal sample, the mean vector of its feature matrix is ​​calculated as a local prototype of a single normal flow. Set the number of prototypes in the normal traffic feature prototype library to m_proto, and divide the normal sample subset into m_proto subsets evenly according to the sample distribution. Each subset contains ⌊N / m_proto⌋ samples. Calculate the mean of the local prototypes within each of the m_proto subsets, and use this as the final global prototype. P j Obtain the normal flow characteristics P, P={ P 1, P 2, P 3,…, P m_proto }; Before model training, the framework function is called to perform tensor format conversion of the normal traffic feature P, obtaining the normal traffic feature library T in tensor format. tensor .

8. A computer system, characterized in that, include: One or more processors; as well as Memory stores instructions that can be operated. When the instructions are executed by one or more processors, they cause the aforementioned one or more processors to perform operations, including the process of executing the network anomaly traffic detection method based on a lightweight contrast-enhanced attention detection model as described in any one of claims 1-7.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions or instruction sets that can be executed by one or more processors; When the instructions or instruction set are executed by one or more processors, they perform the process of the network abnormal traffic detection method based on the lightweight contrast-enhanced attention detection model as described in any one of claims 1-7.

Images