A semi-supervised anomaly traffic detection method for internet of things based on perception-like contrast

Abstract

The application discloses a kind of semi-supervised abnormal traffic detection method of class perception contrast Internet of Things, solve the problem of serious imbalance of abnormal attack traffic data class distribution and insufficient labeled samples in the prior art in Internet of Things system, comprising: the input Internet of Things traffic data is preprocessed and data is enhanced, generates pseudo label;According to the feature vector extracted after enhancement sample, the similarity between features is dynamically calculated and weighted using self-attention mechanism;Supervised contrast matrix and class perception contrast matrix are constructed, intra-class aggregation is formed in feature space, and inter-class separation is formed;Loss function of semi-supervised anomaly detection model is constructed, and model parameters are iteratively updated, and the trained model is used for abnormal traffic detection.Semi-supervised learning mode is used, class perception contrast and self-attention feature weighting are designed simultaneously, and the detection effect of the anomaly detection model for all abnormal attack classes is improved.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of wireless communication and artificial intelligence, and in particular to a semi-supervised method for detecting abnormal traffic in the Internet of Things based on a perception-based contrastive approach. Background Technology

[0002] The widespread deployment and interconnectivity of Internet of Things (IoT) devices have greatly promoted the development of wireless communication technology. As an important component of wireless communication, IoT has been widely applied in various fields such as smart manufacturing, intelligent transportation, and environmental monitoring. However, due to the characteristics of IoT devices, such as low power consumption, limited resources, and large-scale connectivity, these devices have limitations in network transmission, data processing, and storage, making them vulnerable to threats such as cyberattacks. With the rapid growth in the number of connected IoT devices, their security and reliability have become increasingly important concerns. Device anomalies or attacks could lead to data leaks, service interruptions, and even significant losses to the industrial chain and the economy. Ensuring the secure operation of IoT devices and improving anomaly detection capabilities has become an important and urgent task.

[0003] In recent years, anomaly detection models based on machine learning or deep learning have gradually become mainstream. Although they have achieved certain detection results compared to traditional methods, the training of most models relies on a large number of labeled attack samples. On December 19, 2023, the Chinese Patent Office published patent CN117254951A: an abnormal traffic detection method, model training method, and related apparatus. This method acquires labeled stream sequences of multiple traffic packets, where the labeled stream sequences are obtained by labeling multiple traffic packets; it uses a pre-trained language model to perform anomaly detection on the labeled stream sequences, where the pre-trained language model is trained using normal sample packets. This also requires labeling a large number of traffic packets.

[0004] However, in real-world industrial scenarios, labeled attack sample data is often difficult to obtain, and the distribution of attack categories in existing datasets is severely imbalanced. These issues limit the effective learning of anomaly detection models by different attack category features, thus affecting their detection performance. Therefore, developing an efficient anomaly detection method that can fully utilize both a small amount of labeled data and a large amount of unlabeled data is of significant theoretical and practical value for improving the overall security and operational stability of IoT devices. Summary of the Invention

[0005] The purpose of this invention is to address the problems of severely unbalanced distribution of abnormal attack traffic data categories and insufficient labeled samples in existing IoT systems, which lead to low accuracy in detecting abnormal traffic. This invention provides a class-aware contrastive semi-supervised IoT abnormal traffic detection method. It improves the generalization ability of the anomaly detection model for unlabeled data through semi-supervised learning, and designs class-aware contrastive and self-attention feature weighting to enhance the model's ability to distinguish features, especially its feature focusing ability for small-category abnormal attack types, thereby improving the detection performance of the anomaly detection model for all types of abnormal attacks.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A semi-supervised method for detecting abnormal traffic in the Internet of Things (IoT) based on a perceptual comparison model includes the following steps: S1: Preprocess and augment the input IoT traffic data to generate pseudo-tags; S2: Extract feature vectors from the enhanced samples, and dynamically calculate and weight the similarity between features using a self-attention mechanism; S3: Construct supervised contrast matrix and class-aware contrast matrix to form intra-class clustering and inter-class separation in the feature space; S4: Construct the loss function of the semi-supervised anomaly detection model, iteratively update the model parameters, and use the trained model to detect abnormal traffic.

[0007] This invention addresses the challenges of detecting abnormal traffic in the Internet of Things (IoT) by addressing issues such as significant differences in the number of attack types across different categories, imbalanced class distribution, and insufficient labeled training data. It designs a semi-supervised anomaly detection model that fully utilizes both a small amount of labeled data and a large amount of unlabeled data for training. Simultaneously, it employs an attention mechanism for data feature enhancement, enabling adaptive attention to different types of samples. A class-aware contrastive learning approach guides similar samples to approach each other in the feature space while dissimilar samples move away from each other. A loss function is used to train the model, effectively solving the problem of detecting abnormal traffic from imbalanced multi-class samples in the IoT environment, thus improving the accuracy and robustness in detecting imbalanced multi-class anomaly traffic.

[0008] Preferably, the step of dynamically calculating and weighting the similarity between features using a self-attention mechanism includes: mapping the feature matrix to a query space, key space, and value space through a linear transformation; calculating the similarity between the query matrix and the key matrix to measure the degree of association between each feature; obtaining attention weights using an activation function, converting the similarity matrix into a probability distribution, and normalizing each row of the similarity matrix using an activation function; and applying the attention matrix to the value matrix to obtain a weighted feature representation.

[0009] Preferably, step S3 includes: mapping the attention-enhanced features to the contrastive learning embedding space to obtain a low-order embedding vector; constructing sample pairs using features generated by the mapping network: constructing positive samples using class labels and pseudo-labels; for samples of the same class, if their labels or pseudo-labels are the same, selecting them as positive sample pairs; for each sample with a label or pseudo-label, selecting samples with different labels or pseudo-labels as negative samples; constructing a supervised contrast matrix based on the true class of the labeled data; and generating a class-aware contrast matrix based on the true label of the labeled data and the pseudo-label of the unlabeled data.

[0010] Preferably, step S1 includes: preprocessing the input IoT traffic data and dividing the dataset; performing strong and weak enhancement on the preprocessed data; encoding the enhanced features; using a classification head to initially generate prediction results; and taking the category with the highest confidence as the pseudo-label.

[0011] Preferably, generating pseudo-labels includes: passing the weakly augmented unlabeled data samples through the classification head of a semi-supervised anomaly detection model to obtain the probability distribution of each sample belonging to each category; for each sample, selecting the category with the highest probability as the initial pseudo-label for that sample; for each sample, if its maximum category probability exceeds a preset confidence threshold, the pseudo-label is retained, otherwise it is discarded; the selected high-confidence samples are combined with their pseudo-labels to obtain a pseudo-label dataset, and the pseudo-label dataset is added to the labeled training data.

[0012] As a preferred approach, high-confidence pseudo-labels are used for pseudo-label loss calculation in semi-supervised learning: for each high-confidence pseudo-label sample, cross-entropy loss is calculated using the pseudo-label and the prediction results of the semi-supervised anomaly detection model.

[0013] Preferably, S2 includes extracting high-dimensional feature representations from the enhanced samples and mapping the high-dimensional features to a low-dimensional embedding space: inputting the pseudo-label enhanced samples into an encoder to extract features, the encoder including three one-dimensional convolutional layers, each followed by batch normalization and activation functions; adding pooling layers after the convolutional layers, flattening the pooling layer features, and converting them into fixed-size feature representations through fully connected layers; inputting the encoded feature representations to obtain feature vectors.

[0014] Preferably, the preprocessing of the input data and the partitioning of the dataset includes: mapping the input IoT traffic data into numerical features; mapping the numerical features to the [0, 1] interval using linear normalization; randomly sampling p% of samples as labeled data, (1-p)% of samples as unlabeled data, extracting per% of the labeled data and all the unlabeled data as training data, extracting the remaining px% from the labeled data for parameter tuning and validation of the semi-supervised anomaly detection model, and using the remaining labeled data as a test set to evaluate the final performance of the semi-supervised anomaly detection model.

[0015] Preferably, the data augmentation includes: weak augmentation, which applies random changes to the samples; strong augmentation, which randomly sets some features to zero to simulate feature loss, slightly perturbs the class labels of minority class samples, and generates samples from different perspectives; the strongly augmented samples are used to construct positive and negative sample pairs.

[0016] Preferably, in step S4: the loss function includes supervised loss, pseudo-label loss, and class-aware contrastive loss; the parameters of the semi-supervised anomaly detection model are iteratively updated using backpropagation and gradient descent optimization algorithms to minimize the loss function; after the model training is completed, the semi-supervised anomaly detection model is used to infer new input IoT traffic data to generate category labels and confidence scores.

[0017] Therefore, the present invention has the following beneficial effects: 1. Enhanced ability to identify minority class anomalies: By combining semi-supervised learning and class-aware contrastive learning, this invention can make full use of unlabeled data when labeled data is limited, enhance the model's ability to learn and identify features of minority class anomalies, and solve the multi-class imbalance problem.

[0018] 2. Enhance the adaptability and accuracy of the anomaly detection model to IoT traffic data: By introducing a dynamic self-attention mechanism and a data augmentation module, the anomaly detection model can focus on key features and suppress noise, thereby improving the detection accuracy and robustness in diverse and complex traffic environments. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall steps of the semi-supervised IoT abnormal traffic detection method based on class perception and comparison in this invention.

[0020] Figure 2 This is a flowchart of the data augmentation steps in this invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1: Currently, implementing IoT abnormal traffic detection methods in real-world environments requires analysts to spend a significant amount of time and effort labeling the traffic, which does not meet the needs, and there is no mature method for automated traffic labeling.

[0022] This embodiment provides a semi-supervised IoT abnormal traffic detection method based on a perception-based comparison approach, such as... Figure 1 As shown, the operation process is as follows: Step 1, preprocess and augment the input IoT traffic data to generate pseudo-labels; Step 2, extract feature vectors based on the augmented samples, dynamically calculate and weight the similarity between features using a self-attention mechanism; Step 3, construct a supervised contrast matrix and a class-aware contrast matrix to form intra-class aggregation and inter-class separation in the feature space; Step 4, construct the loss function of the semi-supervised anomaly detection model, iteratively update the model parameters, and use the trained model to detect abnormal traffic.

[0023] This embodiment provides a class-aware contrastive semi-supervised IoT anomaly traffic detection method. Considering the imbalanced distribution of attack categories and the insufficient number of labeled attack samples in real-world environments, a semi-supervised model is designed to train using a small amount of labeled data and a large amount of unlabeled data. An attention mechanism is used to enhance data features, and a class-aware contrastive learning module guides similar samples to approach each other in the feature space and dissimilar samples to move away from each other. This improves the model's ability to distinguish features, especially its ability to focus on features of minor anomaly attack types, thereby improving the model's detection performance across all anomaly attack categories. The model is trained using supervised loss, pseudo-label loss, and contrastive loss. Ultimately, this method effectively improves the accuracy and robustness in imbalanced multi-class anomaly traffic detection.

[0024] The following examples and specific application scenarios further illustrate the technical solution and effects of the present invention. The following examples are explanations of the present invention, but the present invention is not limited to the following examples.

[0025] Specifically, such as Figure 1 As shown, a semi-supervised IoT abnormal traffic detection method based on a perception-based contrastive approach specifically includes the following steps: The first step is to preprocess the input IoT traffic data. Preprocessing specifically includes data mapping and encoding, data normalization, and dataset partitioning.

[0026] The input IoT traffic data is mapped into numerical features to facilitate calculation by the semi-supervised anomaly detection model. To mitigate the influence of feature dimensions and accelerate the training process of the semi-supervised anomaly detection model, the numerical features are linearly normalized and mapped to the [0, 1] interval. Finally, the preprocessed features are divided into datasets.

[0027] In this embodiment, the dataset is partitioned as follows: p% (preferably 5-15) of samples are randomly sampled as labeled data, and (1-p)% of samples are used as unlabeled data. per% (preferably 75-85) of the labeled data and all the unlabeled data are extracted as training data to train the semi-supervised anomaly detection model to learn the features of different traffic categories. The remaining px% (preferably 50) of the labeled data is extracted for parameter tuning and validation of the semi-supervised anomaly detection model, and the remaining labeled data is used as the test set to evaluate the final performance of the semi-supervised anomaly detection model.

[0028] The second step is to perform data augmentation on the preprocessed data, using small noise injection, feature mini-permutation and data smoothing, feature jitter, feature occlusion and class label perturbation to improve the robustness of the semi-supervised anomaly detection model to feature changes.

[0029] like Figure 2 As shown in the figure, in this embodiment, data augmentation includes two parts: weak augmentation and strong augmentation.

[0030] (1) Weak enhancement: Apply slight random changes to the samples to ensure that the category information of the samples remains as unchanged as possible, assisting the semi-supervised anomaly detection model in extracting category-invariant features, and is suitable for pseudo-label generation.

[0031] Specifically: Add small-amplitude Gaussian noise to numerical features (such as traffic duration, packet size, etc.), as follows: x , =x+ò,ò~N(0,σ 2 ) Here, x represents each numerical feature, ò represents small-amplitude noise, and σ represents the standard deviation of the noise. Furthermore, some features are randomly permuted with small amplitudes. Simultaneously, for data with imbalanced class distributions, weakly augmented minority class samples are added to smooth the class distribution of the data.

[0032] (2) Strong Augmentation: Strong augmentation is used for contrastive learning. It generates samples from different perspectives through more significant augmentation operations to improve the robustness of the semi-supervised anomaly detection model to feature changes. The strongly augmented samples are used as positive and negative sample pairs to better achieve intra-class clustering and inter-class separation.

[0033] The enhancement process specifically includes the following steps: First, randomly increase or decrease the value of each feature by a large proportion. Then, randomly set some features to zero to simulate feature loss. Finally, slightly perturb the class labels of minority class samples to increase the diversity of features between classes.

[0034] Step 3: Utilize the enhanced features to perform feature encoding and label prediction, generating pseudo-labels.

[0035] The enhanced features are encoded using an encoder, and the classification head of a semi-supervised anomaly detection model is used to generate preliminary prediction results. The category with the highest confidence is taken as the pseudo-label.

[0036] Specifically: Step (3.1): Pass the weakly augmented unlabeled data samples through the model's classification head (a fully connected layer with a softmax activation function) to obtain the probability distribution p of each sample belonging to each class. i .

[0037] p i =P cls (Aug w (x i )) In the formula, p i It is an unlabeled sample x i The class probability distribution, P cls It is the classification head network of a semi-supervised anomaly detection model, Aug w (x i () represents the sample after weak enhancement.

[0038] Then, for each sample x i The category with the highest probability is selected as the initial pseudo-label for the sample. in, It is sample x i The pseudo-label category.

[0039] Step (3.2): To ensure the generated pseudo-labels have sufficient credibility, a confidence threshold t is set to filter the prediction results. For each sample x i If the maximum class probability exceeds the threshold t, the pseudo-label is retained; otherwise, the pseudo-label of the sample is discarded.

[0040] Step (3.3): Generate a pseudo-label dataset and calculate the pseudo-label loss. Combine the selected high-confidence samples with their pseudo-labels to form a "pseudo-label dataset," and add it to the labeled training data. Furthermore, during model training, as the semi-supervised anomaly detection model learns the data distribution more deeply, the pseudo-label dataset can be dynamically updated.

[0041] Simultaneously, high-confidence pseudo-labeled samples are used to calculate the pseudo-label loss in semi-supervised learning. For each high-confidence pseudo-labeled sample, the cross-entropy loss is calculated using its pseudo-label and the model's prediction result: Where μB represents the batch size of unlabeled data. s (x i) is the enhancement feature of this sample.

[0042] Step 4: Input the enhanced sample into the encoder F(·) to extract the high-dimensional feature representation r. Then, use a projection mapping network to map the high-dimensional feature r to the low-dimensional embedding space to obtain the feature vector z.

[0043] Specifically: The pseudo-label-enhanced samples are input into the encoder F(·) to extract features. F(·) contains three one-dimensional convolutional layers, each followed by batch normalization and ReLU activation to extract local features. Finally, a pooling layer is added after the convolutional layers to reduce the feature dimensionality while preserving important features. The pooling layer features are then flattened and transformed into a fixed-size feature representation through a fully connected layer. Finally, the encoded feature representation R∈M is input. N×d, , where d′ is the feature dimension of the encoder output.

[0044] Step 5: Utilize the self-attention mechanism to dynamically calculate the similarity between features, assign higher weights to relatively important features, and make the output features pay more attention to the key features of the minority class.

[0045] Feature extraction is performed using features enhanced by pseudo-labels, and the similarity between features is calculated using a self-attention mechanism to assign different attention levels to different features.

[0046] The weights of imbalanced class samples are calculated using a self-attention mechanism and then weighted. The specific steps include: Let the characteristic matrix R∈M N ×d , Linear transformations map the data to the query space, key space, and value space.

[0047] This can be represented as: Q = RW Q K = RW K V = RW V Among them W Q W K W V Let Q be a query matrix, K be a key matrix, and V be a value matrix. Let V be a query matrix, K be a key matrix, and V be a value matrix.

[0048] The similarity between the query matrix Q and the key matrix K is calculated to measure the degree of association between each feature. In this embodiment, scaled dot product attention is used to calculate the similarity: Among them, QK T ∈M N×N It is a similarity matrix, representing the degree of attention each feature pays to other features. This is the scaling factor.

[0049] Attention weights are obtained using softmax as the activation function. The similarity matrix is ​​transformed into a probability distribution, and the softmax function is used to normalize each row of the similarity matrix. Where A∈M N ×N is the attention weight matrix, a ij This represents the weight of the i-th feature to the j-th feature, and the sum of the weights in each row is 1.

[0050] Step (5.3): Apply the attention weight matrix A to the value matrix V to obtain the weighted feature representation Z: Z = AW.

[0051] in, It is the weighted comprehensive feature matrix of the self-attention output.

[0052] Step 6: Measure the similarity between sample pairs based on cosine similarity, and construct a supervised contrast matrix and a class-aware contrast matrix. Use class-aware contrast loss to constrain feature learning, so that samples of the same class are clustered together and samples of different classes are separated, further improving the ability of the semi-supervised anomaly detection model to distinguish minority class samples.

[0053] By constructing supervised contrast and class contrast matrix learning modules, intra-class aggregation and inter-class separation are formed in the feature space, which helps the semi-supervised anomaly detection model to distinguish different types of abnormal traffic in the feature space.

[0054] Specifically, it includes: Step (6.1): Input the attention-enhanced feature Z into the projection head P. proj In (·), it is mapped to the contrastive learning embedding space to obtain the low-dimensional embedding vector z: z=P proj (Z).

[0055] The mapping network is a multilayer perceptron structure. Specifically, the first layer is a fully connected layer that generates an intermediate mapping h, which can be expressed as: h = ReLU(ZW1 + b1).

[0056] Here, W1 and b1 are both trainable weights and bias parameters.

[0057] Then the intermediate feature h is mapped to the final embedding space, represented as: z = hW2 + b2.

[0058] Where W2 and b2 are trainable parameters, and z is the output feature of the projector.

[0059] Finally, z is subjected to L2 normalization.

[0060] Step (6.2): ​​Construct sample pairs using the features z generated by the mapping network.

[0061] First, positive samples are constructed using class labels and pseudo-labels to achieve within-class aggregation. For samples z of the same class... i and z j If their labels or pseudo-labels are the same, then they are selected as positive sample pairs.

[0062] Furthermore, negative sample pairs are constructed to achieve inter-class separation: for each sample z with or without a label... i Select samples z with different labels or pseudo-labels. k As a negative sample.

[0063] Step (6.3): Construct a class-aware contrast matrix.

[0064] The class-aware contrast matrix includes a supervised contrast matrix and a class-aware contrast matrix. It ensures intra-class similarity and inter-class separation. The supervised contrast matrix W... scon Based on the true categories of the labeled data, samples of the same category are assigned a value of 1, and samples of different categories are assigned a value of 0. This can be represented as:

[0065] In addition, the class-aware contrast matrix W clacon This is generated based on the true labels of the labeled data and the pseudo labels of the unlabeled data. If two samples belong to the same category or have the same pseudo label, a value of 1 is assigned; otherwise, a value of 0 is assigned. This is represented as:

[0066] Step 7: Construct the model loss function, which includes the supervision loss L. x False label loss L u Contrast loss with class perception L c The model parameters are iteratively updated using backpropagation and gradient descent optimization algorithms. After training, the semi-supervised anomaly detection model infers from newly input IoT device traffic data, generating category labels and confidence scores.

[0067] Specifically, it includes: Step (7.1): In the class-aware contrastive learning module, the total loss function of the semi-supervised anomaly detection model consists of three parts: supervised loss, pseudo-label loss, and class-aware contrastive loss, which together optimize the model's performance on imbalanced multi-class data. Specifically, the loss function L is constructed... x For classification tasks involving labeled data, the classification error is calculated using the true labels, and the cross-entropy loss is used to measure the difference between the model's predicted distribution and the true distribution. Where B is the batch size of the labeled samples, C is the number of classes, and yi,c is the true class label of sample i, and is the class probability predicted by the model.

[0068] For unlabeled data, pseudo-labels are first generated, and cross-entropy loss is calculated by selecting samples with high confidence levels.

[0069] Where μB represents the batch size of the unlabeled data, 1(max(p) i The function ≥ t is an indicator function that indicates whether the confidence level of the pseudo-label exceeds the threshold t. To predict probabilities, This is a pseudo-tag.

[0070] To ensure intra-class clustering and inter-class separation in the feature space, and to help semi-supervised anomaly detection models distinguish features from different categories, this embodiment calculates L using a constructed class-aware contrast matrix and similarity matrix. c =H(S,W) target ).

[0071] In the formula, H represents the cross-entropy loss, which is used to measure the similarity matrix S and the class-aware contrast matrix W. target The difference between them. W target It is a contrast matrix that combines supervised contrast matrix and class-aware contrast matrix: L = L x +λ u L u +λ c L c Wherein, λ u and λ c It is a hyperparameter that controls the weights of pseudo-label loss and class-aware contrast loss in the total loss.

[0072] Step (7.2): Finally, the model parameters are iteratively updated using backpropagation and gradient descent optimization algorithms to minimize the total loss function L. After training, the semi-supervised anomaly detection model infers the traffic data of newly input IoT devices and generates the category label and confidence score for each sample.

[0073] This embodiment provides a semi-supervised IoT abnormal traffic detection method based on a perceptual comparison model, which has the following beneficial effects: 1. By employing semi-supervised learning (SSL) to jointly train a large number of unlabeled samples and a small number of labeled samples, the overall distribution of the data is better learned, enhancing the generalization ability of the semi-supervised anomaly detection model to unknown anomalies. Furthermore, contrastive learning, by constructing positive and negative sample pairs, enables the semi-supervised anomaly detection model to achieve intra-class clustering and inter-class separation in the feature space, thereby enhancing the discriminative power of the features.

[0074] 2. By utilizing a dynamic self-attention mechanism, attention weights are adaptively allocated based on the features of the input data, enabling the semi-supervised anomaly detection model to focus on key features related to anomaly detection and suppress noise and redundant information.

[0075] 3. IoT devices often face problems such as imbalanced abnormal traffic data and insufficient labeled samples during operation. The ability of semi-supervised learning to utilize unlabeled data, the feature discrimination ability of contrastive learning, and the key feature focusing ability of self-attention mechanism can effectively solve the above problems, thereby improving the accuracy of abnormal traffic detection in IoT.

[0076] Example 2: This embodiment provides a class-aware comparative semi-supervised IoT abnormal traffic detection method, which is used to implement and verify the class-aware comparative semi-supervised IoT abnormal traffic detection method in Embodiment 1 by incorporating specific scenarios and data.

[0077] Currently, threat detection based on network traffic has been extensively researched, with typical techniques including misuse-based techniques, machine learning techniques, and deep learning techniques. Misuse-based techniques, also known as signature-based network traffic anomaly detection, match network traffic with signatures in a signature database. Once a match is found, an alert is issued to indicate abnormal behavior. Misuse-based network traffic anomaly detection is very accurate, but it heavily relies on manual analysis of network attack traffic and the creation of signatures, resulting in high maintenance costs and the inability to identify attack activities not in the signature database.

[0078] To address the problems associated with misused technologies, machine learning has been used to develop network traffic anomaly detection systems. Machine learning can automatically analyze network attack data to identify patterns and use these patterns to predict unknown data. Therefore, security personnel no longer need to write signatures for attacks; instead, they can extract useful features from network attack traffic and input them into the machine learning model, allowing the model to "learn" to detect network threats. However, the performance of machine learning depends heavily on the base features and traffic characteristics. Security personnel often need to analyze large amounts of labeled data to extract effective features, and feature transfer can be challenging. To completely eliminate reliance on manual traffic analysis, deep learning has been used to develop network traffic anomaly detection systems. Deep learning can learn and identify patterns from large amounts of raw data and then use these patterns to predict unknown data. Compared to machine learning, deep learning has two advantages: first, it can learn directly from raw data without requiring manual analysis to provide usable features; second, its performance can be improved by using more data, while many traditional machine learning algorithms do not continuously increase the amount of knowledge learned with increased data volume. Deep learning technology has completely solved the difficulties of manual analysis of network traffic and has a high detection capability. However, the prerequisite for using deep learning technology is having a large amount of labeled data, and the amount of traffic from different attacks should be as close to the same order of magnitude as possible. Obtaining such network traffic samples in the real environment presents two major challenges: first, even if a large amount of traffic is collected, the amount of benign traffic will far exceed the amount of attack traffic, and the amount of traffic from different attacks will vary greatly; second, labeling network traffic data is very tedious and time-consuming, resulting in high manpower and time costs.

[0079] This embodiment provides a class-aware contrastive semi-supervised method for detecting abnormal traffic in the Internet of Things (IoT). A semi-supervised anomaly detection model is constructed for detecting abnormal attacks using unknown network traffic data. Addressing the challenges of insufficient abnormal traffic data and severely uneven distribution of attack types in the IoT environment, this method combines the advantages of class-aware contrastive learning and self-attention mechanisms. It uses a dataset containing both normal traffic and multi-class abnormal traffic from the IoT as the training foundation, employing self-attention to enhance feature extraction from minority class samples. This allows the semi-supervised anomaly detection model to better focus on minority and abnormal class features. Furthermore, semi-supervised learning is used to weakly and strongly enhance unlabeled samples, generating pseudo-labels. Class-aware contrastive learning is then used to achieve intra-class feature aggregation and inter-class separation. A reweighting mechanism assigns higher weights to high-confidence samples, especially rare abnormal samples, enhancing their feature representation capabilities. The model is trained using supervised loss, pseudo-label loss, and contrastive loss, effectively improving the accuracy and robustness in detecting imbalanced multi-class abnormal traffic.

[0080] In practical applications, the following steps are included: Step 1: Perform data preprocessing on the input data, which includes data mapping and encoding, data normalization, and dataset partitioning.

[0081] The table below shows the statistics for the NSL-KDD dataset, illustrating the distribution of various attack categories. DoS attacks represent denial-of-service attacks, Probe attacks represent probe attacks, U2R attacks represent user-to-root attacks, and R2L attacks represent remote-to-local attacks. DoS attack traffic samples constitute the majority, while U2R and R2L samples are less common. Tag categories Dataset DOS 45926 Nornal 67342 Probe 11655 R2L 994 U2R 51

[0082] This semi-supervised anomaly detection model can perform anomaly detection on a dataset containing both normal traffic and attack-type traffic. It converts character-type features in the dataset, such as protocol type features (e.g., TCP, UDP, ICMP), into one-hot encoding.

[0083] To eliminate the influence of feature dimensions and accelerate the model training process, numerical features are linearly normalized and mapped to the [0,1] interval, as follows: Where X is the original feature, X min and X max X and X' are the minimum and maximum values ​​of the feature, respectively, and X′ is the normalized feature.

[0084] Finally, the preprocessed features were divided into two datasets: 10% of the samples were randomly sampled as labeled data, and 90% were used as unlabeled data. Of the labeled data, 80% and all of the unlabeled data were used as training data to train the model to learn features for different traffic categories. The remaining half of the labeled data was used for model parameter tuning and validation, and the remaining labeled data was used as the test set to evaluate the final performance of the semi-supervised anomaly detection model.

[0085] The second step is to perform data augmentation on the preprocessed data, using small noise injection, feature mini-permutation and data smoothing, feature jitter, feature occlusion and class label perturbation to improve the model's robustness to feature changes.

[0086] Data augmentation includes strong augmentation and weak augmentation. Weak augmentation involves applying slight random changes to the samples to ensure that the class information of the samples remains as unchanged as possible, which is suitable for pseudo-label generation. Specifically, it includes adding small-amplitude Gaussian noise to numerical features (such as traffic duration, packet size, etc.).

[0087] Strong augmentation involves generating samples from different perspectives through more significant augmentation operations. These augmented samples are then used as positive-negative pairs to better achieve intra-class clustering and inter-class separation. A large percentage of each feature value is randomly increased or decreased. Additionally, some features are randomly set to zero to simulate feature loss. Finally, the class labels of minority class samples are slightly perturbed to increase the diversity of features between classes.

[0088] Step 3: Pass the weakly augmented unlabeled data samples through the model's classification head (a fully connected layer plus a Softmax activation function) to obtain the probability distribution of each sample belonging to each category.

[0089] Step 4: Input the enhanced sample into the encoder F(·) to extract the high-dimensional feature representation r. Then, use a projection mapping network to map the high-dimensional feature r to the low-dimensional embedding space to obtain the feature vector z.

[0090] Step 5: Utilize the self-attention mechanism to dynamically calculate the similarity between features, assign higher weights to relatively important features, and make the output features pay more attention to the key features of the minority class.

[0091] Step 6: Measure the similarity between sample pairs based on cosine similarity, and construct supervised contrast matrix and class-aware contrast matrix. Use class-aware contrast loss to constrain feature learning, so that samples of the same class are clustered together and samples of different classes are separated, further improving the model's ability to distinguish minority class samples.

[0092] Step 7: Construct the model loss function, including the supervision loss L. x False label loss L u Contrast loss with class perception L c The model parameters are iteratively updated using backpropagation and gradient descent optimization algorithms. After training, the model infers from newly input IoT device traffic data, generating class labels and confidence scores.

[0093] Among them, constructing loss L x For classification tasks involving labeled data, the classification error is calculated using the true labels, and the cross-entropy loss is used to measure the difference between the model's predicted distribution and the true distribution. In the formula, B represents the batch size of the labeled samples, C represents the number of categories, and y i,c The true class label of sample i is represented by the class probability predicted by the model.

[0094] In addition, for unlabeled data, pseudo-labels are generated first, and cross-entropy loss is calculated by selecting samples with high confidence.

[0095] In the formula, μB represents the batch size of unlabeled data, 1(max(p i The function ≥ t is an indicator function that indicates whether the confidence level of the pseudo-label exceeds the threshold t. To predict probabilities, This is a pseudo-tag.

[0096] Ensuring intra-class clustering and inter-class separation in the feature space helps the model distinguish features from different categories. This is achieved by calculating L using the constructed class-aware contrast matrix and similarity matrix. c =H(S,W) target ).

[0097] In the formula, H represents the cross-entropy loss, which is used to measure the similarity matrix S and the class-aware contrast matrix W. target The gap between them.

[0098] W target It is a contrast matrix that combines supervised contrast matrix and class-aware contrast matrix: L = L x +λ u L u +λ c L c Wherein, λ u and λ c It is a hyperparameter that controls the weights of pseudo-label loss and class-aware contrast loss in the total loss.

[0099] Finally, the model parameters are iteratively updated using backpropagation and gradient descent optimization algorithms to minimize the total loss function L.

[0100] The specific training process is as follows: Step (1): First, for the forward propagation process, labeled and unlabeled data are input, and features are extracted by the encoder F(·). After weighted processing by the self-attention module, a contrastive learning embedding representation is generated by the projection head. The supervision loss, pseudo-label loss, and class-aware contrastive loss are calculated.

[0101] Step (2): Subsequently, the backpropagation process of the model is performed. The gradient of the model parameters is calculated using the total loss L. During the backpropagation phase, the trainable parameters of all modules, including the encoder, self-attention module, and projector, are updated.

[0102] Step (3): Gradient descent update: Use the gradient descent optimization algorithm to update the model parameters so that the total loss gradually decreases.

[0103] Step (4): Update the pseudo-labels of the unlabeled data at multiple stages of training and generate high-confidence pseudo-label samples to be added to the training.

[0104] Step (5): Repeat forward propagation, back propagation and gradient descent updates until the model loss on the validation set converges.

[0105] After training, the model inference process infers from newly input IoT device traffic data, generating a category label and confidence score for each sample.

[0106] This embodiment presents an attention-enhanced, class-aware, contrastive, semi-supervised anomaly detection method. Considering the limitations of real-world IoT environments, such as insufficient anomaly traffic data, severe imbalances in attack category distribution, and a lack of labeled attack samples, a semi-supervised model is designed to train using a small amount of labeled data and a large amount of unlabeled data. An attention mechanism is used for data feature enhancement, and a class-aware, contrastive learning module guides similar samples to approach each other in the feature space and dissimilar samples to move away from each other. The model is trained using supervised loss, pseudo-label loss, and contrastive loss. Ultimately, this method effectively improves the accuracy and robustness in imbalanced multi-class anomaly traffic detection.

[0107] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Other variations and modifications are possible without departing from the technical solutions described in the claims.

Claims

1. A semi-supervised method for detecting abnormal traffic in the Internet of Things (IoT) based on a perceptual comparison model, characterized in that: include: S1: Preprocess and augment the input IoT traffic data to generate pseudo-tags; S2: Extract feature vectors from the enhanced samples, and dynamically calculate and weight the similarity between features using a self-attention mechanism; S3: Construct supervised contrast matrix and class-aware contrast matrix to form intra-class aggregation and inter-class separation in feature space; S4: Construct the loss function of the semi-supervised anomaly detection model, iteratively update the model parameters, and use the trained model to detect abnormal traffic.

2. The method for detecting abnormal traffic in the Internet of Things using a perceptual comparison approach according to claim 1, characterized in that, The method of dynamically calculating and weighting the similarity between features using a self-attention mechanism includes: mapping the feature matrix to a query space, key space, and value space through a linear transformation; calculating the similarity between the query matrix and the key matrix to measure the degree of association between each feature; obtaining attention weights using an activation function; converting the similarity matrix into a probability distribution; normalizing each row of the similarity matrix using an activation function; and applying the attention matrix to the value matrix to obtain a weighted feature representation.

3. The semi-supervised IoT abnormal traffic detection method based on a perception-based comparison according to claim 1, characterized in that, S3 includes: mapping the attention-enhanced features to the contrastive learning embedding space to obtain low-order embedding vectors; constructing sample pairs using features generated by the mapping network: constructing positive samples using class labels and pseudo-labels; for samples of the same class, if their labels or pseudo-labels are the same, selecting them as positive sample pairs; for each sample with a label or pseudo-label, selecting samples with different labels or pseudo-labels as negative samples; constructing a supervised contrast matrix based on the true class of the labeled data; and generating a class-aware contrast matrix based on the true label of the labeled data and the pseudo-label of the unlabeled data.

4. The semi-supervised IoT abnormal traffic detection method based on a perceptual comparison according to claim 1, characterized in that, S1 includes: preprocessing the input IoT traffic data and dividing the dataset; using semi-supervised learning to perform strong and weak enhancement on unlabeled samples; encoding the enhanced features; using a classification head to initially generate prediction results; and taking the category with the highest confidence as the pseudo-label.

5. The semi-supervised IoT abnormal traffic detection method based on a perception-based comparison according to claim 4, characterized in that, Generating pseudo-labels involves: passing the weakly augmented unlabeled data samples through the classification head of a semi-supervised anomaly detection model to obtain the probability distribution of each sample belonging to each category; for each sample, selecting the category with the highest probability as the initial pseudo-label; for each sample, if its maximum category probability exceeds a preset confidence threshold, the pseudo-label is retained, otherwise it is discarded; the selected high-confidence samples are combined with their pseudo-labels to obtain a pseudo-label dataset, and the pseudo-label dataset is added to the labeled training data.

6. A semi-supervised IoT abnormal traffic detection method based on a perceptual comparison according to claim 1, 2, 3, 4, or 5, characterized in that, The pseudo-labels with high confidence are used to calculate the pseudo-label loss in semi-supervised learning: For each high-confidence pseudo-label sample, the cross-entropy loss is calculated using the pseudo-labels and the prediction results of the semi-supervised anomaly detection model.

7. A semi-supervised IoT abnormal traffic detection method based on a perceptual comparison according to claim 1, 2, 3, 4, or 5, characterized in that, S2 includes extracting high-dimensional feature representations from the enhanced samples and mapping the high-dimensional features to a low-dimensional embedding space: the pseudo-label enhanced samples are input into an encoder to extract features, the encoder including three one-dimensional convolutional layers, each followed by batch normalization and activation functions; a pooling layer is added after the convolutional layers, the pooling layer features are flattened, and transformed into a fixed-size feature representation through a fully connected layer; the encoded feature representation is input to obtain a feature vector.

8. A semi-supervised IoT abnormal traffic detection method based on a perceptual comparison according to claim 4 or 5, characterized in that, The preprocessing of the input data and the partitioning of the dataset include: mapping the input IoT traffic data into numerical features; mapping the numerical features to the [0, 1] interval using linear normalization; randomly sampling p% of samples as labeled data, (1-p)% of samples as unlabeled data, extracting per% of the labeled data and all the unlabeled data as training data, extracting the remaining px% from the labeled data for parameter tuning and validation of the semi-supervised anomaly detection model, and using the remaining labeled data as a test set to evaluate the final performance of the semi-supervised anomaly detection model.

9. A semi-supervised IoT abnormal traffic detection method based on a perceptual comparison according to claim 4 or 5, characterized in that, The data augmentation includes: weak augmentation, which applies random changes to the samples; strong augmentation, which randomly sets some features to zero to simulate feature loss, slightly perturbs the class labels of minority class samples, and generates samples from different perspectives; the strongly augmented samples are used to construct positive and negative sample pairs.

10. A semi-supervised IoT abnormal traffic detection method based on a perceptual comparison according to claim 1, 2, 3, 4, or 5, characterized in that, In step S4: the loss function includes supervised loss, pseudo-label loss, and class-aware contrastive loss; the parameters of the semi-supervised anomaly detection model are iteratively updated using backpropagation and gradient descent optimization algorithms to minimize the loss function; after the model training is completed, the semi-supervised anomaly detection model is used to infer new input IoT traffic data to generate category labels and confidence scores.

Images