Malicious traffic hypothesis testing method and device based on attention mechanism and data enhancement

Abstract

The present invention discloses a method and device for detecting malicious traffic hypothesis based on an attention mechanism and data enhancement, wherein the method comprises: using existing publicly available normal traffic and malicious traffic data sets on the Internet to train an encoder, then enhancing the training data through starGAN, and then continuing to fine-tune the Transformer training. After the training is completed, the feature information in the traffic is extracted and then the KS test is performed on the transformer. In the actual traffic detection process, the traffic data is first pre-processed and then encoded through the encoder to obtain its features. Then, the KS hypothesis test is used on the traffic features to give the confidence that the traffic is malicious traffic. Through the confidence hypothesis test, it is possible to determine whether the traffic is malicious traffic within a certain threshold range. The present invention can effectively prevent malicious adversarial attacks on traffic and some attacks by currently unknown methods, and improve the accuracy and robustness of detection under the premise of low false alarm rate.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a malicious traffic hypothesis testing method and device based on an attention mechanism and data enhancement. Background Art

[0002] With the rapid development and ubiquity of the internet, the scale and complexity of network traffic continues to increase. Simultaneously, the methods and techniques used for network attacks are becoming increasingly sophisticated and diverse. The widespread use of encrypted traffic, in particular, enables malware and attackers to conduct malicious activities undetected. Traditional network traffic detection methods primarily rely on feature extraction and analysis of plaintext data, but these methods are significantly less effective when dealing with encrypted traffic. Therefore, effectively detecting encrypted malicious traffic without compromising data privacy has become a pressing issue.

[0003] Malicious network traffic attacks pose a serious threat to network security, often carried out through malware, viruses, worms, Trojan horses, phishing attacks, and distributed denial of service (DDoS). These attacks not only paralyze network systems but can also lead to the leakage of large amounts of sensitive information, posing significant security risks to personal privacy and corporate data. Malware is spread through various means, including email attachments, malicious websites, and infected downloads. Once a user runs the malware, the attacker can remotely control the infected computer, stealing sensitive information or performing other malicious operations. Computer viruses and worms are self-replicating malicious programs that spread automatically over networks, damaging system files and causing large-scale network outages. Trojan horses are malicious programs disguised as legitimate software. Once installed on a user's computer, the attacker can remotely control the infected system, stealing data, and tampering with information. Phishing attacks are online fraud schemes that deceive users into providing sensitive information by impersonating legitimate organizations or individuals. These attacks typically involve forged emails or websites that trick users into entering personal information, thereby enabling identity theft and financial fraud. A DDoS attack uses a large number of botnets to send a large number of requests to the target server, causing the server to be unable to process legitimate requests normally and eventually paralyzing the target server.

[0004] In the process of dealing with these complex attack methods, methods based on machine learning and deep learning have shown great potential and advantages in network traffic detection. In practical applications, a single technology is difficult to cope with complex encrypted network traffic detection tasks. In order to improve the accuracy and robustness of detection, researchers have tried to combine methods based on machine learning and deep learning with other technologies to build a comprehensive detection system. However, these methods still face many challenges when dealing with new and ever-changing malicious traffic: feature extraction difficulties. The features of encrypted traffic data are difficult to extract, and traditional feature extraction methods are difficult to adapt to complex encrypted traffic patterns; label noise problems. Label noise is prevalent in network traffic datasets. Incorrect labels will seriously affect the training effect and detection performance of the model; distribution changes. The distribution characteristics of new malicious traffic may be significantly different from known malicious traffic, resulting in insufficient generalization ability of the detection model, and thus poor traffic detection results. Summary of the Invention

[0005] This paper proposes a malicious traffic hypothesis testing method based on an attention mechanism and data augmentation to address the existing problems of feature extraction difficulties, label noise, and insufficient generalization ability of detection models caused by distribution changes when processing encrypted traffic.

[0006] The technical solution adopted in the present invention is as follows:

[0007] The first aspect provides a malicious traffic hypothesis testing method based on attention mechanism and data enhancement, including:

[0008] Obtain public normal traffic and malicious traffic datasets to construct original training data;

[0009] The DBSCAN clustering algorithm is used to estimate the data distribution of the original training data. The data are relabeled according to the clustering results of the DBSCAN clustering algorithm. The relabeled samples are trained and predicted using the random forest classifier to obtain the corrected training data.

[0010] Use StarGAN to perform data enhancement on the corrected training data to obtain data-enhanced training data;

[0011] The Transformer model is fine-tuned using the data-augmented training data to obtain a traffic detection model.

[0012] The traffic detection model is used to extract characteristic information from the traffic to be detected, and the KS test is performed based on the extracted characteristic information.

[0013] In one embodiment, the DBSCAN clustering algorithm is used to estimate the data distribution of the original training data, the data are relabeled according to the clustering results of the DBSCAN clustering algorithm, and the relabeled samples are trained and predicted using a random forest classifier to obtain corrected training data, including:

[0014] Use the DBSCAN clustering algorithm to cluster the original training data and estimate the distribution of the data;

[0015] Based on the DBSCAN clustering results, samples in high-density areas are re-labeled as normal samples, and samples in low-density areas are re-labeled as malicious samples. The areas above the density threshold are high-density areas, and the areas below the density threshold are low-density areas.

[0016] The relabeled samples are trained and predicted using a random forest classifier, and the labels of the decision boundary samples are corrected to obtain the corrected training data. The decision boundary samples are the samples at the boundary between the high-density area and the low-density area when clustering is performed using the DBSCAN clustering algorithm.

[0017] In one embodiment, clustering the original training data using the DBSCAN clustering algorithm to estimate the distribution of the data includes:

[0018] Set the minimum number of samples minPts and the distance threshold ∈, and calculate the regional density of the sample data point p according to the formula D(p) = {q∈D|dist(p,q)≤∈}, where p and q are two sample data points, D represents the data set consisting of the original training data, D(p) represents the cluster with p as the core, that is, the regional density of the sample data point p, and dist(p,q) represents the distance between the two sample data points.

[0019] In one embodiment, StarGAN is used to perform data augmentation on the corrected training data to obtain data augmented training data, including:

[0020] Define target areas for data augmentation based on the distribution of malicious and normal data;

[0021] Use StarGAN to generate new data so that its distribution conforms to the distribution characteristics of the target area;

[0022] The generated new data is combined with the corrected training data to obtain data-augmented training data.

[0023] In one embodiment, the Transformer model is fine-tuned using the data-augmented training data to obtain a traffic detection model, including:

[0024] Convert the augmented training data into embedding vectors;

[0025] The encoder of the Transformer model is used to process the embedding vector to capture the complex patterns in the sequence data and generate the encoded feature vector F;

[0026] Use the decoder of the Transformer model to restore the encoded feature vector and generate a restored vector R = Decoder(F), where Decoder represents the decoder;

[0027] A combination of self-supervised learning and supervised learning is used to fine-tune and train the model to obtain a traffic detection model.

[0028] In one embodiment, when fine-tuning and training the model using a combination of self-supervised learning and supervised learning, the calculated losses include the reconstruction loss of the encoder and decoder, and the spherical loss during encoder encoding, wherein the spherical loss during encoder encoding is used to confine normal traffic samples within the sphere and exclude malicious traffic samples outside the sphere.

[0029] In one embodiment, a traffic detection model is used to extract traffic feature information from the traffic to be detected, and a KS test is performed based on the extracted feature information, including:

[0030] Pre-process the traffic to be detected.

[0031] The pre-processed traffic to be detected is encoded through the traffic detection model to extract the corresponding traffic feature information;

[0032] The KS hypothesis test is used on the traffic characteristics to give the confidence that the traffic is malicious traffic, and the hypothesis test is performed to calculate the test value. If the test value is greater than the threshold, the traffic to be detected is determined to be malicious traffic. Otherwise, the traffic to be detected is determined to be normal traffic.

[0033] Based on the same inventive concept, the second aspect of the present invention provides a malicious traffic hypothesis testing device based on an attention mechanism and data enhancement, comprising:

[0034] The data acquisition module is used to obtain public normal traffic and malicious traffic data sets to construct original training data;

[0035] The label correction module is used to estimate the data distribution of the original training data using the DBSCAN clustering algorithm, re-label the samples according to the clustering results of the DBSCAN clustering algorithm, and use the random forest classifier to train and predict the re-labeled samples to obtain the corrected training data;

[0036] The data enhancement module is used to perform data enhancement on the corrected training data using StarGAN to obtain data-enhanced training data;

[0037] The fine-tuning training module is used to fine-tune the Transformer model using the data-augmented training data to obtain a traffic detection model;

[0038] The flow detection module is used to extract characteristic information from the flow to be detected by using the flow detection model, and perform KS test based on the extracted characteristic information.

[0039] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the malicious traffic hypothesis testing method based on attention mechanism and data enhancement described in the first aspect.

[0040] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the program, the malicious traffic hypothesis testing method based on attention mechanism and data enhancement described in the first aspect is implemented.

[0041] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0042] The present invention provides a malicious traffic hypothesis testing method based on an attention mechanism and data augmentation. First, publicly available normal and malicious traffic datasets are obtained to construct original training data. Label noise correction is performed using DBSCAN clustering, and data augmentation is performed using StarGAN. The augmented training data is then used to fine-tune the Transformer model to obtain a traffic detection model. Finally, the traffic detection model is used to determine whether the traffic is malicious using a KS test. The feature extractor (the encoder of the Transformer model) is reused, avoiding the training cost of reintroducing a classifier in traditional recognition methods. This method improves detection accuracy and robustness while maintaining a low false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1A flowchart of the encoder training of the Transformer model provided by an embodiment of the present invention;

[0045] Figure 2 This is a flow chart of performing actual malicious traffic detection using a trained detection model in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In the fight against complex attack vectors currently available, machine learning and deep learning-based methods have demonstrated significant potential and advantages in network traffic detection. The Transformer model, in particular, is an ideal choice for processing network traffic data due to its powerful sequence modeling capabilities and parallel computing capabilities. The Transformer architecture, through its self-attention mechanism, can capture long-range dependencies and complex patterns in sequential data, making it well-suited for processing network traffic data. Its encoder and decoder structures offer significant advantages in feature extraction and data restoration. The encoder converts input data into an embedding vector and processes it through multiple layers of self-attention and feedforward neural networks to capture complex sequential patterns. The decoder attempts to restore the feature vectors generated by the encoder to the original data, thereby improving feature extraction accuracy through self-supervised learning. The self-attention mechanism calculates the association weights between each position in the input sequence and other positions, capturing global dependencies and improving the quality of feature representation.

[0047] DBSCAN is a density-based clustering algorithm that effectively discovers clusters of arbitrary shapes and identifies noise points. DBSCAN determines whether data points belong to the same cluster by setting a distance threshold and a minimum number of samples. This algorithm excels in cluster analysis and label noise correction for network traffic data. Density estimation determines whether a data point is a core point, boundary point, or noise point by calculating the neighborhood density of the data point. Cluster formation groups adjacent core and boundary points into clusters based on their density connectivity. Label correction relabels normal and malicious samples based on the clustering results to improve data quality.

[0048] Random forest is an ensemble learning method that improves the accuracy and robustness of classification and regression by building multiple decision trees and combining their outputs. Random forests have significant advantages in handling high-dimensional data and dealing with label noise, and are widely used in network traffic classification tasks. Decision tree construction involves randomly selecting samples and features to construct multiple decision trees, reducing model overfitting. A voting mechanism combines the classification results of all decision trees and determines the final classification result, improving model stability and accuracy. Noise processing, through the integration of multiple trees, reduces the sensitivity of individual decision trees to noisy data and improves the robustness of the overall model.

[0049] To improve the accuracy and robustness of detection, the present invention combines the above technologies to construct a comprehensive detection system. The Transformer model can be used for feature extraction to generate high-quality feature vectors by capturing complex traffic patterns; the DBSCAN clustering algorithm can identify and correct label noise to improve data quality; and the random forest classifier can further enhance the accuracy and robustness of detection.

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0051] Example 1

[0052] The present invention discloses a malicious traffic hypothesis testing method based on attention mechanism and data enhancement, comprising:

[0053] S1: Obtain public normal traffic and malicious traffic datasets to construct original training data;

[0054] S2: Use the DBSCAN clustering algorithm to estimate the data distribution of the original training data, relabel it according to the clustering results of the DBSCAN clustering algorithm, and use the random forest classifier to train and predict the relabeled samples to obtain the corrected training data;

[0055] S3: Use StarGAN to perform data augmentation on the corrected training data to obtain data augmented training data;

[0056] S4: Fine-tune the Transformer model using the data-augmented training data to obtain a traffic detection model;

[0057] S5: Using the traffic detection model to extract characteristic information from the traffic to be detected, and performing a KS test based on the extracted characteristic information.

[0058] Specifically, S1 constructs original training data by collecting normal traffic and malicious traffic datasets publicly available on the Internet.

[0059] In S2, the DBSCAN clustering algorithm calculates the regional density of each data point by setting the distance threshold ∈ and the minimum number of samples minPts, and then distinguishes high-density and low-density areas through the density threshold, relabels the samples in each area, and further uses the random forest classifier to correct the labels of the decision boundary samples.

[0060] S3 uses StarGAN to generate new data to enhance the original training data. StarGAN's generator network generates new malicious traffic data and normal traffic data according to different target areas, and combines these new data with the original training data to obtain data-enhanced training data.

[0061] S4 uses the data-augmented training data obtained in S3 to fine-tune the model.

[0062] S5 is a specific application of the detection model, which uses the encoder of the flow detection model to extract the characteristic information in the flow to be detected, and then performs the KS test.

[0063] In one embodiment, the DBSCAN clustering method is used to correct label noise on the original training data to obtain corrected training data, including:

[0064] Use the DBSCAN clustering algorithm to cluster the original training data and estimate the distribution of the data;

[0065] Based on the DBSCAN clustering results, samples in high-density areas are re-labeled as normal samples, and samples in low-density areas are re-labeled as malicious samples. The areas above the density threshold are high-density areas, and the areas below the density threshold are low-density areas.

[0066] The relabeled samples are trained and predicted using a random forest classifier, and the labels of the decision boundary samples are corrected to obtain the corrected training data. The decision boundary samples are the samples at the boundary between the high-density area and the low-density area when clustering is performed using the DBSCAN clustering algorithm.

[0067] In the specific implementation process, the DBSCAN clustering algorithm identifies high-density areas and low-density areas by setting the distance threshold ∈ and the minimum number of samples minPts.

[0068] When re-labeling, the density threshold ρ is first set, and the area above the density threshold is regarded as the high-density area, and the area below the density threshold is regarded as the low-density area.

[0069] The specific implementation process of using the random forest classifier to train and predict the relabeled samples and correct the labels of the decision boundary samples includes:

[0070] Construct random forest classifier RF;

[0071] The classifier RF is trained using the encoded feature vector F obtained from the Transformer model encoder.

[0072] Use the trained classifier RF to perform label correction on the remaining samples (decision boundary samples) to obtain the corrected label set

[0073] In one embodiment, clustering the original training data using the DBSCAN clustering algorithm to estimate the distribution of the data includes:

[0074] Set the minimum number of samples minPts and the distance threshold ∈, and calculate the regional density of the sample data point p according to the formula D(p) = {q∈D|dist(p,q)≤∈}, where p and q are two sample data points, D represents the data set consisting of the original training data, D(p) represents the cluster with p as the core, that is, the regional density of the sample data point p, and dist(p,q) represents the distance between the two sample data points.

[0075] Among them, dist(p,q) is calculated as follows:

[0076]

[0077] where p(x i ) represents the probability distribution of the sample data point p on the i-th feature. i ) indicates that the sample data point q is in the

[0078] Probability distribution over i features, n is the feature length, i∈[1,n].

[0079] In one embodiment, StarGAN is used to perform data augmentation on the corrected training data to obtain data augmented training data, including:

[0080] Define target areas for data augmentation based on the distribution of malicious and normal data;

[0081] Use StarGAN to generate new data so that its distribution conforms to the distribution characteristics of the target area;

[0082] The generated new data is combined with the corrected training data to obtain data-augmented training data.

[0083] Specifically, the generated new data (new samples) meet the following conditions:

[0084] New samples are generated in the normal data boundary area, and the generation area is set to Ω NB , the corresponding probability density function is p N (x), the new sample should meet the following requirements:

[0085] x∈Ω NB And ρ≤p N (x)

[0086] Where ρ is the density threshold and x is the new sample.

[0087] New samples are generated in the malicious data boundary area, and the generation area is set to Ω MB , the corresponding probability density function is p M (x), the new sample should meet the following requirements:

[0088] x∈Ω MB and ∈≤p M (x)≤ρ

[0089] Where ∈ is the distance threshold mentioned above.

[0090] New samples are generated in an area far away from all data, and the generated area is set to Ω MO , the corresponding probability density function is p O (x), the new sample should meet the following requirements:

[0091] x∈Ω MO And p O (x)<∈

[0092] Among them, among the newly generated samples, the samples generated in the normal data boundary area are marked as normal samples, and the other two types of samples (generated in the malicious data boundary area and generated in the area far away from all data) are marked as malicious samples. The newly generated samples and the corrected training data constitute the enhanced training data for subsequent model fine-tuning training.

[0093] In one embodiment, the Transformer model is fine-tuned using the data-augmented training data to obtain a traffic detection model, including:

[0094] Convert the augmented training data into embedding vectors;

[0095] The encoder of the Transformer model is used to process the embedding vector to capture the complex patterns in the sequence data and generate the encoded feature vector F;

[0096] Use the decoder of the Transformer model to restore the encoded feature vector and generate a restored vector R = Decoder(F), where Decoder represents the decoder;

[0097] A combination of self-supervised learning and supervised learning is used to fine-tune and train the model to obtain a traffic detection model.

[0098] Specifically, the traffic detection model uses the Transformer architecture for feature extraction, and combines self-supervised learning with supervised learning to encode and restore network traffic data, converting the traffic data into feature vectors.

[0099] In the specific implementation process, the training data after data enhancement is X={x1,x2,…,x n} is converted to an embedding vector E = {e1, e2, …, e n}, where x i is the flow data, e i is the embedding vector, where n is the length of the vector, i∈[1,n].

[0100] The embedded vector is then processed using a multi-layer Transformer encoder to capture the complex patterns in the sequence data and generate an encoded feature vector F = Encoder(E), where F = {f1, f2, ..., f n}, where f i is the i-th traffic feature;

[0101] Then use the decoder to try to restore the encoded feature vector to generate a restored vector R = Decoder (F).

[0102] In one embodiment, when fine-tuning and training the model using a combination of self-supervised learning and supervised learning, the calculated losses include the reconstruction loss of the encoder and decoder, and the spherical loss during encoder encoding, wherein the spherical loss during encoder encoding is used to confine normal traffic samples within the sphere and exclude malicious traffic samples outside the sphere.

[0103] Specifically, a training method combining self-supervised learning and supervised learning is adopted, including self-supervised learning loss and supervised learning loss. The self-supervised learning loss is the reconstruction loss of the encoder and decoder. The cross entropy loss function is used to penalize the original data and the restored data. The reconstruction loss function is defined as:

[0104]

[0105] Among them, x i represents the input sample, represents the i-th flow data of the input, n represents the number of flow data (the length of the vector composed of flow data), The data after restoration.

[0106] The supervised learning loss is the spherical loss when the encoder is encoding. The spherical loss is used to constrain the encoded feature vector. The feature vector F of the normal flow normal In the sphere, the characteristic vector F of malicious traffic malicious Placed outside the sphere. For normal flow samples, the loss is defined as:

[0107]

[0108] For malicious traffic samples, the loss is defined as:

[0109]

[0110] Where c is the center of the sphere, R is the radius of the sphere, v is the hyperparameter that controls the relative strength of the two items, and x i is the input sample, and N is the number of samples in the dataset.

[0111] Total loss function Defined as:

[0112]

[0113] Among them, λ1 and λ2 are weight coefficients.

[0114] In one embodiment, a traffic detection model is used to extract traffic feature information from the traffic to be detected, and a KS test is performed based on the extracted feature information, including:

[0115] Pre-process the traffic to be detected.

[0116] The pre-processed traffic to be detected is encoded through the traffic detection model to extract the corresponding traffic feature information;

[0117] The KS hypothesis test is used on the traffic characteristics to give the confidence that the traffic is malicious traffic, and the hypothesis test is performed to calculate the test value. If the test value is greater than the threshold, the traffic to be detected is determined to be malicious traffic. Otherwise, the traffic to be detected is determined to be normal traffic.

[0118] In the specific implementation process, the input data X ts Use Transformer encoder to encode and get the feature vector F ts =Encoder(X ts ), and because the spherical loss is introduced during training, the distance between its feature and the center of the sphere is determined:

[0119]

[0120] It should be noted that the distance between the feature and the center of the sphere is used as an indicator for evaluating each sample in subsequent hypothesis testing. That is, in subsequent testing, hypothesis testing is performed on the data of the distance between the sample feature and the center of the sphere.

[0121] Based on the distance between the features and the center of the sphere calculated above, a hypothesis test is performed. In this embodiment, the null hypothesis H0 is: This sample is normal traffic. The alternative hypothesis H1 is: This sample is malicious traffic. Given a significance level α, the null hypothesis is rejected if the following equation is satisfied:

[0122]

[0123] Where F(x) and G(x) represent the empirical distribution functions of malicious samples and clean samples respectively, and l and m are the sizes of the two samples. is the critical value of the KS distribution at the significance level α, where k is the index notation of the sum, k∈[1,∞). One meaning of the significance level α is the probability of rejecting the null hypothesis incorrectly, that is, the false positive rate.

[0124] This paper primarily uses a self-attention mechanism to extract features and conduct hypothesis testing to identify malicious traffic. Clustering and data augmentation are also introduced to enhance feature extraction and identification, resulting in a highly accurate malicious traffic identification system with low false positives. This method can achieve high robustness even with low-quality training datasets.

[0125] In the specific implementation process, the method provided by the present invention should be realized with the help of computer software technology. Figure 1 The embodiment takes the training of a Transformer feature extractor as an example to specifically illustrate the process of the present invention as follows:

[0126] First, we build a neural network using Python programming and the PyTorch framework. The Transformer model can be directly called from the Torch library. Then, we can connect the various neural networks in series.

[0127] Then we need to define the loss of model training. The loss refers to the loss function described in the previous article. We need to define three losses respectively:

[0128] The first loss is the reconstruction loss of the Encoder and Decoder training. The Encoder inputs the flow data E = {e1, e2, ..., e n The decoder then attempts to restore the original traffic input data by encoding the features encoded by the encoder. The difference between the original input data and the decoded data is calculated using cross-entropy loss to measure the quality of the encoder and decoder. A smaller cross-entropy loss indicates a higher level of encoding and decoding, indicating that both have effectively learned the latent features of the input data. The prefix feature F encoded by the encoder can effectively represent the input data in the latent space.

[0129] The second and third losses are spherical losses during the encoder encoding process. Their purpose is to confine normal samples to the sphere and exclude malicious traffic samples from it. The sphere center c needs to be initialized, typically using random initialization or all-zero initialization. In actual training, multiple initialization attempts should be made to find the optimal location for the sphere center.

[0130] Superimposing the three losses requires manually setting the values of the hyperparameters λ1 and λ2, and adjusting the values of the hyperparameters multiple times during the training process to find the best relationship between the three so that the feature extractor can best learn how to extract features.

[0131] Since self-supervised learning is the main method and supervised learning is used as an auxiliary method, λ1 and λ2 should generally be set to less than 1. That is, it is recommended to start with 0.1 or 0.001 and find the optimal weights through continuous adjustment.

[0132] The Transformer is trained using this loss for 500 epochs, and then the Encoder is used for subsequent feature extraction.

[0133] The trained encoder is used for feature extraction in actual traffic monitoring. It extracts traffic features from the dataset and then performs clustering. Generally, because malicious traffic samples are relatively rare, the minimum sample size minPts should be set to 0, the distance threshold ∈ to 0.1, and the density threshold ρ to 1. These three parameters should also be optimized through multiple training runs in real-world applications.

[0134] Then it needs to be used in the subsequent random forest classification process. Train a classifier to reverse the labels of samples that may have incorrect labels and obtain samples with correct labels.

[0135] Based on the obtained dataset of correctly labeled samples (corrected training data), starGAN is trained. Since traffic has multiple characteristics, conventional GAN is abandoned and starGAN is used in the hope that it can learn different characteristics and perform well in predicting unknown malicious traffic. StarGAN training requires training a Generator (generator network) and a Discriminator (discriminator network). The goal of Generator training is to generate three different types of data, namely: normal samples, malicious traffic samples, and samples on the decision boundary; the goal of Discriminator training is to identify whether the samples generated by the Generator have the same characteristics as the original data, that is, whether the data it generates is as effective as the real data. The loss of Generator training is to reduce the Discriminator's discriminative power for the data it generates, that is, to try to make the Discriminator unable to distinguish the difference between the data it generates and the original data; the loss of the Discriminator is to distinguish the data generated by the Generator from the original data, that is, to try to distinguish the original data from the data generated by the Generator. The training of the two is carried out separately. In each training round, the Discriminator is trained first, for example, for 100 epochs, until it can distinguish between original data and data generated by the Generator. Then, the Discriminator is fixed and the Generator is trained to generate data that the Discriminator cannot distinguish, also for 100 epochs. This cycle repeats for a total of 500 epochs. StarGAN learns from the feature information data obtained by the encoder and uses the Generator to generate new sample data for further fine-tuning of the Transformer. It is worth noting that the GAN training process is highly random and often does not produce good results. Multiple training attempts are required to achieve good results.

[0136] Based on the obtained Generator, use it to generate new data, add it to the original training set, and fine-tune the Transformer with the new dataset (the training data after data augmentation). The fine-tuning process has been introduced in the previous article. This allows the Transformer to perform well on the new dataset, indicating that the feature extractor training is complete.

[0137] Based on the above training process, the trained Encoder can be deployed to the system that needs to perform traffic detection. For the verification process, please refer to Figure 2 , including the following processes:

[0138] The intercepted traffic is encoded in the latent space to obtain an embedding, which is then fed into the encoder to encode the traffic's latent features F. The distance s from the center of the spherical loss is then calculated. Following the hypothesis testing procedure, a significance level α is set, representing the desired false positive level in practice. This is typically set to 0.05, which assumes that 95% of all traffic reported as malicious is indeed malicious, resulting in a high level of accuracy. This traffic data is then mixed with clean samples, and N samples are sampled without replacement, followed by the KS test. The KS test directly calls the ks_2samp function in the SciPy library, which is used to determine whether two independent samples come from the same distribution. This function determines whether the sample and the clean sample share the same distribution, effectively determining whether the sample is a normal traffic sample, given the significance level. The function returns a p-value, which represents the probability of observing the current KS statistic or a more extreme one, given the null hypothesis is true. A smaller p-value indicates a stronger reason to reject the null hypothesis. A KS statistic is also returned, which measures the maximum difference between the cumulative distribution functions of two samples. The larger the value, the greater the difference in the distribution of the two samples. Pre-set thresholds can be used to distinguish between normal and malicious traffic samples. In practice, the threshold setting requires multiple experiments based on different datasets and actual traffic interception requirements.

[0139] Based on the above method, after the feature extractor is trained, it is deployed to the system, and after the traffic features are calculated, the KS test is performed on it, so that malicious traffic can be judged with high accuracy, which has strong practical significance. In general, in order to monitor some malicious traffic in network traffic and protect the server from being attacked by malicious traffic, the present invention discloses a method and system based on deep learning to extract features in traffic by using Transformer, and to try to find the interface between normal traffic and malicious traffic and divide them through spherical division. Before the system is deployed, it is necessary to first use the existing publicly available normal traffic and malicious traffic data sets on the Internet to train the encoder, then enhance the training data through starGAN, and then continue to fine-tune the Transformer training. After the training is completed, the feature information in the traffic is extracted and then the KS test is performed on the device. Then, normal samples can be distinguished from malicious samples with high confidence. Then, it can be deployed in the server traffic detection program. In the actual traffic detection process, the traffic data is first pre-processed and then encoded by the encoder to obtain its features. Then, the KS hypothesis test is used on the traffic features to give the confidence that the traffic is malicious traffic. Through this confidence hypothesis test, it can be determined within a certain threshold range whether the traffic is malicious traffic. Since the present invention adopts a method for finding the interface between normal samples and malicious samples to classify them, it can effectively prevent malicious adversarial attacks on traffic and some attacks with currently unknown methods to a certain extent.

[0140] Example 2

[0141] Based on the same inventive concept, this embodiment discloses a malicious traffic hypothesis detection device based on an attention mechanism and data enhancement, comprising:

[0142] The data acquisition module is used to obtain public normal traffic and malicious traffic data sets to construct original training data;

[0143] The label correction module is used to estimate the data distribution of the original training data using the DBSCAN clustering algorithm, re-label the samples according to the clustering results of the DBSCAN clustering algorithm, and use the random forest classifier to train and predict the re-labeled samples to obtain the corrected training data;

[0144] The data enhancement module is used to perform data enhancement on the corrected training data using StarGAN to obtain data-enhanced training data;

[0145] The fine-tuning training module is used to fine-tune the Transformer model using the data-augmented training data to obtain a traffic detection model;

[0146] The flow detection module is used to extract characteristic information from the flow to be detected by using the flow detection model, and perform KS test based on the extracted characteristic information.

[0147] Since the device described in Example 2 of the present invention is used to implement the malicious traffic hypothesis detection method based on the attention mechanism and data enhancement described in Example 1 of the present invention, the specific structure and variations of the device are readily understood by those skilled in the art based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All devices used in the method described in Example 1 of the present invention fall within the scope of protection of the present invention.

[0148] Example 3

[0149] Based on the same inventive concept, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0150] Since the computer-readable storage medium described in Example 3 of the present invention is the computer-readable storage medium used to implement the malicious traffic hypothesis detection method based on the attention mechanism and data enhancement described in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of the computer-readable storage medium based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All computer-readable storage media used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0151] Example 4

[0152] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first embodiment when executing the program.

[0153] Since the computer device described in Example 4 of the present invention is the computer device used to implement the malicious traffic hypothesis detection method based on the attention mechanism and data enhancement in Example 1 of the present invention, based on the method described in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of the computer device, so they will not be described in detail here. All computer devices used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0154] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, the present invention is intended to include such changes and modifications to the embodiments of the present invention if they fall within the scope of the claims and their equivalents.

Claims

1. A malicious traffic hypothesis testing method based on attention mechanism and data enhancement, characterized by: include: Obtain public normal traffic and malicious traffic datasets to construct original training data; The DBSCAN clustering algorithm is used to estimate the data distribution of the original training data. The data are relabeled according to the clustering results of the DBSCAN clustering algorithm. The relabeled samples are trained and predicted using the random forest classifier to obtain the corrected training data. Use StarGAN to perform data enhancement on the corrected training data to obtain data-enhanced training data; The Transformer model is fine-tuned using the data-augmented training data to obtain a traffic detection model. The traffic detection model is used to extract characteristic information from the traffic to be detected, and the KS test is performed based on the extracted characteristic information; The Transformer model is fine-tuned using the data-augmented training data to obtain a traffic detection model, including: Convert the augmented training data into embedding vectors; The encoder of the Transformer model is used to process the embedded vector to capture the complex patterns in the sequence data and generate the encoded feature vector ; Use the decoder of the Transformer model to restore the encoded feature vector and generate a restored vector , Represents a decoder; A combination of self-supervised learning and supervised learning is used to fine-tune and train the model to obtain a traffic detection model; When fine-tuning and training the model using a combination of self-supervised learning and supervised learning, the calculated losses include the reconstruction loss of the encoder and decoder, and the spherical loss during encoder encoding. The spherical loss during encoder encoding is used to confine normal traffic samples within the sphere and exclude malicious traffic samples from the sphere. The traffic detection model is used to extract traffic feature information from the traffic to be detected, and the KS test is performed based on the extracted feature information, including: Pre-process the traffic to be detected. The pre-processed traffic to be detected is encoded through the traffic detection model to extract the corresponding traffic feature information; The KS hypothesis test is used on the traffic characteristics to give the confidence level that the traffic is malicious traffic, and the hypothesis test is performed to calculate the test value. If the test value is greater than the threshold, the traffic to be detected is determined to be malicious traffic. Otherwise, the traffic to be detected is determined to be normal traffic. The KS hypothesis test is used on the traffic feature to determine the confidence level that the traffic is malicious. The hypothesis test is then performed to calculate the test value. This includes calculating the distance between the feature and the center of the sphere. The distance between the feature and the center of the sphere is used as an indicator for evaluating each sample in subsequent hypothesis tests. The calculation formula for the distance between the feature and the center of the sphere is: ； in, is the traffic to be detected after preprocessing, It represents the feature vector obtained by encoding the pre-processed traffic to be detected using the Transformer encoder. is the center of the sphere, is the distance between the feature and the center of the sphere; Calculate the test value ,in, and represent the empirical distribution functions of malicious samples and clean samples respectively; Determine whether the test value is greater than the threshold If it is greater than, the null hypothesis is rejected and the traffic to be detected is determined to be malicious traffic. Otherwise, the traffic to be detected is determined to be normal traffic. and are the sizes of malicious and clean samples, is the significance level The critical value of the lower KS distribution, , where k is the index symbol of the summation, , significance level represents the probability of falsely rejecting the null hypothesis.

2. The malicious traffic hypothesis testing method based on attention mechanism and data enhancement as claimed in claim 1, characterized in that: The DBSCAN clustering algorithm is used to estimate the data distribution of the original training data. The data is relabeled according to the clustering results of the DBSCAN clustering algorithm. The relabeled samples are trained and predicted using the random forest classifier to obtain the corrected training data, including: Use the DBSCAN clustering algorithm to cluster the original training data and estimate the distribution of the data; Based on the DBSCAN clustering results, samples in high-density areas are re-labeled as normal samples, and samples in low-density areas are re-labeled as malicious samples. The areas above the density threshold are high-density areas, and the areas below the density threshold are low-density areas. The relabeled samples are trained and predicted using a random forest classifier, and the labels of the decision boundary samples are corrected to obtain the corrected training data. The decision boundary samples are the samples at the boundary between the high-density area and the low-density area when clustering is performed using the DBSCAN clustering algorithm.

3. The malicious traffic hypothesis testing method based on attention mechanism and data enhancement as claimed in claim 1, characterized in that: Use the DBSCAN clustering algorithm to cluster the original training data and estimate the distribution of the data, including: Set the minimum number of samples and distance threshold , according to the formula Calculate the sample data points The regional density, and are two sample data points, represents the dataset consisting of the original training data, Indicates The cluster with the core as the core, that is, the sample data point The regional density, Represents the distance between two sample data points.

4. The malicious traffic hypothesis testing method based on attention mechanism and data enhancement as claimed in claim 1, characterized in that: StarGAN is used to perform data augmentation on the corrected training data to obtain data augmented training data, including: Define target areas for data augmentation based on the distribution of malicious and normal data; Use StarGAN to generate new data so that its distribution conforms to the distribution characteristics of the target area; The generated new data is combined with the corrected training data to obtain data-augmented training data.

5. A malicious traffic hypothesis testing device based on attention mechanism and data enhancement, characterized in that: Based on the malicious traffic hypothesis testing method based on the attention mechanism and data enhancement described in claim 1, the device includes: The data acquisition module is used to obtain public normal traffic and malicious traffic data sets to construct original training data; The label correction module is used to estimate the data distribution of the original training data using the DBSCAN clustering algorithm, re-label the samples according to the clustering results of the DBSCAN clustering algorithm, and use the random forest classifier to train and predict the re-labeled samples to obtain the corrected training data; The data enhancement module is used to perform data enhancement on the corrected training data using StarGAN to obtain data-enhanced training data; The fine-tuning training module is used to fine-tune the Transformer model using the data-augmented training data to obtain a traffic detection model; The flow detection module is used to extract characteristic information from the flow to be detected by using the flow detection model, and perform KS test based on the extracted characteristic information.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the malicious traffic hypothesis testing method based on the attention mechanism and data enhancement is implemented as described in any one of claims 1 to 4.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the malicious traffic hypothesis testing method based on the attention mechanism and data enhancement is implemented as described in any one of claims 1 to 4.

Images