Classification method and system for class imbalance encrypted network traffic

Abstract

The invention provides a classification method and system for class imbalance encrypted network traffic, and relates to the technical field of network traffic classification. Comprising the following steps: acquiring unbalanced encrypted network traffic of an original category, performing preprocessing, and generating a real Markov image representing network traffic characteristics; the method comprises the following steps: establishing a CWGAN-GP network comprising a generator and a discriminator, inputting a real Markov image into the CWGAN-GP network, generating a Markov image by using the generator, and carrying out category discrimination on the generated Markov image and the real Markov image by using the discriminator, thereby realizing the classification of the unbalanced encrypted network traffic of the original category. According to the method, the optimization problem and the sample reliability problem in the WGAN are solved, the class balance of sample generation is ensured, and more efficient calculation is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of network traffic classification, and in particular relates to a classification method and system for classifying unbalanced encrypted network traffic. Background Art

[0002] With the popularization of the Internet and the continuous development of digital technology, network traffic has experienced explosive growth. More and more human activities, such as online video, social media, cloud computing, etc., rely on network connections, which has led to a continuous increase in network traffic. The huge amount of network traffic not only brings huge operational pressure to network operators, but also poses a considerable challenge to regulatory authorities in effectively supervising network situations. In this context, it is of great significance to effectively classify network application traffic. Accurate and efficient network traffic classification can help network operators optimize network resource allocation, improve network performance, provide better user experience, enhance user satisfaction, thereby increasing customer loyalty and market competitiveness. It can also help regulatory authorities better identify and respond to network threats, strengthen network security monitoring, protect user privacy, maintain cyberspace order, and ensure national information security.

[0003] Deep learning technology has significant advantages in the classification of network traffic: it can autonomously identify key features from raw data without human intervention, which gives it the flexibility to adapt to various types of traffic and effectively cope with the challenges caused by encrypted traffic, thereby achieving significant improvements in classification accuracy. However, deep learning can usually provide higher accuracy in the classification task of encrypted traffic, but this is based on the relatively balanced distribution of the data set used for model training. In actual network traffic analysis, class imbalance is a common problem. In addition, most current traffic classification studies tend to use overall accuracy to evaluate the performance of classification algorithms, which may not fully reflect the performance of the algorithm in different categories, and may ignore the shortcomings of the algorithm when processing minority category samples. For example, in an enterprise network, normal data transmission traffic may account for 99%, while malicious attack traffic such as phishing attempts or intrusion attempts may only account for 1%. In this case, if the classification algorithm only focuses on the overall classification accuracy, it may get a seemingly good score due to the high recognition rate of normal traffic, but in fact it may completely ignore the 1% of malicious traffic, which is far from enough for network security.

[0004] Therefore, strengthening the classification of minority classes in traffic is very important for managing networks. Currently, a common method for classifying unbalanced traffic data is called oversampling. Its basic idea is to increase the number of samples of minority classes to match or approach the number of samples of majority classes, thereby improving the model's ability to identify minority classes. Existing oversampling methods are mainly divided into traditional methods and methods based on generative adversarial networks. Most traditional oversampling methods are based on SMOTE (Synthetic Minority Over-sampling Technique) technology. However, SMOTE increases the dimension of data when generating synthetic samples, resulting in increased computational complexity and memory usage, and is prone to generating too many samples near the decision boundary, resulting in model overfitting. On the other hand, since SMOTE generates synthetic samples based on neighboring samples, it may cause the distribution of the original data to be distorted. Due to the above shortcomings, the generative adversarial network (GAN) method came into being. GAN is a generative model that consists of two networks, a generator and a discriminator, and generates realistic data samples through adversarial training models. Compared with the SMOTE method, GAN can generate more realistic data samples without causing problems such as increased data dimension or data distortion.

[0005] However, the traditional GAN ​​training process is complex and unstable, and the following problems may occur:

[0006] ① Gradient disappearance: This is because the GAN model uses Jensen-Shannon divergence (JS divergence). When the balance between the generator and the discriminator is broken, the gradient may disappear, affecting the training of the model.

[0007] ② Mode collapse: The generator may tend to generate samples that are highly similar to real samples, which limits the diversity of generated samples, which is called mode collapse.

[0008] ③ Training difficulties: The training process of GAN is very sensitive to the choice of hyperparameters. If you are not careful, the generated model may fluctuate greatly or fail to converge.

[0009] In order to solve the above problems, Wasserstein GAN (WGAN) came into being. WGAN successfully improved the gradient vanishing problem and made some improvements to the mode collapse problem by introducing Wasserstein distance instead of JS divergence. However, it needs to satisfy the Lipschitz constraint and is implemented through weight clipping, which may lead to extreme network weights, increase the difficulty of optimization, and affect the reliability of generated samples. In addition, WGAN is an unsupervised model, and its generated samples are random, which makes it unsuitable for use in multi-class imbalanced scenarios in the current network environment.

[0010] In addition, traditional GANs are still limited in reproducing global image structures and capturing long-distance dependencies across image regions. In addition, there are limitations in computational cost and algorithmic efficiency. These problems highlight the essential limitations of convolution operations, that is, although they can capture the spatial invariance of natural images, they are difficult to effectively model complex geometric shapes and long-distance dependencies. To address these problems, the self-attention mechanism was proposed and applied to the GAN architecture. Self-attention is a technique that can create long-range dependencies in sequence data. It emphasizes important features while suppressing unimportant features by dynamically redistributing attention weights. This mechanism first achieved great success in the field of natural language processing and was widely used in Transformer models. In image generation, by calculating the pairing relationship of all elements in the image features, the self-attention mechanism can help the model understand and reconstruct more complex global structures and long-distance dependencies.

[0011] However, the introduction of self-attention also brings some new challenges. First, self-attention is computationally inefficient, as the memory and time complexity of the standard dense attention layer (how the self-attention mechanism is calculated) is quadratic to the input size. Second, it is also statistically inefficient: self-attention layers require a large number of training samples to be effectively trained, which is more pronounced when multiple attention heads or layers are introduced. In addition, a major problem with dense attention in image processing is that they do not make good use of a key feature of image data - locality. In an image, most pixels have a strong correlation with their immediate neighbors, because most dependencies in an image are related to the neighboring pixel neighborhood. However, dense attention layers consider the relationships between all pixels in the image equally when calculating, which leads to a lot of unnecessary calculations, because the relationships between distant pixels are usually not as important as those between nearby pixels. This computational waste reduces the statistical efficiency of the training process.

[0012] On the other hand, due to the diversity of the original sizes of network traffic data, traditional techniques for standardizing image sizes often require truncating or supplementing bytes to achieve size consistency. Each pixel in the image maps local information of the data byte. However, this direct mapping method may ignore the relevance of byte information in encrypted traffic identification. As a result, traditional image generation strategies may cause omissions of key information and include some irrelevant redundant data, which in turn may affect the performance of encrypted traffic classification.

[0013] In summary, traditional generative adversarial networks (GANs) face multiple challenges during training, including gradient vanishing, mode collapse, and training difficulties. These problems are partially alleviated by Wasserstein GAN (WGAN) by introducing Wasserstein distance, but WGAN's own Lipschitz constraint and weight clipping bring new optimization challenges. In addition, traditional GANs have limitations in reproducing global image structures and capturing long-distance dependencies, and are also insufficient in computational cost and algorithmic efficiency. To overcome these limitations, the self-attention mechanism is introduced into the GAN architecture, which enhances the modeling ability of complex global structures and long-distance dependencies by dynamically adjusting the importance of features. However, the self-attention mechanism also brings computational and statistical inefficiencies, as well as a high demand for the number of training samples. In terms of image processing of network traffic data, traditional size unification techniques, such as byte truncation or padding, may cause information loss and redundancy, affecting the classification effect of encrypted traffic. Summary of the invention

[0014] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a classification method and system for category-imbalanced encrypted network traffic, uses a new grayscale image generation technology, utilizes a Markov chain based on byte transition probability to create a grayscale image, replaces the original byte truncation and padding method, establishes a CWGAN-GP network model, and replaces weight clipping with gradient penalty GP (Gradient Penalty), thereby overcoming the optimization difficulties and sample reliability problems caused by the Lipschitz constraint in WGAN; secondly, introduces a conditional generation mechanism, improves the randomness limitations of WGAN in multi-category imbalance scenarios, ensures the category balance of sample generation, and uses a new local sparse attention layer to replace the traditional dense attention layer, thereby ensuring more efficient calculation.

[0015] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0016] A first aspect of the present invention provides a classification method for class-unbalanced encrypted network traffic.

[0017] A classification method for class-imbalanced encrypted network traffic, comprising the following steps:

[0018] Obtain the original class-imbalanced encrypted network traffic, perform preprocessing, and generate a real Markov image that characterizes the network traffic characteristics;

[0019] Build a CWGAN-GP network including a generator and a discriminator, input the real Markov image into the CWGAN-GP network, use the generator to generate the Markov image, and use the discriminator to classify the generated Markov image and the real Markov image, so as to classify the original category-unbalanced encrypted network traffic;

[0020] Among them, local sparse attention layers are introduced in both the generator and the discriminator to create a sparse pattern mask that is selective to the calculation in the attention mechanism. The calculation results of the query and key are modified based on the sparse pattern mask, and then the output feature matrix modified by the sparse pattern mask is calculated.

[0021] A second aspect of the present invention provides a classification system for class-unbalanced encrypted network traffic.

[0022] A classification system for class-imbalanced encrypted network traffic, including:

[0023] The preprocessing module is configured to: obtain the original class-imbalanced encrypted network traffic, perform preprocessing, and generate a real Markov image representing the characteristics of the network traffic;

[0024] The generation and discrimination module is configured to: build a CWGAN-GP network including a generator and a discriminator, input the real Markov image into the CWGAN-GP network, use the generator to generate the Markov image, and use the discriminator to classify the generated Markov image and the real Markov image, so as to classify the original category-unbalanced encrypted network traffic;

[0025] Among them, local sparse attention layers are introduced in both the generator and the discriminator to create a sparse pattern mask that is selective to the calculation in the attention mechanism. The calculation results of the query and key are modified based on the sparse pattern mask, and then the output feature matrix modified by the sparse pattern mask is calculated.

[0026] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the classification method for classifying category-unbalanced encrypted network traffic as described in the first aspect of the present invention.

[0027] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the classification method for category-unbalanced encrypted network traffic as described in the first aspect of the present invention are implemented.

[0028] One or more of the above technical solutions have the following beneficial effects:

[0029] ① Efficient image generation technology: The present invention provides a classification method and system for category-imbalanced encrypted network traffic. It adopts a new grayscale image generation method based on Markov chain to generate Markov images that characterize network traffic characteristics, effectively retaining key data information while reducing redundancy, thereby enhancing the accuracy of image classification and the generalization ability of the model.

[0030] ② Optimized WGAN solution: The conditional Wasserstein generative adversarial network (CWGAN-GP) is introduced, and the gradient penalty technology is used to replace the traditional weight clipping. The gradient of the discriminator output is penalized to avoid the problem of gradient disappearance or explosion, which solves the Lipschitz constraint problem in WGAN and improves the reliability of sample generation.

[0031] ③ Improved sample generation balance: Through the conditional generation mechanism, category labels are introduced as additional conditions in the generator and discriminator of the CWGAN-GP network, and the corresponding category labels and the real Markov images or the generated Markov images are used as conditional constraints, which overcomes the limitations of WGAN in multi-class imbalance scenarios and ensures the category balance of sample generation.

[0032] ④ Computationally efficient self-attention mechanism: The present invention uses a local sparse attention layer to replace the traditional dense attention layer. By creating a binary sparse pattern mask, the sparse pattern mask is applied to the query matrix Q and the key matrix K to obtain a sparse query matrix and a sparse key matrix respectively. The sparse query matrix and the sparse key matrix are used to calculate the attention weights, and the obtained attention weights are used to perform weighted summation on the value matrix V to obtain the final output feature matrix, which maintains the locality and geometric structure of the image while reducing the amount of computation and improving efficiency.

[0033] ⑤ Significant performance improvement: The structural improvement of the present invention has achieved significant improvements in FID, Inception score and visual effects, proving its superior performance in the field of image generation.

[0034] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0036] Figure 1 This is a flow chart of the method of the first embodiment.

[0037] Figure 2is the generation process of Markov images.

[0038] Figure 3 FIG. 4 is a diagram of the CWGAN-GP network architecture in the first embodiment. DETAILED DESCRIPTION

[0039] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0041] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0042] Embodiment 1

[0043] As mentioned above, this embodiment provides a classification solution with excellent performance and strong robustness for the classification of category-unbalanced encrypted network traffic, which is specifically used to solve the following technical problems:

[0044] ① Lipschitz constraint problem of Wasserstein GAN (WGAN): WGAN satisfies the Lipschitz constraint through weight clipping, which may lead to extreme network weights, increase the difficulty of optimization, and affect the reliability of generated samples.

[0045] ② Randomness problem of unsupervised models: As an unsupervised model, WGAN generates samples randomly, which limits its use in multi-class imbalance scenarios and cannot guarantee the category balance of generated samples.

[0046] ③Computational and statistical efficiency issues of the self-attention mechanism: Although the self-attention mechanism can create long-range dependencies, it has high computational complexity, requires a large number of training samples, and has low statistical efficiency.

[0047] ④ Locality problem of dense attention layer: Since dense attention (the calculation method of self-attention mechanism) is not conducive to benefiting from the locality of image pixels, it leads to reduced efficiency.

[0048] ⑤The problem of key information loss in network traffic data: When processing network traffic data, traditional technologies achieve size consistency by truncating or supplementing bytes, which may ignore the relevance of byte information, causing key information to be missed and affecting classification performance.

[0049] ⑥ The inclusion of irrelevant redundant data: Traditional image generation strategies may contain irrelevant redundant data, affecting the performance of encrypted traffic classification.

[0050] The processing method for classifying encrypted network traffic with imbalanced categories proposed in this embodiment includes:

[0051] First, this embodiment uses a new grayscale image generation technology, using a Markov chain based on byte transition probability to create grayscale images, replacing the original byte truncation and padding method. This improvement not only retains the key information of the data, but also reduces redundancy, enhances the accuracy of image classification and the generalization of the model.

[0052] Then, this embodiment proposes an improved solution based on the conditional Wasserstein generative adversarial network, CWGAN-GP (Conditional Wasserstein Generative Adversarial Network with Gradient Penalty), which effectively solves the two major problems faced by WGAN: first, by replacing weight clipping with gradient penalty GP (Gradient Penalty), the optimization difficulties and sample reliability problems caused by the Lipschitz constraint in WGAN are overcome; secondly, the conditional generation mechanism is introduced to improve the randomness limitations of WGAN in multi-class imbalance scenarios and ensure the category balance of sample generation.

[0053] At the same time, this embodiment uses a new local sparse attention layer to replace the traditional dense attention layer, which maintains the locality and geometric structure of the two-dimensional image. Compared with the traditional dense attention layer (the calculation method of the self-attention mechanism), this layer is more computationally efficient because it reduces the number of attention weights that need to be calculated. Replacing the dense attention layer of SAGAN with this structure can obtain more significant FID, Inception score and visual effects.

[0054] To achieve the above purpose, the specific implementation process of this embodiment includes: generating Markov images, building a CWGAN-GP network, and introducing a local sparse attention layer. Figure 1 As shown:

[0055] (1) Generate Markov image

[0056] Step 1: The initial step involves grouping the raw pcap files of network traffic into separate session files based on five key elements: source IP, destination IP, source port, destination port, and the transport layer protocol used.

[0057] Step 2: Subsequently, an analysis tool such as Wireshark is used to filter the grouped network data packets in order to remove unnecessary information, such as redundant packets such as TCP retransmissions, in order to purify the data set.

[0058] Step 3: Consider the byte sequence in each session file as a Markov process and estimate the transition probability between bytes by analyzing the follow-up relationship between them.

[0059] Step 4: Based on the calculated transition probabilities, create a matrix with a dimension of 256x256, which represents the transition possibilities between bytes and is used to describe the characteristics of the Markov chain.

[0060] Step 5: Finally, use the above probability matrix to generate images, which represent the Markov chain state of network traffic and can be used as input data for network analysis.

[0061] (2) Building the CWGAN-GP network

[0062] ① Generator network: The generator network accepts random noise vectors and conditional inputs, and generates images through a series of neural network layers (such as convolutional layers, activation layers, etc.). The goal of the generator is to generate images that are as close to the real data distribution as possible.

[0063] ②Discriminator network: The discriminator network receives real images and their conditional inputs or images produced by the generator and their conditional inputs and tries to distinguish them. The goal of the discriminator is to maximize its ability to distinguish between real and fake images.

[0064] ③Wasserstein distance: CWGAN-GP uses Wasserstein distance as the main loss function, which requires the discriminator network to output an unbounded authenticity score instead of a probability.

[0065] ④ Gradient Penalty (GP): In order to stabilize the training process, CWGAN-GP introduces gradient penalty, which involves penalizing the gradient of the discriminator output to avoid the problem of gradient disappearance or explosion.

[0066] (3) Introduction of local sparse attention layer

[0067] Step 1: Design a sparse pattern, which is usually a binary mask that determines which locations are active (i.e., can participate in the calculation) and which are ignored (i.e., do not participate in the calculation) in the attention calculation. This mask is based on the principle of two-dimensional locality and only selects adjacent pixels or features in the image.

[0068] Step 2: Use Information Flow Graphs (IFGs) to ensure that information flows efficiently in the network even after sparsification. IFGs is a directed acyclic graph used to model and optimize the information flow in the attention layer.

[0069] Step 3: Similar to dense attention, the local sparse attention layer also needs to calculate the query (Q), key (K), and value (V) matrices. These matrices are obtained by multiplying the input feature matrix X with the corresponding weight matrix:

[0070] Q=X*W Q ,K=X*W K ,V=X*W V ;

[0071] Among them, W Q， W K， W V is a learnable weight matrix.

[0072] Step 4: Apply a sparse pattern mask to the query and key calculations to reduce the number of pairs of pixels or features that need to be considered. This is usually done by element-wise multiplication (Hadamard product):

[0073] Sparse Q =Q⊙M,Sparse K =K⊙M.

[0074] Where M is the sparse pattern mask.

[0075] Step 5: Use the modified query and key matrices to compute the attention weights. This typically involves computing their dot product and then applying a softmax function to obtain normalized weights:

[0076] Attention Weights=softmax(Sparse Q *Sparse K T );

[0077] Step 6: Use the obtained attention weights to perform weighted summation on the value matrix V to obtain the final output feature matrix:

[0078] Output=Attention Weights*V;

[0079] Step 7: Use the ESA (Enumerate, Shift, Apply) framework to adjust the one-dimensional sparse pattern to fit the two-dimensional image structure. This process includes:

[0080] ① Enumerate: Renumber pixels according to their Manhattan distance.

[0081] ②Shift: Adjust the sparsity pattern to match the Manhattan distance-based enumeration.

[0082] ③ Apply: Apply the adjusted sparsity pattern to attention calculation.

[0083] Step 9: During the training process, the weight matrix W is updated through optimization algorithms such as back propagation and gradient descent Q， W K， W V and sparse pattern mask to minimize the loss function.

[0084] Furthermore, the specific implementation of this embodiment is as follows:

[0085] Step 1: Preprocess the collected original encrypted traffic to generate a Markov image;

[0086] Step 2: Build a CWGAN-GP network and introduce a local sparse attention layer into it. Then input the generated Markov image into the CWGAN-GP network for image generation, which is used to balance the network traffic dataset so that the number of minority classes can be increased.

[0087] First, let’s elaborate on step 1:

[0088] Compared with traditional grayscale images, the Markov image is generated by using the byte transfer probability in the session file, that is, the probability of a certain byte transferring to another byte, as the pixel point in the image, rather than a single byte. Because the byte range is 0-255, the Markov image generation method does not require truncation or padding operations regardless of the size of the session file, and can finally be unified into the same size, that is, 256x256, so it can effectively avoid information loss and reduce information redundancy. At the same time, because the Markov image is a fixed size of 256x256, it can adapt to various data sets, thereby effectively improving generalization.

[0089] In network traffic analysis, the immediate state of traffic is usually only affected by the most recent state, which is called the Markov property. This property allows traffic analysis to focus on the most recent data bytes rather than the entire data sequence.

[0090] like Figure 2 As shown, the generation process of the Markov image is as follows:

[0091] ①First, the original network data packets (pcap files) need to be grouped according to the session mode. A session refers to a collection of data packets with the same five-tuple characteristics (source IP, source port, destination IP, destination port and protocol type).

[0092] ② Use professional tools, such as Wireshark, to filter data packets and exclude unnecessary information, such as TCP retransmission packets, to reduce data redundancy and generate a more concise conversation data set.

[0093] ③ Further process the session dataset to remove unnecessary information in the packet header, such as noise data, detailed information of the data link layer, and IP addresses, in order to reduce the model's learning of these irrelevant features and thus improve the model's generalization ability.

[0094] ④The binary file of the network session can be regarded as a sequence of bytes arranged in order, which follows the Markov property, that is, the appearance of each byte depends only on its immediate predecessor. The probability formula of the Markov chain is as follows:

[0095] P(Byte i+1 |Byte 0 , ..., Byte i )=P(Byte i+1 |Byte i )

[0096] Among them, Byte i+1 |Byte 0 , ..., Byte i It is expressed as the probability of the i+1th byte appearing given the first i bytes. i+1 |Byte i It represents the characteristic of Markov chain, that is, the probability of occurrence of the i+1th byte depends only on the i-th byte and has nothing to do with earlier bytes in the sequence.

[0097] Since each byte can take integer values ​​from 0 to 255, there is a 256x256 transition probability matrix.

[0098] ⑤ Specific calculation of transition probability: transition probability P xy , which represents the probability that byte x is followed by byte y, can be calculated by the following formula:

[0099]

[0100] Where f(x, y) is the number of times byte x is followed by byte y, and the denominator is the sum of f(x, y) for all possible subsequent bytes y.

[0101] ⑥ By analyzing the byte sequence in the session file, the transition probability of each byte pair is calculated, and then a 256x256 transition probability matrix M is generated, as shown below:

[0102]

[0103] ⑦ Using this probability matrix, a Markov chain image can be generated. In this process, each element in the matrix represents a pixel in the image, and these pixels can be used to construct a characteristic image of network traffic. The above overall process is as follows Figure 2 shown.

[0104] exist Figure 2 In the example, the original category unbalanced encrypted network traffic is grouped to form independent session files. Each independent session file can be expressed as Figure 2 The leftmost frame in ;

[0105] After that, the byte sequence in each session file is regarded as a Markov process. The transition probability between bytes is estimated by analyzing the follow-up relationship between bytes. The probability of transition from one byte to another byte (including the byte itself) is expressed as Figure 2 middle frame;

[0106] Finally, based on the calculated transition probability, a transition probability matrix representing the possibility of transition between bytes is generated, and a Markov image is generated based on the transition probability matrix, using each element in the transition probability matrix to represent a pixel in the Markov image. The corresponding process can be expressed as Figure 2 The rightmost frame in the .

[0107] Next, step 2 is described in detail:

[0108] 1. Build the CWGAN-GP network

[0109] Definition: CWGAN-GP is an improved model of WGAN that overcomes the limitations of WGAN by adopting a new gradient penalty mechanism. This gradient penalty method replaces the original weight clipping technique, helps to avoid extreme distribution of weight values ​​and ensures that the discriminator can better follow the Lipschitz continuity condition. The overall architecture is as follows Figure 3 shown.

[0110] Specifically, the implementation of the gradient penalty involves calculating the two-norm between the gradient of the discriminator output and a predefined constant K. This process first randomly selects a value in the interval between 0 and 1, and then mixes the real data samples with the generated data samples in this random ratio. In this way, the Lipschitz continuity can be achieved in each batch of data.

[0111] In the sampling process, two distributions are used: the distribution of real data P_r and the distribution of generated data P_g. Sampling is performed using the following formula, where α represents the intermediate value obtained by sampling, which is a mixture of the real sample x and the generated sample x' according to the weight of the random variable λ, which is uniformly distributed between 0 and 1:

[0112] α=λx+(1-λ)x′,λ~Uniform[0,1],x~P r ,x′~P g

[0113] In this formula, x is the sample obtained by sampling from the true data distribution, x' is the sample obtained by sampling from the generated data distribution, and λ is the random variable that controls the weights of x and x' in α.

[0114] In this way, CWGAN-GP is able to effectively interpolate between real data and generated data, thereby improving the performance and stability of the model.

[0115] At the same time, in order to construct the process of supervised data generation, the CWGAN-GP model introduces additional condition information c in the discriminator and generator to adapt to the multi-category imbalance problem in modern network environments. In the present invention, the category label of the traffic data is used as an additional condition and input into the model together with the feature image of the traffic data to form a conditional constraint. The objective function of the CWGAN-GP model can be expressed as:

[0116]

[0117] In this formula, represents the mathematical expectation of the sample, c represents the additional conditional information, P r and P g Represent the distribution of real data and generated data respectively. The discriminator D evaluates whether the sample x or α is real or generated according to the condition c. λ is a regulation term used to control the sensitivity of the discriminator to the generated samples, and the gradient penalty term λ Ensure that the output of the discriminator changes smoothly in the sample space.

[0118] The objective function minmaxV(D,G) reflects the competition between the generator G and the discriminator D. In this process, the generator G strives to produce samples so that they are consistent with the real sample distribution P under the evaluation of the discriminator D. r No difference. The discriminator D tries to maximize its ability to distinguish between real samples and generated samples. This design leads to a dynamic balance between the two, where G continuously improves its generation quality and D continuously improves its discrimination ability.

[0119] The loss functions of the discriminator and generator are defined as follows:

[0120] The loss function L(D) of the discriminator is defined as the difference between the expected value of the discriminant probability of the real sample and the generated sample, plus a gradient penalty term. The specific formula is as follows:

[0121]

[0122] Here, the first term is the negative expected value of the probability that the discriminator D will judge the real sample x as real under given conditions c. The second term is the expected value of the probability that the generated sample α is judged as real under the same conditions. The last term is the gradient penalty, which is used to smooth the decision boundary of the discriminator, where λ is a hyperparameter that controls the strength of the penalty.

[0123] The loss function L(G) of the generator is the negative expected value of the probability that the generated sample is judged as true by the discriminator. The formula is as follows:

[0124]

[0125] This means that the goal of the generator is to maximize D(α|c), that is, the probability that the generated sample α is considered real by the discriminator D.

[0126] These loss functions are the core components in Generative Adversarial Networks, which guide the training process of the generator and discriminator to produce high-quality and indistinguishable samples.

[0127] Generator execution flow:

[0128] ① Input layer: First, the input layer receives a random noise vector z with a dimension of 100 and a set of category labels, which are combined with the noise vector to form an initial noise feature vector.

[0129] ②Dense layer: Next, the feature vector passes through a fully connected layer (Dense layer), which maps the input to a feature space with a higher dimension, and the output dimension is (b, 20480), where b represents the number of samples in the batch.

[0130] ③Reshaping operation: The output of the Dense layer then undergoes a reshaping operation to convert it into a dimension suitable for subsequent convolution operations, i.e. (b, 8, 8, 640).

[0131] ④ Transposed convolution layer: The feature map is then upsampled through a series of transposed convolution layers. The first layer of transposed convolution has a 3×3 convolution kernel, a stride of 2, uses 256 convolution kernels, and the output dimension is (b, 16, 16, 256).

[0132] ⑤ Local sparse attention layer: In order to enhance the model's ability to capture local features and introduce sparsity in the feature map, we introduce a local sparse attention layer after the second layer of transposed convolution. This layer improves the quality of the generated image by strengthening important local features and suppressing unimportant features. The output dimension of this layer is (b, 32, 32, 256).

[0133] ⑥Final transposed convolution layer: The output of the local sparse attention layer then enters the final transposed convolution layer, which uses a 2×2 convolution kernel, a step size of 2, and only one convolution kernel to upsample the feature map to (b, 256, 256, 1).

[0134] ⑦Output layer: Finally, the feature map is nonlinearly transformed through an activation function (such as ReLU or tanh) and passes through the output layer to generate the final 256×256×1 image sample.

[0135] With this structure, the generator is able to generate images with rich details and high realism. At the same time, the introduction of the local sparse attention layer further improves the model's sensitivity to local features of the image and the generation quality.

[0136] Discriminator execution process:

[0137] ① Input layer: The input layer receives image data, and its size is (b, 256, 256, 1), where b represents the batch size and 256x256x1 represents the resolution and number of color channels of the image.

[0138] ② Convolutional layer: The next two layers are convolutional layers. The first layer uses a 3x3 convolution kernel with 16 convolution kernels and a stride of 1 to keep the image size unchanged. The second layer uses a 4x4 convolution kernel with 32 convolution kernels and a stride of 2 to reduce the spatial dimension of the image. The dimensions of the output feature maps are (b, 256, 256, 16) and (b, 128, 128, 32) respectively.

[0139] ③ Local sparse attention layer: After the second convolution layer, we introduce the local sparse attention layer, which enhances the model's sensitivity to local features of the image by strengthening important local features and suppressing unimportant features. The output feature map size of this layer is (b, 64, 64, 32).

[0140] ④ Further convolution layer: The output of the local sparse attention layer then passes through two more convolution layers with kernel sizes of 3x3 and 2x2, the number of kernels of 64 and 128, and a stride of 2 to further reduce the image size and the number of channels. The dimensions of the output feature maps are (b, 32, 32, 64) and (b, 16, 16, 128).

[0141] ⑤Flatten layer: The output of the sixth layer is flattened into a one-dimensional vector through the Flatten layer, ready for dense connection.

[0142] ⑥Dense layer: Finally, the flattened vector is input to the Dense layer, which calculates the probability that a given image belongs to the true category, as well as its distribution among different categories.

[0143] 2. Introducing local sparse attention layer

[0144] ① Define the local neighborhood: Determine the scope of locality. For example, you can choose an n×n window (such as 3×3 or 5×5), which means that you only focus on each pixel and its directly adjacent pixels.

[0145] ② Create a binary mask: For each pixel position (i, j) of the input feature map, create a binary mask M of the same size. Each element M in the mask pq Determined by its positional relationship with the center pixel (i, j):

[0146]

[0147] Here, (i,j) is the center pixel position, (p,q) is the surrounding pixel position, and n is the local range.

[0148] ③Use Information Flow Graphs (IFGs): IFGs are directed acyclic graphs used to simulate the flow of data in the attention layer. In the attention mechanism, each pixel or feature can be regarded as an information unit, and the nodes in IFGs represent the states of these information units at different processing stages. By designing IFGs, we ensure that information can flow effectively in the network even after sparse processing. This involves ensuring that each node in the graph is connected to at least one path, so that information can flow from the input layer to the output layer. A good IFG should have the "full information" property, which means that for any two nodes, there is at least one path connecting them in the graph, ensuring that information can flow to any location in the network.

[0149] ④ Calculate query (Q), key (K), and value (V) matrices: In the attention mechanism, the input feature matrix X is used to generate query (Q), key (K), and value (V) matrices, which are obtained by combining with the learnable weight matrix W Q， W K， W V The specific calculation is as follows:

[0150] The calculation formula of the query matrix Q is: Q = X*W Q ;

[0151] The calculation formula of the key matrix K is: K = X * W K ;

[0152] The calculation formula of the value matrix V is: V = X * W V ;

[0153] Here, W Q， W K， W V are the parameters of the model, which are learned and updated by the back-propagation algorithm during training. The size of the Q, K, V matrices depends on the size of the input feature matrix X and the size of the weight matrix.

[0154] ⑤ Apply sparse pattern mask: Apply the sparse pattern mask M to the Q and K matrices to reduce the number of pixels or features involved in the calculation:

[0155] Sparse query matrix Sparse Q The calculation formula is: Sparse Q = = Q⊙M;

[0156] Sparse key matrix Sparse K The calculation formula is: Sparse k =K⊙M;

[0157] Here, ⊙ represents the Hadamard product.

[0158] ⑥ Calculate attention weights: Use the modified sparse query and key matrices to calculate attention weights. First calculate their dot product, and then apply the softmax function to obtain normalized weights. This process can be expressed as:

[0159] Attention Weights=softmax(Sparse Q *Sparse K T )

[0160] Each element w of the attention weight matrix ij It represents the attention strength of the jth position to the ith position in the input feature matrix. The softmax function ensures that for each position i, its corresponding weight vector w i The elements of are summed to 1, which helps in the weighted summation in the next step.

[0161] ⑦ Weighted summation to obtain the output feature matrix: Use the obtained attention weights to perform weighted summation on the value matrix V to obtain the final output feature matrix. The calculation of this step can be expressed as:

[0162] Output=Attention Weights*V;

[0163] ⑧ Apply the ESA (Enumerate, Shift, Apply) framework: The main purpose of the ESA framework is to adapt the one-dimensional sparse pattern to the two-dimensional image structure, thereby maintaining the spatial relationship between pixels, especially in attention calculations.

[0164] Manhattan distance is a method of measuring the distance between two points on a grid. It is equal to the sum of the absolute differences between the two points on each coordinate axis. In two-dimensional space, if the coordinates of the two points are (x 1 ,y 1 ) and (x 2 ,y 2 ), then the Manhattan distance d between them is:

[0165] d=|x 2 -x 1 |+|y 2 -y 1 |

[0166] The ESA framework steps are:

[0167] First, we renumber the pixels in the image according to their Manhattan distance. This helps us determine the local neighborhood of each pixel, since the Manhattan distance of a pixel reflects its positional relationship in two-dimensional space.

[0168] Next, we adjust the sparsity pattern based on the enumeration result of the Manhattan distance. This means that we map the index of the one-dimensional sparse pattern into a two-dimensional space, ensuring that the sparse pattern correctly represents the local connectivity of the pixels in the image.

[0169] Finally, we apply the adjusted sparsity pattern to the attention computation. This involves using a binary mask M to selectively activate or disable certain computations in the attention mechanism, thereby reducing the amount of computation and maintaining locality.

[0170] Through the ESA framework, we can effectively implement the sparse attention mechanism while maintaining the 2D image data structure. This approach not only improves computational efficiency, but also enhances the model's ability to capture local features by considering the spatial relationship of pixels. Combined with the use of IFGs, we can further ensure the effective flow of information in the network, thereby optimizing the performance of the attention layer.

[0171] ⑨Optimization process: During the training process, the weight matrix W is updated through optimization algorithms such as back propagation and gradient descent Q ,W K ,W V and the sparse pattern mask M to minimize the loss function.

[0172] Through these steps, we design a local sparse attention layer, which exploits the 2D locality principle and information flow graph to optimize information flow and adapt to the 2D image structure. This design not only improves computational efficiency but also maintains the model's sensitivity to local features, thereby improving the performance of the generative model.

[0173] Embodiment 2

[0174] This embodiment discloses a classification system for class-unbalanced encrypted network traffic.

[0175] A classification system for class-imbalanced encrypted network traffic, including:

[0176] The preprocessing module is configured to: obtain the original class-imbalanced encrypted network traffic, perform preprocessing, and generate a real Markov image representing the characteristics of the network traffic;

[0177] The generation and discrimination module is configured to: build a CWGAN-GP network including a generator and a discriminator, input the real Markov image into the CWGAN-GP network, use the generator to generate the Markov image, and use the discriminator to classify the generated Markov image and the real Markov image, so as to classify the original category-unbalanced encrypted network traffic;

[0178] Among them, local sparse attention layers are introduced in both the generator and the discriminator to create a sparse pattern mask that is selective to the calculation in the attention mechanism. The calculation results of the query and key are modified based on the sparse pattern mask, and then the output feature matrix modified by the sparse pattern mask is calculated.

[0179] Embodiment 3

[0180] The purpose of this embodiment is to provide a computer-readable storage medium.

[0181] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the classification method for classifying category-unbalanced encrypted network traffic as described in Embodiment 1 of the present disclosure.

[0182] Embodiment 4

[0183] The purpose of this embodiment is to provide an electronic device.

[0184] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the classification method for category-unbalanced encrypted network traffic as described in Example 1 of the present disclosure are implemented.

[0185] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0186] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0187] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A classification method for class-imbalanced encrypted network traffic, characterized in that: The following steps are involved: Obtain the original class-imbalanced encrypted network traffic, perform preprocessing, and generate a real Markov image that characterizes the network traffic characteristics; Build a CWGAN-GP network including a generator and a discriminator, input the real Markov image into the CWGAN-GP network, use the generator to generate the Markov image, and use the discriminator to classify the generated Markov image and the real Markov image, so as to classify the original category-unbalanced encrypted network traffic; Among them, local sparse attention layers are introduced in both the generator and the discriminator to create a sparse pattern mask that is selective to the calculation in the attention mechanism. The calculation results of the query and key are modified based on the sparse pattern mask, and then the output feature matrix modified by the sparse pattern mask is calculated.

2. The classification method for class imbalanced encrypted network traffic according to claim 1, characterized in that: Generates realistic Markov graphs that characterize network traffic, including: Grouping the original class-unbalanced encrypted network traffic based on five key elements to form independent session files; the five key elements include source IP, destination IP, source port, destination port, and transport layer protocol used; Filter the grouped network data packets; The byte sequence in each session file is regarded as a Markov process, and the transition probability between bytes is estimated by analyzing the follow-up relationship between bytes; According to the calculated transition probabilities, a transition probability matrix representing the possibility of transition between bytes is generated; A Markov image is generated based on the transition probability matrix, and each element in the transition probability matrix represents a pixel point in the Markov image.

3. The classification method for class imbalanced encrypted network traffic according to claim 2, characterized in that: The calculation formula of the transition probability is: Where f(x,y) is the number of times byte x is followed by byte y. is the sum of f(x,y) over all possible subsequent bytes y; P(y|x) is the probability that byte x is followed by byte y.

4. The classification method for class imbalanced encrypted network traffic according to claim 1, characterized in that: The category label is introduced as an additional condition in the generator and discriminator of the CWGAN-GP network, and the corresponding category label and the real Markov image or the generated Markov image are used as conditional constraints.

5. The classification method for class imbalanced encrypted network traffic according to claim 4, characterized in that: During the training of the CWGAN-GP network, the gradient penalty method is used to penalize the gradient of the discriminator output: The loss function L(D) of the discriminator is defined as: Among them, the first It is the negative expected value of the probability that the discriminator D will judge the real sample x as real under given condition c; the second term is the expected value of the probability that the generated sample α is judged to be true under the same conditions; the last term is the gradient penalty used to smooth the decision boundary of the discriminator, where λ is a hyperparameter that controls the strength of the penalty; represents the mathematical expectation of the sample, c represents additional conditional information, P r and P g Represent the distribution of real data and generated data respectively; The loss function L(G) of the generator is defined as:

6. The classification method for class imbalanced encrypted network traffic according to claim 1, characterized in that: The local sparse attention layer is introduced in both the generator and the discriminator. The specific process is as follows: Based on each pixel and its immediate neighbors, a local neighborhood is defined; Based on the local neighborhood, a binary sparse pattern mask is created, where each element in the mask is determined by its positional relationship with the central pixel of the local neighborhood; Design information flow diagrams to ensure that information can flow effectively in the CWGAN-GP network; Based on the input feature matrix, generate the query matrix Q, key matrix K and value matrix V; Based on the Hadamard product calculation, the sparse pattern mask is applied to the query matrix Q and the key matrix K to obtain the sparse query matrix and the sparse key matrix respectively; Compute attention weights using sparse query matrix and sparse key matrix; Use the obtained attention weights to perform weighted summation on the value matrix V to obtain the final output feature matrix.

7. The classification method for class imbalanced encrypted network traffic according to claim 6, characterized in that: Also includes: Through the ESA framework, the one-dimensional sparse pattern is adjusted to adapt to the two-dimensional image structure, specifically: Renumber pixels according to the Manhattan distance of the Markov image pixels; Adjust the sparse pattern mask so that it matches the Manhattan distance based enumeration; Apply the adjusted sparse pattern mask to the attention computation; Also includes: During training, the weight matrix and the sparsity pattern mask are updated to minimize the loss function.

8. A classification system for class-imbalanced encrypted network traffic, characterized in that: include: The preprocessing module is configured to: obtain the original class-imbalanced encrypted network traffic, perform preprocessing, and generate a real Markov image representing the characteristics of the network traffic; The generation and discrimination module is configured to: build a CWGAN-GP network including a generator and a discriminator, input the real Markov image into the CWGAN-GP network, use the generator to generate the Markov image, and use the discriminator to classify the generated Markov image and the real Markov image, so as to classify the original category-unbalanced encrypted network traffic; Among them, local sparse attention layers are introduced in both the generator and the discriminator to create a sparse pattern mask that is selective to the calculation in the attention mechanism. The calculation results of the query and key are modified based on the sparse pattern mask, and then the output feature matrix modified by the sparse pattern mask is calculated.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the classification method for classifying category-unbalanced encrypted network traffic as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the classification method for classifying category-unbalanced encrypted network traffic as described in any one of claims 1 to 7 are implemented.

Images