A method and system for identifying encrypted traffic network behavior by adding a self-attention mechanism and maximizing the additive angle of the interval layer.

Abstract

This invention relates to a method and system for identifying encrypted traffic network behavior by adding a self-attention mechanism and an additive angle maximization interval layer. The method includes: preprocessing and embedding representations of traffic generated by encrypted application network behavior; enhancing the semantics of the embedded representations of traffic using a self-attention mechanism; constructing an encrypted application behavior traffic identification model, and extracting deep features of the traffic based on the results obtained from the self-attention mechanism; setting an additive angle maximization interval layer in the encrypted application behavior traffic identification model to maximize the interval between different categories of traffic features, increasing the discriminability between different category vectors; and identifying encrypted traffic network behavior through the encrypted application behavior traffic identification model. This invention solves the problem of high misclassification rates caused by existing technologies failing to effectively represent traffic features and maximize the distance between traffic feature vectors generated by different network behaviors when identifying encrypted traffic network behavior.

Description

Technical Field

[0001] This invention belongs to the field of network measurement and behavior analysis, specifically relating to a method for identifying encrypted traffic network behavior based on self-attention mechanism embedded representation and additive angle maximization interval layer optimization representation. Background Technology

[0002] With the development of mobile internet information technology and services, the types and functions of internet applications are becoming increasingly diverse. Mobile internet has become an important part of people's lives and work. Users engage in a wide variety of online behaviors, which can be broadly categorized into five common types: FTP file transfer, instant messaging, email, online video, and web browsing.

[0003] Encrypted traffic network behavior analysis is a method of identifying specific network behaviors by analyzing the communication flow of web applications and extracting network communication characteristics. Analyzing mobile application network behavior provides a more realistic understanding of the network behavior characteristics and information dissemination patterns of mobile applications, which helps in the more refined planning, management, and security of the mobile internet. Encrypted network behavior can be effectively identified using network traffic identification methods.

[0004] Existing network traffic identification methods can be categorized into traditional methods, machine learning-based methods, and deep learning-based methods. Traditional methods include port-based and payload-based methods. Problems with these methods include the performance degradation of port-based methods due to the widespread use of network port translation technology, and the failure of payload-based methods when dealing with encrypted traffic. However, with the increasing trend of traffic encryption, more and more traffic is being encrypted. The characteristics of encrypted traffic may change, posing challenges to encrypted traffic analysis and identification. Machine learning and deep learning-based methods for encrypted traffic identification have emerged. Machine learning-based traffic classification can be divided into unsupervised methods (k-means, k-nearest neighbors) and supervised methods (logistic regression, SVM). These methods rely on manual feature extraction and selection, resulting in low classification accuracy. Deep learning-based methods have been applied to the field of encrypted traffic classification with the development of deep learning technology.

[0005] Network traffic classification technology, as one of the fundamental technologies for enhancing network controllability, not only helps network operators provide better service quality but also enables effective supervision and management of the network, ensuring network security. Accurate network traffic classification contributes to intelligent network operation and maintenance, network quality service assurance, and network security maintenance. It is of great significance for providing differentiated network service quality, network intrusion detection, traffic shaping, and resource allocation.

[0006] Therefore, in order to strengthen network security management and provide differentiated network service quality, an effective method for encrypted traffic network behavior analysis is needed to discover the characteristics of traffic generated by different behaviors, increase the distinguishability between different network behaviors, and achieve the identification of different network behaviors. Summary of the Invention

[0007] The technical problem this invention aims to solve is to propose a method and system for identifying encrypted traffic network behavior. This invention extracts long sequence information and message type information from traffic data packets, utilizes embedding representation, and combines it with deep learning algorithms to address the problem that existing technologies fail to effectively represent traffic characteristics and maximize the distance between traffic feature vectors generated by different network behaviors when identifying encrypted traffic network behavior, thus leading to a high misclassification rate.

[0008] The technical solution adopted in this invention is as follows:

[0009] A method for identifying encrypted traffic network behavior by adding a self-attention mechanism and an additive angle maximization gap layer includes the following steps:

[0010] Preprocessing and embedding representations of traffic generated by network behavior of encrypted applications;

[0011] Enhance the semantics of the embedded representation of traffic by utilizing a self-attention mechanism;

[0012] Construct a traffic identification model for encrypted application behavior, and extract deep features of the traffic based on the results obtained from the self-attention mechanism.

[0013] In the encrypted application behavior traffic identification model, an additive angle maximization interval layer is set to maximize the interval between different types of traffic features and increase the discriminability between different type vectors.

[0014] The encrypted application behavior traffic identification model is used to identify encrypted traffic network behavior.

[0015] Furthermore, the preprocessing and embedding representation of traffic generated by the network behavior of encrypted applications includes:

[0016] Traffic preprocessing includes: filtering out stray traffic, including traffic without payload and retransmitted traffic; segmenting network session packets according to the five-tuple information of the traffic, extracting the packet length sequence of the flow and the message type corresponding to each packet; removing flow sequences with fewer than 5 payloads, and truncating the first N packet payload sequences to obtain a model input of uniform length.

[0017] Each value in the traffic input sequence, which consists of packet length vector, message type vector, and location information, is embedded and represented. The packet length vector stores the length and direction information of the first n data packets, the message type vector stores the message type information of the first n data packets, and the location vector contains the index information of each packet length in the byte vector.

[0018] Furthermore, the self-attention mechanism enables the neural network to notice the correlation between different parts of the entire input and the differences in the importance of different parts; the self-attention mechanism includes three key parts: Q, K, and V. First, Q and K calculate the correlation between every two input vectors, that is, calculate the Attention value. Then, the obtained Attention value is activated to obtain A'. Finally, A' and V are used to calculate the output vector b of the self-attention layer corresponding to each input vector a, which is input into the subsequent neural network for feature extraction.

[0019] Furthermore, the method of extracting deep features of traffic using the encrypted application behavior traffic identification model adopts a deep neural network structure based on convolutional neural networks. Two one-dimensional CNNs are applied to the same input through two CNN networks, and their outputs are merged together to obtain the final output, which is the extracted deep features.

[0020] Furthermore, the additive angle-maximization interval layer employs additive angle-maximization loss to normalize the feature vector and add an angle interval, thereby maximizing the classification boundary in the angle space and improving intra-class separability and inter-class differences.

[0021] Furthermore, the identification of encrypted traffic network behavior through the encrypted application behavior traffic identification model includes: obtaining the feature vector of the traffic to be tested, and using cosine similarity to calculate the similarity between the feature vectors of the training traffic and the traffic to be tested, thereby determining the category of the traffic to be tested.

[0022] A cryptographic traffic network behavior recognition system that incorporates a self-attention mechanism and an additive angle maximization interval layer, comprising:

[0023] The traffic preprocessing and representation module is used to preprocess and embed representations of traffic generated by the network behavior of encrypted applications;

[0024] The traffic representation semantic enhancement module is used to enhance the semantics of the embedded representation of traffic by leveraging a self-attention mechanism;

[0025] The traffic deep feature extraction module is used to build an encrypted application behavior traffic identification model. Based on the results obtained by the self-attention mechanism, the deep features of the traffic are extracted using the encrypted application behavior traffic identification model.

[0026] The traffic feature optimization representation module is used to set an additive angle maximization interval layer in the encrypted application behavior traffic identification model to maximize the interval between different types of traffic features and increase the discriminability between different type vectors.

[0027] The traffic identification module is used to identify encrypted traffic network behavior through an encrypted application behavior traffic identification model.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] This invention uses a data packet length sequence and a message type sequence as input. Compared to using only the data packet length sequence, the message type reflects the state information of the traffic and serves as supplementary information to the data packet length sequence. By combining the data packet length sequence and the message type sequence, it can capture the characteristic differences in the length and direction of the traffic and describe the state information of the traffic.

[0030] This invention employs an embedding representation based on a self-attention mechanism to represent the input traffic information. Compared to direct input, the self-attention mechanism reflects the relationship and importance of different parts within the traffic, thereby enhancing the semantic information of the traffic.

[0031] This invention adds an angle interval maximization layer to the neural network, which optimizes the feature representation of flow. Compared with a neural network without an angle interval maximization layer, the angle interval maximization layer maximizes the interval between feature vectors, thereby increasing the discriminability between vectors of different categories. Attached Figure Description

[0032] Figure 1 A traffic identification model for encrypted application behavior.

[0033] Figure 2 This is a schematic diagram of the modules of an encrypted application behavior traffic identification system. Detailed Implementation

[0034] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0035] The present invention provides a method for identifying encrypted traffic network behavior by adding a self-attention mechanism and an additive angle maximization interval layer, comprising the following steps:

[0036] S1: Traffic preprocessing and embedded representation;

[0037] S2: Enhanced semantic information representation of flow;

[0038] S3: Traffic depth feature extraction;

[0039] S4: Maximize the angle interval;

[0040] S5: Traffic Identification.

[0041] Step S1: Preprocess and embed representation of the traffic generated by the network behavior of encrypted applications.

[0042] S1-1: First, the traffic is preprocessed to filter out unwanted streams, including no-load traffic and retransmissions. Using the Flowcontainer tool, network session packets are segmented according to the five-tuple information of the traffic (source IP, destination IP, source port, destination port, transport layer protocol), extracting the packet length sequence of the flow and the message type corresponding to each packet. Next, flow sequences with fewer than 5 payloads are removed, and the first N packet payload sequences are truncated to obtain a model input of uniform length. N is determined based on the average length of all flow sequences.

[0043] S1-2: Using the `nn.Embedding` function in the PyTorch deep learning framework, each value in the input sequence of the flow, consisting of packet length vector, message type vector, and location information, is embedded. The packet length vector stores the length and direction information of the first N packets, the message type vector stores the message type information of the first N packets, and the location vector contains the index information of each packet length in the byte vector. This embedding representation enhances the semantic representation of each flow.

[0044] Step S2: Enhance the semantics of the flow representation using a self-attention mechanism, allowing the neural network to notice the correlations and differences in importance between different parts of the input. The self-attention mechanism mainly consists of three key parts: Query(Q), Key(K), and Value(V), where Query(Q) = Key(K) = Value(V) = the output vector from steps S1-2. First, Q and K calculate the correlation between every two input vectors, i.e., calculate the Attention value, typically using a dot product. Then, apply softmax or ReLU activation operations to the obtained Attention value to obtain A'. Finally, use A' and V to calculate the output vector b of the self-attention layer corresponding to each input vector a, and input it into the subsequent neural network for further feature extraction.

[0045] Step S3: Design a deep neural network to extract the depth features of the traffic. A deep neural network structure based on convolutional neural networks is adopted. Two CNN networks are designed, each with a one-dimensional CNN applied to the same input, and their outputs are merged to obtain the final output.

[0046] feature1 = 1D-CNN1(input)

[0047] feature2 = 1D-CNN2(input)

[0048] output=[feature1,feature2]

[0049] Here, input represents the output of step S2, feature1 and feature2 represent the extracted features respectively, 1D-CNN1 and 1D-CNN2 are two one-dimensional convolutional neural networks, and output represents the feature vector finally output by step S3.

[0050] One-dimensional convolutional neural networks (CNNs) are commonly used for extracting sequence features. The role of a convolutional layer is to extract features from a local region. After the convolutional layer, a non-linear activation function is applied to obtain a feature vector. While convolutional layers can significantly reduce the number of connections in the network, the number of neurons in the feature map group is not significantly reduced. If a classifier is followed, the input dimensionality of the classifier remains high, easily leading to overfitting. To address this issue, a max-pooling layer is added after the convolutional layer, thereby reducing the feature dimensionality and avoiding overfitting. By designing two different sizes of convolutional kernels to extract features at different depths, and finally concatenating the feature vectors obtained from the two networks, features from different parts of the traffic can be extracted, thus enriching the feature representation of the traffic and aiding in subsequent traffic classification.

[0051] Step S4: Further optimize the feature vectors obtained in Step S3. An additive angle-maximization margin layer is added after the last layer of the deep learning network architecture in Step S3. This maximizes the angular margin between the class and the feature vector, further processing the traffic feature representation obtained in S3 to make it more class-discriminative. The additive angle-maximization loss is an improvement on the SoftMax loss. Specifically, it normalizes the feature vectors and adds an angular margin, maximizing the classification boundary in the angular space, thereby improving intra-class separability and inter-class diversity.

[0052] The original softmax loss, derived from the formula combining the softmax function and the cross-entropy loss function, can be expressed as follows:

[0053]

[0054] Where N represents the amount of data processed in batches, n represents the number of categories, j represents the j-th column of the W matrix, and x i This represents the feature representation of the i-th sample, which belongs to the y-th sample. i kind, Represents category y i The weight vector; This is a bias term, and to simplify the calculation process, it is set to 0.

[0055] S4-1: Normalize the feature representation and weight vector obtained in S3.

[0056] Will Substituting into the above formula, where the weight W j The j-th row, i.e. Using L2 regularization, fix each weight ||W j || = 1 and feature ||x i || = 1. Therefore, we get: Where, θ j Weight W j and feature x i The angle between them, specifically, Weight W j Each line in (i.e., category y) i (weight vector) and feature x i The angle between them. At this point, L can be... SoftMax Rewritten as follows:

[0057]

[0058] Here, s is the scaling factor, typically set to 30, but can be adjusted according to the actual situation. The feature vector output by the network is scaled (re-scaled) by the scaling factor s.

[0059] S4-2: In the feature representation x i and weight vector An additional angular interval m (usually 0.5, but can be adjusted according to the actual situation) is added between L′. The expression for the additive angle maximization loss is further obtained from L′ as follows:

[0060]

[0061] Step S5: Utilize the deep neural network model from Step S3, combined with the additive angle maximization layer from Step S4, to identify behavioral traffic generated by the encrypted application. The additive angle maximization layer is only used during the training phase (training traffic); it is not needed for test traffic. Test samples first undergo steps S1, S2, and S3 to obtain feature vectors for the traffic. Cosine similarity is used to calculate the similarity between the feature vectors of the training and test traffic, thereby determining the category of behavioral traffic in the test set. Cosine similarity uses the cosine of the angle between two vectors in vector space as a measure of the difference between two individuals. When the angle between two vectors approaches 0, the closer the two vectors are, the smaller the difference. The closer the cosine value is to 1, the more similar the traffic is. The most similar traffic category is selected as the predicted label for the samples in the test set. Specifically, as shown... Figure 1As shown, classification can be achieved using the Softmax function. The formula for calculating cosine similarity is as follows:

[0062]

[0063] Where Ai represents the i-th class A vector, Bi represents the i-th class B vector, and n represents the dimensions of the class A and class B vectors.

[0064] Another embodiment of the present invention provides an encrypted traffic network behavior recognition system that incorporates a self-attention mechanism and an additive angle maximization interval layer, such as... Figure 2 As shown, it includes the following modules:

[0065] The traffic preprocessing and representation module preprocesses the input traffic and performs embedded representation.

[0066] The traffic representation semantic enhancement module adds a self-attention mechanism to the embedded representation vector to learn the internal relationships of traffic and pay attention to different degrees of importance.

[0067] The traffic deep feature extraction module constructs an encrypted application behavior traffic identification model and extracts the deep features of the traffic through convolution operations.

[0068] The traffic feature optimization representation module increases the discriminativeness between different category vectors by adding an additive angle maximization interval layer to maximize the interval between different categories of traffic features.

[0069] The traffic identification module identifies behavioral traffic through an encrypted application behavior traffic identification model.

[0070] Through the above technical solution, the present invention provides an effective method and system for identifying encrypted application behavior traffic.

[0071] Taking the behavioral traffic generated by encrypted applications as the object, step S1 first preprocesses the traffic, filtering out irrelevant traffic and extracting the packet length sequence and message type of the stream using a 5-tuple approach. Furthermore, the resulting stream is divided into training, validation, and test sets using a cross-validation method. Then, the extracted traffic information is embedded and represented using the deep learning framework PyTorch.

[0072] The semantic representation of the flow is enhanced as described in step S2. A self-attention mechanism is added to the embedding representation obtained in step S1 to capture the correlation information within each flow and to note the importance of different parts.

[0073] Following step S3, establish an encrypted application behavior traffic identification model and extract deep features from the traffic. Construct two model structures based on convolutional neural networks and connect the outputs of the two networks as the final extracted deep feature vector.

[0074] The feature vectors obtained in S3 are further optimized according to step S4. The angle between the different categories of feature vectors obtained in different steps S3 is maximized. First, the feature vectors and weight vectors are normalized. Second, an additional angle is added between the feature vectors and weight vectors.

[0075] The encrypted application behavior is identified using the model obtained in steps S3 and S4, as described in step S5. The test set generated in step S1 is then used in step S2 to test the model proposed in step S3.

[0076] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0077] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0078] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A method for identifying encrypted traffic network behavior by adding a self-attention mechanism and an additive angle maximization interval layer, characterized in that, Includes the following steps: Preprocessing and embedding representations of traffic generated by network behavior of encrypted applications; Enhance the semantics of the embedded representation of traffic by utilizing a self-attention mechanism; Construct a traffic identification model for encrypted application behavior, and extract deep features of the traffic based on the results obtained from the self-attention mechanism. The encrypted application behavior traffic recognition model adopts a deep neural network structure based on convolutional neural networks. It applies two one-dimensional CNNs to the same input and merges their outputs to obtain the final output, which is the extracted deep features. In the encrypted application behavior traffic identification model, an additive angle maximization interval layer is set to maximize the interval between different types of traffic features and increase the discriminability between different type vectors. The encrypted application behavior traffic identification model is used to identify encrypted traffic network behavior.

2. The method according to claim 1, characterized in that, The preprocessing and embedding representation of traffic generated by encrypted application network behavior includes: Traffic preprocessing includes: filtering out stray traffic, including traffic without payload and retransmitted traffic; segmenting network session packets according to the five-tuple information of the traffic, extracting the packet length sequence of the flow and the message type corresponding to each packet; removing flow sequences with fewer than 5 payloads, and truncating the first N packet payload sequences to obtain a model input of uniform length. Each value in the traffic input sequence, which consists of packet length vector, message type vector, and location information, is embedded and represented. The packet length vector stores the length and direction information of the first n data packets, the message type vector stores the message type information of the first n data packets, and the location vector contains the index information of each packet length in the byte vector.

3. The method according to claim 1, characterized in that, The self-attention mechanism enables the neural network to notice the correlation between different parts of the entire input and the differences in the importance of different parts. The self-attention mechanism includes three key parts: Q, K, and V. First, Q and K calculate the correlation between every two input vectors, that is, calculate the Attention value. Then, the obtained Attention value is activated to obtain A'. Finally, A' and V are used to calculate the output vector b of the self-attention layer corresponding to each input vector a, which is then input into the subsequent neural network for feature extraction.

4. The method according to claim 1, characterized in that, The additive angle-maximization margin layer uses additive angle-maximization loss to normalize the feature vector and add an angle margin, thereby maximizing the classification boundary in the angle space and improving intra-class separability and inter-class differences.

5. The method according to claim 1, characterized in that, The additive angle maximization loss is calculated using the following formula: Where N represents the amount of data processed in batches; s is the scaling factor; m is the additional angular interval added between the feature representation and the weight vector; θ j The weight vector W j and eigenvector x i The angle between them The weight vector W j Each line in and eigenvector x i The angle between them.

6. The method according to claim 1, characterized in that, The method of identifying encrypted traffic network behavior through the encrypted application behavior traffic identification model includes: obtaining the feature vector of the traffic to be tested, and using cosine similarity to calculate the similarity between the feature vectors of the training traffic and the traffic to be tested, thereby determining the category of the traffic to be tested.

7. A cryptographic traffic network behavior recognition system that incorporates a self-attention mechanism and an additive angle maximization interval layer, characterized in that, include: The traffic preprocessing and representation module is used to preprocess and embed representations of traffic generated by the network behavior of encrypted applications; The traffic representation semantic enhancement module is used to enhance the semantics of the embedded representation of traffic by leveraging a self-attention mechanism; The traffic deep feature extraction module is used to build an encrypted application behavior traffic identification model. Based on the results obtained by the self-attention mechanism, the deep features of the traffic are extracted using the encrypted application behavior traffic identification model. The encrypted application behavior traffic recognition model adopts a deep neural network structure based on convolutional neural networks. It applies two one-dimensional CNNs to the same input and merges their outputs to obtain the final output, which is the extracted deep features. The traffic feature optimization representation module is used to set an additive angle maximization interval layer in the encrypted application behavior traffic identification model to maximize the interval between different types of traffic features and increase the discriminability between different type vectors. The traffic identification module is used to identify encrypted traffic network behavior through an encrypted application behavior traffic identification model.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 6.

Images