Network encryption traffic classification method and system based on data packet header
By preprocessing and feature recalibration of network traffic data by preprocessing packet headers and combining with LSTM networks, the problem of low interpretability in network traffic classification in the prior art is solved, and higher classification performance and model interpretability are achieved.
Patent Information
- Application Number
- CN202410850359.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-06-28
AI Technical Summary
The prior art fails to effectively utilize the relative independence and protocol fields between data packets in network traffic classification and anomaly detection, resulting in high model performance but low interpretability, making it difficult to optimize and improve.
Using preprocessing methods based on data packet headers, the original traffic data is converted into structured data, and encrypted traffic is classified through feature recalibration and long and short-term memory network (LSTM). The specific steps include re-length, alignment, and protocol field segmentation of data packet headers. and feature recalibration to improve the model's interpretability and classification performance of data packet header features.
The classification performance and interpretability of the model are improved, so that each feature of the data packet header and the weight parameters of the model can be corresponding, supporting subsequent optimization and improvement, and showing excellent encrypted traffic classification effects in the experiment.
Smart Images

Figure CN118410404B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network security, and in particular to a method and system for classifying network encrypted traffic based on data packet headers. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] There is a large amount of traffic data in cyberspace. Existing technologies usually use deep learning models to detect anomalies and handle them in a timely manner when the traffic data has not reached the host user, thereby handling abnormal events at the source, maintaining the security of cyberspace, and preventing websites, systems or servers from being occupied by malicious traffic, causing system crashes and affecting normal access.
[0004] Currently, the deep learning model is used for network traffic classification or abnormal traffic detection technology, which mostly adopts a session-based approach. The first several bytes of the session are taken as the input of the deep learning model, without considering the relative independence of the data packets in a session, nor distinguishing between the header and the payload, and even less considering the protocol fields with specific meanings in the header of the network traffic data packet. Although the deep learning model has achieved good performance, it has low interpretability, which is not conducive to subsequent optimization and improvement. Summary of the invention
[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a network encrypted traffic classification method and system based on data packet header, preprocesses the characteristics of the original traffic data, and proposes feature recalibration and long short-term memory network to classify the encrypted traffic.
[0006] In order to achieve the above object, the present invention adopts the following technical solution:
[0007] A first aspect of the present invention provides a method for classifying network encrypted traffic based on a data packet header, comprising the following steps:
[0008] Obtain the original network traffic data, separate the header information of the data packet, and convert it into structured data after preprocessing;
[0009] The structured data obtained is based on the trained encrypted traffic classification model to obtain the packet classification result;
[0010] Among them, the specific process of obtaining the data packet classification result based on the trained encrypted traffic classification model is: extract the importance of each feature in the structured data, multiply different features by the corresponding importance, obtain the structured data after feature recalibration, and obtain the classification result based on the structured data after feature recalibration.
[0011] Furthermore, the original network traffic is obtained and the header information of the data packet is separated, specifically: the original traffic of the network node is extracted based on the packet capture tool, outputted as a file of a set format, and the header information of the data packet is extracted.
[0012] Further, preprocessing is specifically: padding and alignment based on the header information of the data packet, converting it into structured data based on bits, bytes or protocol fields, and normalizing it.
[0013] Furthermore, the lengthening is as follows: 0 and 1 are two normal states of the original traffic data, and -1 indicates a state where the data does not exist.
[0014] Furthermore, alignment is specifically: aligning the protocol fields of the data packet header to obtain a data packet header of a uniform length.
[0015] Furthermore, after the header information of the data packet is converted into the protocol field, the protocol field is segmented, and the length of each field after segmentation is no more than 8 bits.
[0016] Furthermore, the encrypted traffic classification model extracts the importance of each feature in the structured data. Specifically, the structured data is processed by dimensionality reduction to obtain the correlation between the features, and then the weight of each feature is determined by dimensionality increase. After the number of features is restored to the original dimension, the importance of each feature is obtained.
[0017] Furthermore, the structured data is reduced in dimension through a fully connected layer, and then increased in dimension through another fully connected layer to obtain parameters of the same dimension as the feature data. After passing through an activation function, the parameters are restricted to a set range to obtain the weight of each feature, and the feature-recalibrated structured data is obtained through a multiplication operation.
[0018] Furthermore, classification results are obtained based on the structured data after feature recalibration. Specifically, three types of structured data based on bit, byte and protocol fields after feature recalibration are respectively input into the LSTM network for classification.
[0019] A second aspect of the present invention provides a system for implementing the above method, comprising:
[0020] The data preprocessing module is configured to: obtain the original network traffic, separate the header information of the data packet, and convert it into structured data after preprocessing;
[0021] The traffic classification module is configured to: obtain the data packet classification result based on the obtained structured data and the trained encrypted traffic classification model;
[0022] Among them, the specific process of obtaining the data packet classification result based on the trained encrypted traffic classification model is to extract the importance of each feature in the structured data, multiply different features by the corresponding importance, obtain the structured data after feature recalibration, and obtain the classification result based on the structured data after feature recalibration.
[0023] A third aspect of the present invention provides a computer-readable storage medium.
[0024] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned method for classifying network encrypted traffic based on data packet headers.
[0025] A fourth aspect of the present invention provides a computer device.
[0026] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the network encryption traffic classification method based on data packet header as described above are implemented.
[0027] Compared with the prior art, one or more of the above technical solutions have the following beneficial effects:
[0028] Based on the header data of the data packet, it is padded and aligned, and converted into structured data to increase interpretability. It uses three representation forms: bit, byte and protocol field, and normalizes it. The structured data is processed using feature recalibration, and the structured data after feature recalibration is input into the LSTM network. Each feature of the data packet header and the weight parameter of the model can be matched, which can not only improve the classification performance of the model but also facilitate subsequent optimization and improvement. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0030] Figure 1 It is a schematic diagram of a flow chart of preprocessing data in a traffic classification process provided by one or more embodiments of the present invention;
[0031] Figure 2 is a schematic diagram of a Rew module in a traffic classification process provided by one or more embodiments of the present invention;
[0032] Figure 3 is a schematic diagram of the overall flow of traffic classification provided by one or more embodiments of the present invention;
[0033] Figure 4It is a schematic diagram of experimental results of the traffic classification method provided by one or more embodiments of the present invention. DETAILED DESCRIPTION
[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0035] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0036] Network traffic classification plays a very important role in quality of service (QoS) control, malware / Trojan detection, network attack identification and network intrusion detection. However, current network traffic classification or abnormal traffic detection technologies mostly adopt a session-based approach, taking the first several bytes of a session as the input of the deep learning model, without considering the relative independence between data packets in a session, nor distinguishing between the header and the payload, and even less considering the protocol fields with specific meanings in the header of the network traffic data packet. Even if the model achieves good performance, it is unknown why, and the interpretability is low.
[0037] Therefore, the following embodiments provide a network encrypted traffic classification method and system based on data packet headers, preprocess the original traffic data according to its characteristics, and propose a Rew-LSTM (Reweight-Long Short Term Memory) method and system to classify encrypted traffic.
[0038] Embodiment 1:
[0039] like Figure 1-Figure 4 As shown, the network encryption traffic classification method based on the data packet header includes the following steps:
[0040] Obtain the original network traffic data, separate the header information of the data packet, and convert it into structured data after preprocessing;
[0041] The structured data obtained is based on the trained encrypted traffic classification model to obtain the packet classification result;
[0042] Among them, the specific process of obtaining the data packet classification result based on the trained encrypted traffic classification model is to extract the importance of each feature in the structured data, multiply different features by the corresponding importance, obtain the structured data after feature recalibration, and obtain the classification result based on the structured data after feature recalibration.
[0043] Specific:
[0044] This embodiment uses the method applied to a server as an example. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.
[0045] In this embodiment, the method includes the following steps:
[0046] Step 1: Get the original network traffic, use the Wireshark packet capture tool to directly extract the original traffic of the set network node, and directly output it to the pcap file format (Packet Capture, a common network data packet capture format);
[0047] Step 2: Preprocess the network traffic data, such as Figure 1 As shown, the details are as follows:
[0048] The nprint tool is used to extract the header of the data packet and complete the data lengthening and alignment of the header. Then, the python script is used to convert it into structured data based on bits, bytes or protocol fields.
[0049] When padding, -1 is used for padding, which is different from the commonly used zero padding. Here, 0 and 1 are considered to be two normal states of the original traffic data, and -1 indicates a third state, that is, data does not exist. For example, if a session uses the TCP protocol and the IP protocol but does not use the UDP protocol, and most other traffic sessions use the UDP protocol, this embodiment pads the traffic data packet header in the session, that is, adds the UDP protocol field with -1 data, thereby unifying the data packet header.
[0050] Alignment means aligning the protocol field of the data packet header. After the data packet header is padded, the length of the data packet header is unified, and then the protocol message of the data packet header is aligned, for example, 1-m bits are IP protocol messages, mn bits are TCP protocol, np bits are UDP protocol, etc.
[0051] In this embodiment, three types of structured data are considered, namely: structured data based on bits, bytes, and protocol fields.
[0052] Bit-based structured data: The original traffic data is binary data, that is, data represented by 0 and 1. After the packet header is padded, there are three representations: 0, 1, and -1, and the total number of features is 1088.
[0053] Byte-based structured data: Generally, when processing raw traffic data, the bit representation is converted to byte representation for two reasons: first, the two types of data are reversible, and second, it can reduce the amount of data and improve learning efficiency. After conversion to bytes, the number of features is 136.
[0054] Structured data based on protocol fields: Since each field of the protocol is given a specific meaning, each protocol field is considered to be a feature here. It should be noted that the flag bit in the protocol field is still processed bit by bit. Since the optional part in the data packet header may contain more information about traffic anomalies, the optional part in the protocol field is also processed bit by bit here. In addition, since the computer's floating point storage accuracy is 8 decimal places, this embodiment divides the longer protocol fields so that the length of each field is less than 8 bits.
[0055] Step 3: Perform feature normalization on bit and byte-based structured data, and perform feature normalization and global normalization on protocol field-based structured data.
[0056] Perform column normalization on structured data, that is, feature normalization, as shown in Formula 1:
[0057] (1)
[0058] in is the feature column.
[0059] The normalization method can be replaced by a global normalization method. In particular, for the segmented protocol field, the formula is:
[0060] , (2)
[0061] in is the feature matrix, The number of bits of data in the feature column. For example, in byte-based structured data, one byte is equal to 8 bits, so ; In the structured data based on the protocol field, after data segmentation, . is the sequence number of the feature column. is the base, in binary, .
[0062] Since the number of bits of bit data and byte data is fixed, the traditional column normalization method and the global normalization method are almost the same in operation and are not distinguished here.
[0063] Step 4: Input the structured data based on bit, byte and protocol field into the Rew module for feature recalibration.
[0064] The Rew module is a feature-based attention mechanism module, also known as a feature recalibration module. Figure 2 As shown, it is possible to transform the dimensions of the data, learn the importance of different features, and then enhance important features and suppress useless features.
[0065] For example, when the input data is a bit, byte or protocol field, different bits, bytes or protocol fields have different importance. For example, the characteristic data padded with -1 is unimportant, and the optional fields in the packet header protocol message may be important. In this case, the Rew module will suppress the padded data field and enhance the optional field.
[0066] exist Figure 2 In the process of dimensionality reduction, for structured data, such as one-dimensional data composed of n-dimensional features, that is, X = [x1, x2,…,xn], the correlation between features is explicitly modeled and the amount of calculation is reduced. Then, the weight of each feature is obtained by training the parameter w. Therefore, this step is a dimensionality increase operation, which restores the number of features to n dimensions.
[0067] After this step, the importance w of each feature is obtained, and then it is multiplied by the feature data X input into this module to obtain the structured data after feature recalibration .
[0068] In addition, it should be noted that when performing dimensionality reduction, in order to better model the correlation between features, there is a hyperparameter k to control the amount of dimensionality reduction.
[0069] from Figure 3 As can be seen in the figure, in the Rew module, the feature data is first directly reduced in dimension through the fully connected layer. Then the nonlinearity of the module is increased through the Relu layer, and then the dimension is increased again through a fully connected layer to obtain parameters with the same dimension as the feature data. Then, the parameters are restricted to [0, 1] through the Sigmoid function to obtain the weight of each feature, and then the structured data after feature recalibration is obtained through the multiplication operation.
[0070] Step 5: Input the structured data based on bit, byte, and protocol field after feature recalibration into the LSTM network for classification.
[0071] like Figure 3 As shown in the figure, the LSTM network consists of an LSTM layer, a fully connected layer, a Relu layer and a softmax layer.
[0072] The above method is based on the header data of the data packet and performs padding and alignment. It converts it into structured data, uses three representation forms: bit, byte and protocol field, normalizes it, uses the Rew module (feature recalibration module), and then inputs the data into the LSTM network, which increases the interpretability and improves the classification performance of the model.
[0073] In this embodiment, a comparative experiment was also carried out, and the results are as follows: Figure 4 The evaluation index is the macro-average F1 value, which considers all categories to be equally important and is not affected by the big data sample category. It reflects the average performance of the model for all categories.
[0074] from Figure 4 As can be seen in the figure, the classification method provided in this example has achieved good results in the overall detection rate, proving that the classification method provided in this example is effective in the classification of encrypted traffic. In different data forms (bit data, byte data, and field_split data), the classification performance of the Rew-LSTM model is better than that of the LSTM model, proving the effectiveness of the Rew module. Both models have achieved the best results on bit-based structured data, proving that bit-based structured data is the most effective. In the LSTM model, after using the global normalization method for field_split data, the Macro-F1 value has been significantly improved, reaching or even exceeding the Rew-LSTM model under the same conditions, proving the effectiveness of the global normalization method.
[0075] Embodiment 2:
[0076] A system for implementing the above method comprises:
[0077] The data preprocessing module is configured to: obtain the original network traffic, separate the header information of the data packet, and convert it into structured data after preprocessing;
[0078] The traffic classification module is configured to: obtain the data packet classification result based on the obtained structured data and the trained encrypted traffic classification model;
[0079] Among them, the specific process of obtaining the data packet classification result based on the trained encrypted traffic classification model is to extract the importance of each feature in the structured data, multiply different features by the corresponding importance, obtain the structured data after feature recalibration, and obtain the classification result based on the structured data after feature recalibration.
[0080] The above system is based on the header data of the data packet and performs padding and alignment. By converting it into structured data, it increases interpretability, uses three representation forms of bits, bytes and protocol fields for normalization, uses feature recalibration to process structured data, and inputs the structured data after feature recalibration into the LSTM network, in which each feature of the data packet header and the weight parameter of the model can be matched, which can not only improve the classification performance of the model but also facilitate subsequent optimization and improvement.
[0081] Embodiment three:
[0082] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the network encryption traffic classification method based on the data packet header as described in the above-mentioned embodiment 1 are implemented.
[0083] Embodiment 4:
[0084] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for classifying network encrypted traffic based on data packet headers as described in the first embodiment above are implemented.
[0085] The steps or networks involved in the above embodiments 2 to 4 correspond to those in embodiment 1. For the specific implementation, please refer to the relevant description of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.
[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A network encryption traffic classification method based on data packet header, characterized in that: The following steps are involved: Obtain the original network traffic data, separate the header information of the data packet, and convert it into structured data after preprocessing, specifically: perform padding and alignment based on the header information of the data packet, convert it into structured data based on bits, bytes or protocol fields, and normalize it; The structured data obtained is based on the trained encrypted traffic classification model to obtain the packet classification result; Among them, based on the trained encrypted traffic classification model, the specific process of obtaining the data packet classification result is to extract the importance of each feature in the structured data, multiply different features by the corresponding importance, and obtain the structured data after feature recalibration; The classification results are obtained based on the structured data after feature recalibration. Specifically, three types of structured data based on bit, byte and protocol fields after feature recalibration are input into the LSTM network for classification.
2. The network encrypted traffic classification method based on data packet header according to claim 1 is characterized in that: Obtain the original network traffic and separate the header information of the data packet. Specifically, extract the original traffic of the network node based on the packet capture tool, output it as a file in a set format, and extract the header information of the data packet.
3. The network encrypted traffic classification method based on data packet header according to claim 1 is characterized in that: The length of the padding is as follows: 0 and 1 are the two normal states of the original traffic data, and -1 indicates the state where the data does not exist.
4. The network encrypted traffic classification method based on data packet header according to claim 1 is characterized in that: Alignment, specifically: aligning the protocol fields of the data packet header to obtain a data packet header of uniform length.
5. The network encrypted traffic classification method based on data packet header according to claim 1 is characterized in that: After the header information of the data packet is converted into the protocol field, the protocol field is segmented, and the length of each field after segmentation is no more than 8 bits.
6. The network encrypted traffic classification method based on data packet header according to claim 1 is characterized in that: The encrypted traffic classification model is based on the importance of each feature in the structured data. Specifically, the structured data is processed by dimensionality reduction to obtain the correlation between features, and then the weight of each feature is determined by dimensionality increase. After the number of features is restored to the original dimension, the importance of each feature is obtained.
7. The network encrypted traffic classification method based on data packet header according to claim 5 is characterized in that: The structured data is reduced in dimension through a fully connected layer, and then increased in dimension through another fully connected layer to obtain parameters of the same dimension as the feature data. After passing through an activation function, the parameters are restricted to a set range to obtain the weight of each feature, and the recalibrated structured data is obtained through a multiplication operation.
8. A network encrypted traffic classification system based on data packet headers, characterized in that: include: The data preprocessing module is configured to: obtain the original network traffic data, separate the header information of the data packet, and convert it into structured data after preprocessing, specifically: perform length padding and alignment based on the header information of the data packet, convert it into structured data based on bits, bytes or protocol fields, and perform normalization; The traffic classification module is configured to: obtain the data packet classification result based on the obtained structured data and the trained encrypted traffic classification model; Among them, based on the trained encrypted traffic classification model, the specific process of obtaining the data packet classification result is to extract the importance of each feature in the structured data, multiply different features by the corresponding importance, and obtain the structured data after feature recalibration; The classification results are obtained based on the structured data after feature recalibration. Specifically, three types of structured data based on bit, byte and protocol fields after feature recalibration are input into the LSTM network for classification.
