A network flow independent coding method based on self-supervised learning

Abstract

The present invention provides a network flow independent encoding method based on self-supervised learning, S1, collecting data; S2, assigning the data packets formed by the data to the corresponding flow with multi-tuple information; S3, pre-processing the data packets to represent the original binary data and generating a unified data packet to obtain a multi-dimensional vector about the unified data packet; S4, extracting flow features based on the attention-based ED model with the multi-dimensional vector as input. The present invention can deeply study the network flow features by realizing the unified representation that can generate network flows - collectively analyzing the basic information of network data packets, flows and their relationships.

Description

Technical Field

[0001] The invention belongs to the field of flow data processing, and in particular relates to a network flow independent encoding method based on self-supervisory learning. Background Art

[0002] Nowadays, cyber attacks have penetrated into various critical Internet applications, such as IoT systems, enterprise networks, and industrial infrastructure. Abnormal traffic analysis is an important means of detecting cyber attacks. Abnormal traffic analysis based on deep learning has achieved remarkable results, but there are still some shortcomings.

[0003] (1) Traffic analysis can only be performed in specific scenarios. For example, the Chinese patent with publication number CN113705619A is only for the detection of encrypted malicious traffic. The Chinese patent with publication number CN114268468A is only for vulnerability detection in smart grid systems.

[0004] (2) There are still difficulties in processing large-scale traffic data. For example, the patent with publication number CN113347659B uses LSTM and CNN to analyze traffic. However, LSTM cannot process sequences of large magnitude. For sequences of more than 1,000 magnitude or longer, problems such as gradient explosion may occur. In addition, each LSTM unit node means that there are 4 fully connected layers. If the time series span is large and the network is deep, there will be problems such as large amount of calculation and high time consumption. Due to the CNN pooling layer, using it to extract traffic features will lose certain valuable information.

[0005] (3) Lack of labeled data. Some existing deep learning models are supervised learning and require a large amount of labeled data, but these labeled data are lacking in actual scenarios. For example, the patent CN114268468A obtains data information in a preset format from the data stream and then manually screens the information to obtain the target information. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to provide a network flow independent encoding method based on self-supervised learning, which can generate a unified representation of network flows and conduct in-depth research on network flow characteristics - collectively analyzing the basic information of network data packets, flows and their relationships.

[0007] The above technical objectives of the present invention are achieved through the following solutions:

[0008] A network flow independent encoding method based on self-supervised learning, comprising:

[0009] S1. Collect data;

[0010] S2, assigning the data packet formed by the data to the corresponding flow with multi-tuple information;

[0011] S3, preprocessing the data packet to represent it as original binary data and generating a unified data packet, thereby obtaining a multi-dimensional vector of the unified data packet;

[0012] S4. Extract flow features using an attention-based ED model with the multidimensional vector as input.

[0013] Furthermore, in step S3, the preprocessing includes

[0014] Under the preset multiple protocols, the protocol is represented by the maximum packet header protocol allowed by each protocol;

[0015] Combine the headers of different protocols into a fixed-length packet header;

[0016] If any of the participating protocols is not included in the data packet, the corresponding position in the fixed-length packet header corresponding to the protocol is filled with equal bytes of 0.

[0017] Furthermore, the tuple is a quintuple, and the quintuple information is expressed as <source IP, source port, destination IP, destination port, protocol>.

[0018] Further, the S4 specifically includes:

[0019] S401, generating a vector sequence based on a multi-dimensional vector;

[0020] S402, sampling the vector sequence with a sliding window of a preset length to obtain multiple subsequences;

[0021] S403, encoding a vector of a first length in a preset length to obtain a hidden length sequence, and decoding a vector of a second length in the preset length;

[0022] The sequence length in the hidden length sequence is equal to the preset length;

[0023] S404, calculating the correlation of a sequence of the hidden length sequence equivalent to the second length level with respect to a sequence of two levels equivalent to the first length, and obtaining an attention vector mean, that is, a sampling feature of the data packet;

[0024] S405: Calculate the flow feature, where the flow feature is the average value of the attention vector means in the data packet.

[0025] Furthermore, the sum of the first length and the second length is equal to the preset length.

[0026] Furthermore, in S403, the encoding is performed using the graph neural network LSTM.

[0027] The above technical solution of the present invention has the following advantages compared with the prior art:

[0028] (1) The present invention realizes large-scale traffic processing. In the network, the same network flow is usually divided into multiple data packets for transmission, and the data packets of multiple network flows may be mixed together for transmission. Therefore, in network traffic analysis, it is difficult to accurately attribute each data packet to the corresponding flow. In addition, the data packet structure of the traffic from different protocols is also different. The present invention attributes the data packet to the corresponding flow according to the five-element information of the data packet <source IP, source port, destination IP, destination port, protocol>. And the formatting of the data packet is realized, so that no matter which protocol the traffic comes from, the binary bits of each data packet in the flow can be unified to facilitate the subsequent encoding of the network flow. The above content enables the model to process each network flow separately, so it can process multiple flows in parallel, solving the problem of difficulty in large-scale traffic processing.

[0029] (2) The invention calculates the similarity between data packets, that is, extracts the most relevant features of the data packets as the features of a network flow, and retains the context information of the data packet sequence, realizing the unified encoding of the network flow. It can support some subsequent work, such as identifying and preventing various network attacks and threats such as DDos attacks and malware; it can monitor the performance and response time of the application to make corresponding optimizations, and understand the user's behavior and preferences to provide personalized services for the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of a method provided by an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of the process of S4 in the method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] This embodiment provides a network flow independent encoding method based on self-supervised learning, such as Figure 1 As shown: It includes the following steps:

[0034] A network flow independent encoding method based on self-supervised learning, comprising:

[0035] S1. Collect data;

[0036] S2, assigning the data packet formed by the data to the corresponding flow with multi-tuple information;

[0037] S3, preprocessing the data packet to represent it as original binary data and generating a unified data packet, thereby obtaining a multi-dimensional vector of the unified data packet;

[0038] S4. Extract flow features using an attention-based ED model with the multidimensional vector as input.

[0039] Furthermore, in step S3, the preprocessing includes

[0040] Under the preset multiple protocols, the protocol is represented by the maximum packet header protocol allowed by each protocol;

[0041] Combine the headers of different protocols into a fixed-length packet header;

[0042] If any of the participating protocols is not included in the data packet, the corresponding position in the fixed-length packet header corresponding to the protocol is filled with equal bytes of 0.

[0043] Furthermore, the tuple is a quintuple, and the quintuple information is expressed as <source IP, source port, destination IP, destination port, protocol>.

[0044] Further, the S4 specifically includes:

[0045] S401, generating a vector sequence based on a multi-dimensional vector;

[0046] S402, sampling the vector sequence with a sliding window of a preset length to obtain multiple subsequences;

[0047] S403, encoding a vector of a first length in a preset length to obtain a hidden length sequence, and decoding a vector of a second length in the preset length;

[0048] The sequence length in the hidden length sequence is equal to the preset length;

[0049] S404, calculating the correlation of a sequence of the hidden length sequence equivalent to the second length level with respect to a sequence of two levels equivalent to the first length, and obtaining an attention vector mean, that is, a sampling feature of the data packet;

[0050] S405: Calculate the flow feature, which is the average of the means of all subsequence attention vectors.

[0051] Furthermore, the sum of the first length and the second length is equal to the preset length.

[0052] Furthermore, in S403, the encoding is performed using the graph neural network LSTM.

[0053] Specifically, in this embodiment, the stream coding model takes the original traffic data, i.e., data packets collected in the file pcap format, as input, and attributes each data packet to the corresponding stream according to the five-tuple information of the data packet <source IP, source port, destination IP, destination port, protocol>.

[0054] Before generating a unified network flow code, the data packets must first be uniformly encoded. To achieve unified encoding of data packets: process the original traffic data to form a standardized data representation format so that the traffic data can be directly used as the input of subsequent machine learning models. First, for a data packet, it is represented as a raw binary data model, and the maximum header length allowed by each protocol is used to represent the protocol. At the same time, the headers of different protocols are spliced ​​together into a fixed-length header. Specifically, the headers of the IPv4, TCP, UDP, and ICMP protocols are spliced ​​together with the Payload. If a packet does not have the header part of a certain protocol, it is filled with 0 to achieve an aligned binary representation.

[0055] In this embodiment, the following examples are provided:

[0056] TCP protocol packet header {t1, t2, ..., t 480} does not have the header part of the UDP protocol packet and the ICMP protocol packet, fill the corresponding bits of the UDP protocol and ICMP protocol with 0 to ensure that the binary representation of this packet has the same length as other packets. The result after filling is {IPv4 header, t1, t2, ..., t 480 , O1, O2, ..., O 128 , Payload}; UDP protocol packet {u1, u2, ..., u 64}, does not have the header part of the TCP protocol packet and the ICMP protocol packet, fill 0 in the corresponding part of the TCP protocol and ICMP protocol to ensure that the binary representation of this packet is the same as the length of other packets. The result after filling is {IPv4 header, O1, O2, ..., O 480 ,u1,u2,...,u 64 , O1, O2, ..., O 64 , Payload}. The principle of implementing unified encoding of data packets is to convert a data packet into a 1×n matrix, where each bit is a feature, and then use this matrix to train the model.

[0057] The unified encoding of network flows is based on the attention ED model.

[0058] flow The packet sequence In each Will be converted into a 148-dimensional vector Get vector sequence Since the data packet sequence is too long to be suitable for training, we use a length of l w Sampling the vector sequence with a sliding window can obtain right Put each subsequence before l e The vector is input into the encoder to get the first l e The output of the encoder, the output of the previous decoder, and the remaining l w -l e The decoder takes a sequence of packet vectors as input and calculates the hidden state of the current packet vector. The encoder and decoder use a deep LSTM structure. This process is similar to the translation task, which takes the first few words of a sentence as input and translates the rest of the words. In this work, since each packet in each network flow carries part of the overall data, the previous packet can be used to "translate" the next packet. For each sample, the graph neural network LSTM is used to encode the first l e vectors to get the hidden vector sequence The remaining l w -l e Vector decoding.

[0059] In this embodiment, the following examples are provided:

[0060] l w =8,l e =5, and there is a vector sequence [v 0 , …, v 7 ]. Then use the first 5 vectors as input and translate the rest. w is the preset length disclosed in this embodiment, l e is the first length, and the second length is the preset length minus the first length.

[0061] Use LSTM to calculate the hidden vector [h 0 ,…,h 7 The hidden vector [h 0 ,…,h 4 The calculation process of ] is as follows:

[0062] z i =σ(μ z v i +W z h i-1 +b z )

[0063] f i =σ(μf v i +W f h i-1 +b z )

[0064] o i =σ(μ o v i +W o h i-1 +b o )

[0065] c i =tanh(μ c v i +W c h i-1 +b c )

[0066] c i =f i *c i-1 +z i *c i

[0067] h i =tanh(c i )*o i

[0068] where h i ∈R 128×1 is the hidden state output of the ith content. * denotes the Hadamard product. μ j ∈R 128×148 , W j ∈R 128×128 , b j ∈R 128×1 (j∈z,f,o,c) are learnable parameters. i , f i , o i They are respectively the vector of the forget gate in LSTM, the i-th content input gate vector and the output gate vector.

[0069] Each hidden state is input into the attention layer, and the importance weight of each encoder hidden state for the current time step is calculated based on all the hidden state encodings of the encoder and the hidden state of the decoder at the current time step. Specifically, in the decoder, assume that the current hidden state is h 5 , h 5 With h 0 ,h 1 ,h 2 ,h 3 ,h 4 Compare them respectively and calculate h 5For the importance weight of each encoder hidden state, all encoder hidden states are weighted and summed according to the calculated weights to obtain an attention vector α5. The attention vector represents the encoder context information at the current time step, and the specific calculation formula is as follows:

[0070] α t =Softmax(sim(h t ,h0),...,sim(h t , h4))*(h0,...,h4), sim() is the similarity function

[0071] The attention vector and the decoder's current time step hidden state are connected in series to update the current step hidden state. The specific formula is as follows:

[0072] Indicates a serial operation

[0073] h t +h t ′ is used as the feature vector for the next time step input. The least squares error (LSE) loss function is used to optimize the parameters.

[0074] Attention layer output α p as the mean of the attention vector. Specifically, Represents a packet subsequence [p 0 ,…,p 7 ] context information. Calculate the attention vector α of all subsequences p After that, α p The mean of is the attention vector of the network flow, i.e., the context information. Through the above steps, the present invention can independently encode each network flow and self-supervise the generation of an attention vector with context information.

[0075] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A network flow independent coding method based on self-supervised learning, characterized in that: The steps include: S1. Collect data; S2, assigning the data packet formed by the data to the corresponding flow with multi-tuple information; S3, preprocessing the data packet to represent it as original binary data and generating a unified data packet, thereby obtaining a multi-dimensional vector of the unified data packet; The pre-processing comprises: Under the preset multiple protocols, the protocol is represented by the maximum packet header protocol allowed by each protocol; Combine the headers of different protocols into a fixed-length packet header; If any of the participating protocols is not included in the data packet, fill the corresponding position in the fixed-length packet header corresponding to the protocol with equal bytes of 0; S4, extracting flow features by using the attention-based ED model with the multidimensional vector as input; The S4 specifically includes: S401, generating a vector sequence based on a multi-dimensional vector; S402, sampling the vector sequence with a sliding window of a preset length to obtain multiple subsequences; S403, encoding a vector of a first length in a preset length to obtain a hidden length sequence, and decoding a vector of a second length in the preset length; The sum of the first length and the second length is equal to the preset length; The sequence length in the hidden length sequence is equal to the preset length; S404, calculating the correlation of a sequence equivalent to the second length magnitude in the hidden length sequence with a sequence equivalent to the first length magnitude, and obtaining a mean of the attention scores, that is, a sampling feature of the data packet; S405: Calculate a flow feature, where the flow feature is an average of the mean values ​​of the attention scores in the data packets.

2. The network flow independent coding method based on self-supervised learning according to claim 1 is characterized in that: The tuple is a quintuple, and the quintuple information is expressed as <source IP, source port, destination IP, destination port, protocol>.

3. The network flow independent coding method based on self-supervised learning according to claim 2 is characterized in that: In S403, encoding is performed using the graph neural network LSTM.

Images