Deep learning industrial malicious encrypted traffic identification method based on multi-feature fusion

Abstract

The invention relates to the technical field of encrypted traffic identification, and discloses a deep learning industrial malicious encrypted traffic identification method based on multi-feature fusion. According to the method, traffic basic features and traffic fingerprint features of a training sample data packet are extracted, feature fusion is carried out, the features are used as input of a deep neural network model, the deep neural network model is trained, a multi-head self-attention mechanism is adopted to carry out deep extraction on an original data packet, and global feature values are obtained; the extracted traffic feature vector has context information, and a residual network and nonlinear operation are introduced, so that the problem of gradient explosion / disappearance in the training process is solved, and the accuracy of the traffic feature vector is improved. The method has the characteristics of high detection precision and accurate identification for the industrial malicious encrypted traffic.

Description

Technical Field

[0001] The present invention belongs to the technical field of encrypted traffic identification, and specifically relates to a method for identifying industrial malicious encrypted traffic based on deep learning of multiple feature fusion. Background Art

[0002] In recent years, with the development of encryption technology, traffic encryption has been widely used on the Internet, but it has also brought many Internet security problems and become an opportunity for cyber criminals to hide malware and evade detection. Especially from the perspective of network security, more and more connected devices are attacked by network attackers. Attackers encrypt malware traffic to invade and attack systems anonymously.

[0003] The existing encrypted traffic detection method inputs traffic features into the LSTM (Long Short-Term Memory) network, obtains the output, and uses it as the query, key, and value of the multi-head attention mechanism. After multiple linear transformations, the global features are merged. This method extracts traffic timing characteristics through the LSTM network, but the LSTM network cannot be calculated in parallel and does not completely solve the gradient vanishing / exploding problem. It takes a long time to train the model and the accuracy is not high. In addition, in the calculation process of the multi-head attention mechanism, nonlinear transformation is not introduced, which easily leads to underfitting of the model and cannot identify malicious traffic well. At the same time, this method is aimed at detecting malicious encrypted traffic in traditional IT networks, and does not introduce the characteristics of encrypted traffic in industrial networks. Therefore, it cannot be applied to the industrial field.

[0004] Existing encrypted traffic identification technologies are mainly divided into three categories: deep packet inspection-based, machine learning-based, and deep learning-based.

[0005] Deep packet inspection technology deeply analyzes the contents of data packets, extracts specified field information, and forms a string, usually called an encryption fingerprint, to identify the type of encrypted flow. However, this method can only detect encryption features of known data sets and cannot detect newly generated malicious encrypted traffic. In addition, with the development of encryption technology, the number of payloads will continue to decrease in the future, making the deep packet inspection-based method less and less suitable for the detection of encrypted traffic.

[0006] Based on machine learning technology, the features of the traffic are extracted, and the supervised learning algorithm is used to train the algorithm using samples containing real value labeled output as a classifier to learn the association between the features and the required labeled output. By using samples containing real value labeled output, a suitable classification model is established for training. However, the accuracy of encrypted traffic detection based on machine learning technology is overly dependent on the effectiveness of features, and feature extraction is time-consuming and labor-intensive, and prone to human errors.

[0007] Based on deep learning technology, high-level features are obtained by automatically extracting raw traffic data in depth, thereby improving the accuracy of detection. Existing methods for detecting encrypted traffic based on deep learning technology generally use deep neural networks as algorithms. Among them, the detection technology using the CNN (convolutional neural network) algorithm does not consider the relationship between the local and the whole, resulting in the lack of sufficient contextual information in the extracted features, and the generated prediction model is not accurate; the detection technology using the RNN (recurrent neural network) algorithm and the LSTM (long short-term memory) algorithm has the defect that their algorithms cannot be calculated in parallel, resulting in low detection efficiency. In addition, the existing detection methods based on deep learning do not consider the gradient disappearance / explosion problem, and cannot perform higher-order feature extraction, which affects the accuracy of the model.

[0008] The existing encrypted traffic detection technology is only targeted at IT networks, does not take into account the characteristics of industrial control networks, and cannot be directly applied in industry. Summary of the invention

[0009] The present invention aims to address the deficiencies in the prior art and provide a method for identifying industrial malicious encrypted traffic based on deep learning based on multi-feature fusion, which has high detection accuracy and can quickly and accurately identify industrial malicious encrypted traffic. The method aims to extract multi-modal features and perform feature fusion, train a deep neural network model, and use a multi-head self-attention mechanism to deeply extract traffic data, extract the global eigenvalues ​​of traffic data, introduce residual networks and nonlinear operations, solve the problem of gradient explosion / vanishing during training, and improve the accuracy of traffic feature vectors.

[0010] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0011] A method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion, including training process and detection process;

[0012] The training process includes: preprocessing the training sample data packets, extracting the basic traffic features and traffic fingerprint features and fusing them to obtain traffic features, inputting the traffic features into the deep neural network model for deep learning, and then performing random mask shielding and error data replacement on the training sample data packets, and inputting the training sample data packets into the deep neural network model for training;

[0013] The detection process includes: preprocessing the original data packet, extracting the basic traffic features and traffic fingerprint features and fusing them to obtain traffic features, using a multi-head self-attention mechanism to deeply extract the traffic features and obtain global eigenvalues, and using residual networks and nonlinear operations based on the global eigenvalues ​​to obtain higher-order traffic feature vectors. The higher-order traffic feature vectors are input into the fully connected layer of the trained deep neural network model for calculation, and finally a one-dimensional traffic feature vector is obtained through operation with the weight matrix. The one-dimensional traffic feature vector is input into the softmax function, and a probability vector is output to obtain the detection result of the original data packet.

[0014] Preferably, the training process includes the following steps:

[0015] S1. The industrial network encrypted traffic training sample data packet includes input data and data type, and the industrial network encrypted traffic training sample data packet is preprocessed;

[0016] S2, extracting the basic traffic features and traffic fingerprint features of the training sample data packets obtained in step S1, the extracted traffic features include interactive behavior features, traffic sequence features, and traffic fingerprint features, and fusing the traffic features, and inputting the traffic features into a deep neural network model for deep learning;

[0017] S3, randomly select data with a proportion E in the traffic sequence of the training sample data packet to perform MASK shielding and erroneous data replacement;

[0018] S4. Train the deep neural network model according to the data type of the training sample data packet.

[0019] Preferably, the detection process comprises the following steps:

[0020] P1. Preprocess the original data packets of the industrial network encrypted traffic, extract the basic traffic features and traffic fingerprint features of the original data packets, and fuse them to obtain traffic features. The preprocessing process, feature extraction process, and fusion process are the same as those in the training process.

[0021] P2: Weight the traffic features, perform multi-head self-attention calculation, fuse them again, and output the fused global feature value;

[0022] P3, input the fused global eigenvalues ​​and traffic features into the residual network, and calculate the traffic feature vector after introducing the residual connection;

[0023] P4, perform nonlinear operation on the flow feature vector after the residual connection is introduced;

[0024] P5, input the flow characteristic vector after nonlinear operation into step P2, and repeat steps P2-P4 to obtain a high-order flow characteristic vector;

[0025] P6. Input the high-order traffic feature vector into the fully connected layer of the trained deep neural network model for calculation, and obtain a one-dimensional vector representing the high-order traffic feature vector by operating with the weight matrix;

[0026] P7. Input the one-dimensional vector output from step P6 into softmax, output the probability vector, and obtain the detection result of the original data packet.

[0027] Preferably, in step S1, the specific steps of preprocessing the industrial network encrypted traffic training sample data packet are as follows:

[0028] S101, traffic cleaning: remove multicast messages, broadcast messages, and messages without application layer protocols from training sample data packets;

[0029] S102, traffic segmentation: dividing the cleaned training sample data packets into multiple files according to the session;

[0030] S103, traffic camouflage: randomly replace the IP and MAC information contained in each session;

[0031] S104, traffic sampling: obtaining n messages of the training sample data packet processed in step S103.

[0032] Preferably, in step S2, the specific steps of obtaining the flow characteristics are as follows:

[0033] S201, extracting basic traffic characteristics of n messages in step S104;

[0034] S202, performing vector transposition on the basic flow characteristics extracted in step S201 to obtain a basic flow characteristic matrix;

[0035] S203, converting the basic traffic features extracted in step S201 into an image, processing the image using convolution, and then extracting image features, where the extracted image features are traffic fingerprint features;

[0036] S204: The basic traffic feature matrix and the traffic fingerprint feature are integrated to obtain traffic features.

[0037] Preferably, in step S201, the basic traffic characteristics of a single message are described as a single data message, and the following characteristics can be extracted from a single data message:

[0038] p θ =x θ ,b θ ,t θ ,d θ ,syn θ ,psh θ ,urgθ ,fin θ ,ack θ ,rst θ

[0039] Among them, p represents a single data packet, θ represents the sequence number of the packet, θ∈n, x represents the five-tuple information of the data packet, which belongs to the interactive behavior feature, b represents the load of the data packet, which belongs to the traffic sequence feature, t represents the sending time of the data packet, which belongs to the traffic sequence feature, d represents the direction of the data packet, which belongs to the traffic sequence feature, syn represents whether it contains the SYN flag, which belongs to the interactive behavior feature, psh represents whether it contains the PSH flag, which belongs to the interactive behavior feature, urg represents whether it contains the URG flag, which belongs to the interactive behavior feature, fin represents whether it contains the FIN flag, which belongs to the interactive behavior feature, ack represents whether it contains the ACK flag, which belongs to the interactive behavior feature, and rst represents whether it contains the RST flag, which belongs to the interactive behavior feature;

[0040] The basic characteristics of the traffic of n messages can be described as a data packet file, which can be expressed as:

[0041] P 0 ={p 1 =(x 1 ,b 1 ,t 1 ,d 1 ,syn 1 ,psh 1 ,urg 1 ,fin 1 ,ack 1 ,rst 1 ,...,

[0042] p n =(x n ,b n ,t n ,d n ,syn n ,psh n ,urg n ,fin n ,ack n ,rst n )}

[0043] Where P 0 Represents a data packet file;

[0044] Perform vector transposition on the data packet file obtained in step S201 to obtain the basic traffic feature matrix.

[0045]

[0046] Among them, P1 represents the basic characteristic matrix of traffic.

[0047] Preferably, in step S203, the image size of the traffic basic feature conversion is W in *H in , use convolution to process the image, the output image size is W out *H out :

[0048]

[0049] Where W out Indicates the output image width, W in Indicates the width of the input image, W filter represents the width of the convolution kernel, H out Indicates the output image height, H in Indicates the input image height, H filter Represents the height of the convolution kernel, C represents the number of border filling layers, and S represents the convolution kernel step size;

[0050] Then, the convolution-processed image is passed to the fully connected layer of the trained deep neural network model to extract image features. The extracted image features are the extracted traffic fingerprint features. The traffic fingerprint features of a single message can be expressed as a 1*m-dimensional matrix:

[0051] A1={a 1,1 ,a 1,2 ,...,a 1,m}

[0052] A1 represents the traffic fingerprint feature of the first message, and a represents the feature data;

[0053] The traffic fingerprint feature of a single message is converted into an m*1 dimensional matrix, and n messages are concatenated to obtain an m*n dimensional matrix, which is expressed as:

[0054]

[0055] Among them, P2 represents the traffic fingerprint feature matrix;

[0056] The basic traffic feature matrix P1 and the traffic fingerprint feature matrix P2 are concatenated in the order of messages to obtain the fused basic traffic features, as follows:

[0057]

[0058] P is a (m+10)·n dimensional matrix.

[0059] Preferably, in step S3, of the selected data with a proportion of E, data with a proportion of F is masked by MASK, and the remaining data is replaced with erroneous data.

[0060] Preferably, in step P2, the fused global eigenvalue calculation process is as follows:

[0061] Assume the flow characteristic is P y , P y As input, we weight the vectors q, k, and v respectively.

[0062]

[0063] Among them, i∈r, r represents the number of heads, represents the weight matrix of the i-th query vector, represents the weight matrix of the i-th head key vector, represents the weight matrix of the i-th head value vector, q i represents the i-th query vector, k i represents the i-th head key vector, v i represents the i-th head value vector;

[0064] Then calculate the attention score, the calculation formula is:

[0065]

[0066] After expansion:

[0067]

[0068] After normalization, the attention score is obtained:

[0069] α' i,j =softmax(α i,j )

[0070] Among them, i∈r, j∈t, r, t both represent the number of heads, α i,j represents the attention score of the i-th query vector to the j-th key vector, represents the transpose of the i-th query vector, k j represents the j-th head key vector, d k represents the dimension of the key matrix;

[0071] According to the calculated attention score and the i-th head value vector v i Sum and get the global eigenvalue B of the i-th head i , the calculation formula is:

[0072]

[0073] The global eigenvalues ​​B of each head i The splicing and fusion are performed, and the dimension reduction is performed through the dimension reduction matrix to obtain the fused global eigenvalue B, and the calculation formula is:

[0074]

[0075] Among them, B represents the fused global eigenvalue, W 0 Represents a dimensionally reduced matrix.

[0076] Preferably, according to the method for identifying industrial malicious encrypted traffic based on deep learning based on multi-feature fusion according to claim 9, it is characterized in that in step P3, the calculation process of the traffic feature vector after introducing the residual connection is as follows:

[0077] B'=B+P y

[0078] Among them, B' represents the traffic feature vector after the residual connection is introduced.

[0079] Compared with the prior art, the beneficial effects of the present invention are:

[0080] (1) The present invention extracts basic traffic features and traffic fingerprint features of abnormal traffic based on the characteristics of industrial network traffic, and fuses them to obtain traffic features, including interactive behavior features, traffic sequence features, and traffic fingerprint features. The extracted features have multimodal characteristics, and after feature fusion, they are used as inputs of a deep neural network model, and the deep neural network model is trained, so that the deep neural network model can better identify malicious traffic in the industrial network;

[0081] (2) The present invention uses a multi-head self-attention mechanism to perform deep extraction on traffic data and extract the global eigenvalues ​​of the traffic data. The obtained traffic features have contextual information, and the multi-head self-attention mechanism supports parallel computing, which can improve the training efficiency to a certain extent. The residual network and nonlinear operation are introduced, so that the accuracy of the traffic feature vector increases with the number of layers of the multi-head self-attention mechanism, thereby obtaining a more accurate traffic feature vector, improving the detection accuracy, and avoiding the gradient explosion / vanishing problem that may occur during the training process;

[0082] (3) The present invention combines the characteristics of industrial control network traffic and introduces a MASK mask mechanism when training the model. It randomly selects a proportion E of data in the traffic sequence of the training sample data for MASK shielding and erroneous data replacement. This allows the model to pay more attention to the traffic sequence information during training, thereby improving the accuracy of identifying attacks against industrial networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is a flow chart of a method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] Example 1

[0086] Combination Figure 1 As shown, an embodiment of the present invention provides a method for identifying industrial malicious encrypted traffic based on deep learning of multiple feature fusion, including a training process and a detection process, wherein the training process includes the following steps:

[0087] S1. The industrial network encrypted traffic training sample data packet includes input data and data type, and the industrial network encrypted traffic training sample data packet is preprocessed;

[0088] S2, extracting the basic traffic features and traffic fingerprint features of the training sample data packets obtained in step S1, the extracted traffic features include interactive behavior features, traffic sequence features, and traffic fingerprint features, and fusing the traffic features, and inputting the traffic features into a deep neural network model for deep learning;

[0089] Specifically, a multi-head self-attention mechanism can be used to extract the basic traffic features and traffic fingerprint features of the training sample data packets of the industrial network encrypted traffic;

[0090] S3, randomly select data with a proportion E in the traffic sequence of the training sample data packet to perform MASK shielding and erroneous data replacement;

[0091] S4, training the deep neural network model according to the data type of the training sample data packet;

[0092] The detection process includes the following steps:

[0093] P1. Preprocess the original data packets of the industrial network encrypted traffic, extract the basic traffic features and traffic fingerprint features of the original data packets, and fuse them to obtain traffic features. The preprocessing process, feature extraction process, and fusion process are the same as those in the training process.

[0094] Specifically, a multi-head self-attention mechanism can be used to extract the basic traffic features and traffic fingerprint features of the original data packets of the encrypted traffic of the industrial network;

[0095] P2: Weight the traffic features, perform multi-head self-attention calculation, fuse them again, and output the fused global feature value;

[0096] P3, input the fused global eigenvalues ​​and traffic features into the residual network, and calculate the traffic feature vector after the residual connection is introduced;

[0097] P4. Perform nonlinear operation on the flow feature vector after the residual connection is introduced. Specifically, the ReLU function is used for nonlinear operation.

[0098] P5, input the flow characteristic vector after nonlinear operation into step P2, and repeat steps P2-P4 to obtain a high-order flow characteristic vector;

[0099] P6. Input the high-order traffic feature vector into the fully connected layer of the trained deep neural network model for calculation, and obtain a one-dimensional vector representing the high-order traffic feature vector by operating with the weight matrix;

[0100] P7. Input the one-dimensional vector output from step P6 into softmax, output the probability vector, and obtain the detection result of the original data packet.

[0101] Example 2

[0102] Different from Example 1, in this embodiment, in step S101, the specific steps of preprocessing the industrial network encrypted traffic training sample data packet are as follows:

[0103] S101, traffic cleaning: remove multicast messages (such as IGMP), broadcast messages (such as ARP), and messages without application layer protocols in the training sample data packets;

[0104] S102, traffic segmentation: dividing the cleaned training sample data packets into multiple files according to the session;

[0105] S103, traffic camouflage: randomly replace the IP and MAC information contained in each session to prevent fixed address information from affecting model parameters;

[0106] S104, traffic sampling: obtaining n messages of the training sample data packet processed in step S103.

[0107] Furthermore, in step S2, the extracted basic traffic features include interactive behavior features and traffic sequence features, and the traffic features include interactive behavior features, traffic sequence features, and traffic fingerprint features.

[0108] Furthermore, in step S2, the specific steps of obtaining the flow characteristics are as follows:

[0109] S201, extracting basic traffic characteristics of n messages in step S104;

[0110] S202, performing vector transposition on the basic flow characteristics extracted in step S201 to obtain a basic flow characteristic matrix;

[0111] S203, converting the basic traffic features extracted in step S201 into an image, processing the image using convolution, and then extracting image features, where the extracted image features are traffic fingerprint features;

[0112] S204: The basic traffic feature matrix and the traffic fingerprint feature are integrated to obtain traffic features.

[0113] Example 3

[0114] Different from the first and second embodiments, in the present embodiment, in step S201, the basic traffic characteristics of a single message are described as a single data message, and the following characteristics can be extracted from a single data message:

[0115] p θ =x θ ,b θ ,t θ ,d θ ,syn θ ,psh θ ,urg θ ,fin θ ,ack θ ,rst θ

[0116] Among them, p represents a single data packet, θ represents the sequence number of the packet, θ∈n, x represents the five-tuple information of the data packet (specifically, the source and destination IP, source and destination port, and transport layer protocol), which belongs to the interactive behavior feature, b represents the data packet payload, its unit is Byte, which belongs to the flow sequence feature, t represents the data packet sending time, which belongs to the flow sequence feature, d represents the data packet direction (1 is toserver, 2 is toclient), which belongs to the flow sequence feature, syn represents whether the SYN flag is included (1 is included, 0 is not included, UDP is 0), which belongs to the interactive behavior feature, psh Indicates whether the PSH flag is included (1 for inclusion, 0 for exclusion, and 0 for UDP), which is an interactive behavior feature. urg indicates whether the URG flag is included (1 for inclusion, 0 for exclusion, and 0 for UDP), which is an interactive behavior feature. fin indicates whether the FIN flag is included (1 for inclusion, 0 for exclusion, and 0 for UDP), which is an interactive behavior feature. ack indicates whether the ACK flag is included (1 for inclusion, 0 for exclusion, and 0 for UDP), which is an interactive behavior feature. rst indicates whether the RST flag is included (1 for inclusion, 0 for exclusion, and 0 for UDP), which is an interactive behavior feature.

[0117] The basic characteristics of the traffic of n messages can be described as a data packet file, which can be expressed as:

[0118] P 0 ={p 1 =(x 1 ,b 1 ,t 1 ,d 1 ,syn 1 ,psh 1 ,urg 1 ,fin 1 ,ack 1 ,rst 1 ,...,

[0119] p n =(x n ,b n ,t n ,d n ,syn n ,psh n ,urg n ,fin n ,ack n ,rst n )}

[0120] Where P 0 Indicates a data package file.

[0121] Furthermore, in step S202, the data packet file obtained in step S201 is vector-transposed to obtain a basic traffic feature matrix.

[0122]

[0123] Among them, P1 represents the basic characteristic matrix of traffic.

[0124] Further, in step S203, the basic flow characteristics of the n messages extracted in step S201 are converted into a size W in *H in The image is processed using convolution, and the output image size is W out *H out Specifically, the size of the output image is 32*32, that is, the size is 1024 bits. The part exceeding 1024 bits is truncated and the insufficient part is filled. The bytes are converted to pixel grayscale values, 0x00 is black, 0xff is white, and the image is passed through a 3-channel convolution layer. The convolution kernel size is 3*3, that is, the convolution kernel width * convolution kernel height = 3*3, the convolution kernel step size is 1, and the number of boundary filling layers is 1. The output image size is 32*32, that is, the image width * image height = 32*32:

[0125]

[0126] Where W out Indicates the output image width, W in Indicates the width of the input image, W filter represents the width of the convolution kernel, H out Indicates the output image height, H in Indicates the input image height, H filter Represents the height of the convolution kernel, C represents the number of border filling layers, and S represents the convolution kernel step size;

[0127] Then, the convolution-processed image is passed to the fully connected layer of the trained deep neural network model to extract image features. The extracted image features are the extracted traffic fingerprint features. The traffic fingerprint features of a single message can be expressed as a 1*m-dimensional matrix:

[0128] A1={a 1,1 ,a 1,2 ,...,a 1,m}

[0129] A1 represents the traffic fingerprint feature of the first message, and a represents the feature data;

[0130] The traffic fingerprint feature of a single message is converted into an m*1 dimensional matrix, and n messages are concatenated to obtain an m*n dimensional matrix, which is expressed as:

[0131]

[0132] Among them, P2 represents the traffic fingerprint feature matrix.

[0133] Furthermore, in step S102 and step S204, the traffic basic feature matrix P1 and the traffic fingerprint feature matrix P2 are concatenated according to the message sequence to obtain the fused traffic basic feature, as follows:

[0134]

[0135] P is a (m+10)·n dimensional matrix.

[0136] Further, in step S3, among the selected data with a proportion of E, data accounting for F are masked by MASK, and the remaining data are replaced with erroneous data. In this embodiment, 15% of the data are randomly selected for MASK masking and erroneous data replacement. Among the 15% of the data, 80% of the data are masked by MASK, and 20% of the data are replaced with erroneous data, so that when the model is trained, it can pay more attention to the traffic sequence information, so that the model can understand the semantic information of the traffic.

[0137] Furthermore, in step P2, the calculation process of the fused global eigenvalue is as follows:

[0138] Assume the flow characteristic is P y , P y As input, we weight the vectors q, k, and v respectively.

[0139]

[0140] Among them, i∈r, r represents the number of heads, represents the weight matrix of the i-th query vector, represents the weight matrix of the i-th head key vector, represents the weight matrix of the i-th head value vector, q i represents the i-th query vector, k i represents the i-th head key vector, v i represents the i-th head value vector;

[0141] Then calculate the attention score, the calculation formula is:

[0142]

[0143] After expansion:

[0144]

[0145] After normalization, the attention score is obtained:

[0146] α' i,j =softmax(α i,j )

[0147] Where i∈r, j∈t, r and t both represent the number of heads. In this embodiment, the values ​​of r and t are both 8. i,j represents the attention score of the i-th query vector to the j-th key vector, q i T represents the transpose of the i-th query vector, k j represents the j-th head key vector, d k Indicates the dimension of the key matrix, parameter d k The role of is to avoid the gradient being too small after being input into the softmax function;

[0148] According to the calculated attention score and the i-th head value vector v i Sum and get the global eigenvalue B of the i-th head i , the calculation formula is:

[0149]

[0150] The global eigenvalues ​​B of each head iThe splicing and fusion are performed, and the dimension reduction is performed through the dimension reduction matrix to obtain the fused global eigenvalue B, and the calculation formula is:

[0151]

[0152] Among them, B represents the fused global eigenvalue, W 0 represents the dimension reduction matrix;

[0153] Furthermore, in step P3, the flow feature vector calculation process after the residual connection is introduced is as follows:

[0154] B'=B+P y

[0155] Among them, B' represents the traffic feature vector after the residual connection is introduced;

[0156] By extracting the basic traffic features P of the original data packet y The input is fed into the residual network and added to the fused global eigenvalue B to form the message feature vector after the residual connection is introduced to avoid gradient disappearance / explosion.

[0157] The above description is only an embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement and improvement made within the application scope of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying industrial malicious encrypted traffic based on deep learning of multiple feature fusion, characterized in that: Including training process and testing process; The training process includes: preprocessing the training sample data packets, extracting the basic traffic features and traffic fingerprint features and fusing them to obtain traffic features, inputting the traffic features into the deep neural network model for deep learning, and then performing random mask shielding and error data replacement on the training sample data packets, and inputting the training sample data packets into the deep neural network model for training; The detection process includes: preprocessing the original data packet, extracting the basic traffic features and traffic fingerprint features and fusing them to obtain traffic features, using a multi-head self-attention mechanism to deeply extract the traffic features and obtain global eigenvalues, and using residual networks and nonlinear operations based on the global eigenvalues ​​to obtain higher-order traffic feature vectors. The higher-order traffic feature vectors are input into the fully connected layer of the trained deep neural network model for calculation, and finally a one-dimensional traffic feature vector is obtained through operation with the weight matrix. The one-dimensional traffic feature vector is input into the softmax function, and a probability vector is output to obtain the detection result of the original data packet.

2. According to the method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 1, it is characterized in that: The training process includes the following steps: S1. The industrial network encrypted traffic training sample data packet includes input data and data type, and the industrial network encrypted traffic training sample data packet is preprocessed; S2, extracting the basic flow characteristics and flow fingerprint characteristics of the training sample data packets obtained in step S1 to obtain flow characteristics, wherein the extracted flow characteristics include interactive behavior characteristics, flow sequence characteristics, and flow fingerprint characteristics, and fusing the flow characteristics, and inputting the flow characteristics into a deep neural network model for deep learning; S3, randomly select data with a proportion E in the traffic sequence of the training sample data packet to perform MASK shielding and erroneous data replacement; S4. Train the deep neural network model according to the data type of the training sample data packet.

3. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 2 is characterized in that: The detection process includes the following steps: P1. Preprocess the original data packets of the industrial network encrypted traffic, extract the basic traffic features and traffic fingerprint features of the original data packets, and fuse them to obtain traffic features. The preprocessing process, feature extraction process, and fusion process are the same as those in the training process. P2: Weight the traffic features, perform multi-head self-attention calculation, fuse them again, and output the fused global feature value; P3, input the fused global eigenvalues ​​and traffic features into the residual network, and calculate the traffic feature vector after introducing the residual connection; P4, perform nonlinear operation on the flow feature vector after the residual connection is introduced; P5, input the flow characteristic vector after nonlinear operation into step P2, and repeat steps P2-P4 to obtain a high-order flow characteristic vector; P6. Input the high-order traffic feature vector into the fully connected layer of the trained deep neural network model for calculation, and obtain a one-dimensional vector representing the high-order traffic feature vector by operating with the weight matrix; P7. Input the one-dimensional vector output from step P6 into softmax, output the probability vector, and obtain the detection result of the original data packet.

4. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 3 is characterized in that: In step S1, the specific steps of preprocessing the industrial network encrypted traffic training sample data packet are as follows: S101, traffic cleaning: remove multicast messages, broadcast messages, and messages without application layer protocols from training sample data packets; S102, traffic segmentation: dividing the cleaned training sample data packets into multiple files according to the session; S103, traffic camouflage: randomly replace the IP and MAC information contained in each session; S104, traffic sampling: obtaining n messages of the training sample data packet processed in step S103.

5. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 4 is characterized in that: In step S2, the specific steps of obtaining traffic characteristics are as follows: S201, extracting basic traffic characteristics of n messages in step S104; S202, performing vector transposition on the basic flow characteristics extracted in step S201 to obtain a basic flow characteristic matrix; S203, converting the basic traffic features extracted in step S201 into an image, processing the image using convolution, and then extracting image features, where the extracted image features are traffic fingerprint features; S204: The basic traffic feature matrix and the traffic fingerprint feature are integrated to obtain traffic features.

6. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 5 is characterized in that: In step S201, the basic traffic characteristics of a single message are described as a single data message, and the following characteristics can be extracted from a single data message: p θ =x θ ,b θ ,t θ ,d θ ,syn θ ,psh θ ,urg θ ,fin θ ,ack θ ,rst θ Among them, p represents a single data packet, θ represents the sequence number of the packet, θ∈n, x represents the five-tuple information of the data packet, which belongs to the interactive behavior feature, b represents the load of the data packet, which belongs to the traffic sequence feature, t represents the sending time of the data packet, which belongs to the traffic sequence feature, d represents the direction of the data packet, which belongs to the traffic sequence feature, syn represents whether it contains the SYN flag, which belongs to the interactive behavior feature, psh represents whether it contains the PSH flag, which belongs to the interactive behavior feature, urg represents whether it contains the URG flag, which belongs to the interactive behavior feature, fin represents whether it contains the FIN flag, which belongs to the interactive behavior feature, ack represents whether it contains the ACK flag, which belongs to the interactive behavior feature, and rst represents whether it contains the RST flag, which belongs to the interactive behavior feature; The basic traffic characteristics of n messages can be described as a data packet file, which can be expressed as: P 0 ={p 1 =(x 1 ,b 1 ,t 1 ,d 1 ,syn 1 ,psh 1 ,urg 1 ,fin 1 ,ack 1 ,rst 1 ,..., p n =(x n ,b n ,t n ,d n ,syn n ,psh n ,urg n ,fin n ,ack n ,rst n )} Where P 0 Represents a data packet file; Perform vector transposition on the data packet file obtained in step S201 to obtain the basic traffic feature matrix. Among them, P1 represents the basic characteristic matrix of traffic.

7. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 6 is characterized in that: In step S203, the image size of the traffic basic feature conversion is W in *H in , use convolution to process the image, the output image size is W out *H out : Where W out Indicates the output image width, W in Indicates the width of the input image, W filter represents the width of the convolution kernel, H out Indicates the output image height, H in Indicates the input image height, H filter Represents the height of the convolution kernel, C represents the number of border filling layers, and S represents the convolution kernel step size; Then, the convolution-processed image is passed to the fully connected layer of the trained deep neural network model to extract image features. The extracted image features are the extracted traffic fingerprint features. The traffic fingerprint features of a single message can be expressed as a 1*m-dimensional matrix: A1={a 1,1 ,a 1,2 ,...,a 1,m } A1 represents the traffic fingerprint feature of the first message, and a represents the feature data; The traffic fingerprint feature of a single message is converted into an m*1 dimensional matrix, and n messages are concatenated to obtain an m*n dimensional matrix, which is expressed as: Among them, P2 represents the traffic fingerprint feature matrix; The basic traffic feature matrix P1 and the traffic fingerprint feature matrix P2 are concatenated in the order of messages to obtain the fused basic traffic features, as follows: P is a (m+10)·n dimensional matrix.

8. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 7 is characterized in that: In step S3, among the selected data with a proportion of E, data with a proportion of F is shielded by MASK, and the remaining data is replaced with erroneous data.

9. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 8 is characterized in that: In step P2, the calculation process of the fused global eigenvalue is as follows: Assume the flow characteristic is P y , P y As input, we weight the vectors q, k, and v respectively. q i =W i Q ·P y k i =W i K ·P y v i =W i V ·P y Among them, i∈r, r represents the number of heads, W i Q represents the weight matrix of the i-th query vector, W i K represents the weight matrix of the i-th key vector, W i V represents the weight matrix of the i-th head value vector, q i represents the i-th query vector, k i represents the i-th head key vector, v i represents the i-th head value vector; Then calculate the attention score, the calculation formula is: After expansion: After normalization, the attention score is obtained: a' i,j =softmax(a i,j ) Among them, i∈r, j∈t, r, t both represent the number of heads, α i,j represents the attention score of the i-th query vector to the j-th key vector, q i T represents the transpose of the i-th query vector, k j represents the j-th head key vector, d k represents the dimension of the key matrix; According to the calculated attention score and the i-th head value vector v i Sum and get the global eigenvalue B of the i-th head i , the calculation formula is: The global eigenvalues ​​B of each head i The splicing and fusion are performed, and the dimension reduction is performed through the dimension reduction matrix to obtain the fused global eigenvalue B, and the calculation formula is: Among them, B represents the fused global eigenvalue, W 0 Represents a dimensionally reduced matrix.

10. The method for identifying industrial malicious encrypted traffic based on deep learning and multi-feature fusion according to claim 9 is characterized in that: In step P3, the flow feature vector calculation process after the residual connection is introduced is as follows: B'=B+P y Among them, B' represents the traffic feature vector after the residual connection is introduced.

Images