A website content behavior identification method and system based on encrypted traffic bidirectional burst sequence

Abstract

The application relates to a website content behavior recognition method and system based on bidirectional burst sequences of encrypted traffic. The method comprises the following steps: obtaining behavior traffic data of an encrypted website; preprocessing the behavior traffic data into bidirectional burst sequences; establishing a website content behavior recognition model, training the website content behavior recognition model by taking the bidirectional burst sequences as input; and using the trained website content behavior recognition model to recognize the website content behavior of the encrypted website. The application selects bidirectional burst sequences as input, can better capture the differences between website content behaviors, adopts a convolutional neural network to construct a traffic representation model, realizes automatic traffic representation and feature extraction, avoids manual feature extraction and selection, and finally achieves the purpose of accurately recognizing encrypted website content behavior traffic.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of network measurement and behavior analysis, and particularly relates to a website content behavior identification method based on bidirectional burst sequence of encrypted traffic. BACKGROUND

[0002] Website content behavior refers to the specific content of a user's behavior on a website, including behaviors mainly based on browsing text, behaviors mainly based on pictures, and behaviors mainly based on videos, i.e. text behavior, picture behavior, and video behavior. Website content behavior identification mainly infers the specific content of a user's behavior on a website through the traffic generated by the user's behavior on the website.

[0003] In recent years, due to the importance of privacy protection and data security transmission, the HTTPS protocol is gradually replacing the original HTTP protocol to avoid illegal monitoring and tampering of data during transmission and ensure data transmission security. More and more websites use the https protocol for encrypted transmission.

[0004] With the encryption of websites and the promotion of TLS1.3, traditional encrypted website identification methods based on SNI or certificate matching are ineffective. It is necessary to build more complex website fingerprints to support website identification. Existing website identification methods use information such as traffic timing, packet direction, and packet length to assist machine learning and deep learning algorithms to extract deep features to achieve website identification. Website fingerprints refer to the characteristics of the traffic generated when a user sends and receives data when accessing a website. They have unique characteristics and can be used to identify websites, web pages, and analyze website behavior.

[0005] Encrypted websites have the characteristics of diversity. Website types are diverse, including social, video, news, and various types of websites. Encrypted website behavior has the characteristics of similarity and difference. Similarity is reflected in the fact that different websites have similar network behaviors, such as different websites that may produce picture behavior. The traffic generated by the same website has certain similarity due to the same receiver and sender. Difference is reflected in the fact that the same website may produce different behaviors, such as text behavior, picture behavior, etc. Different website behaviors, such as text behavior and picture behavior, have certain differences in data packet length.

[0006] But encryption is accompanied by some security problems. Due to the encryption of traffic, network monitoring and detection face greater challenges. Encrypted website traffic provides a breeding ground for malicious network behavior, and attackers use encrypted traffic as a cover for malicious behavior, threatening network security. Malicious network behaviors such as network fraud, network rumors, and network viruses spread on the network. Through the analysis of website behavior, these malicious behaviors can be effectively prevented from breeding. Therefore, in order to effectively protect the security of encrypted websites and even the entire network, as well as the discovery of malicious website behavior, accurate identification of user behavior in encrypted websites can not only provide necessary information support for network monitoring, but also be the premise of malicious network behavior detection and the basis of maintaining network security.

[0007] The existing network behavior related research of encrypted traffic mainly focuses on the behavior identification of encrypted applications, and the identification methods can be mainly divided into application-intra behavior identification and application-behavior identification. The research on application-intra behavior identification focuses on the identification of one or several application-intra behaviors, uses the statistical characteristics or packet length of traffic, and adopts a relatively simple classification method such as machine learning. This kind of method depends on manual feature extraction and selection, and in addition, the statistical characteristics of the traffic of the same website have similarity, and the behavior identification accuracy in the same website is not high. The research on application-behavior identification is less, and mainly focuses on the specific behavior identification of instant messaging applications.

[0008] The traffic generated by network content behavior can reflect the composition of the access content of the user on the website, and is conducive to monitoring the bad content in the website. The elements of the webpage are closely related to the burst of traffic (referring to a large amount of traffic generated in a short time). Therefore, a method for inferring website content behavior through bidirectional burst sequence of traffic is needed, and under the premise of obtaining bidirectional traffic, the burst sequence of traffic is further extracted, so as to identify the corresponding website content behavior. SUMMARY

[0009] The present application provides a website content behavior identification method and system based on bidirectional burst sequence of encrypted traffic. The present application extracts the bidirectional burst sequence of encrypted traffic for traffic analysis, and combines a deep learning algorithm to solve the problem that the prior art cannot effectively capture the characteristics of the traffic corresponding to the webpage content when identifying network content behavior, thereby causing low identification accuracy and high algorithm complexity.

[0010] The technical scheme adopted by the present application is as follows:

[0011] A website content behavior identification method based on bidirectional burst sequence of encrypted traffic, comprising the following steps:

[0012] Obtain behavior traffic data of an encrypted website;

[0013] Preprocess the behavior traffic data into bidirectional burst sequence;

[0014] The website content behavior recognition model is trained by taking the bidirectional burst sequence as input.

[0015] The website content behavior recognition model is trained by taking the bidirectional burst sequence as input.

[0016] Further, the behavior traffic data of the encrypted website is obtained by online capturing traffic or using collected offline data, and the traffic data is saved with the file extension.pcap.

[0017] Further, the behavior traffic data of the encrypted website is obtained by online capturing traffic or using collected offline data, and the traffic data is saved with the file extension.pcap.

[0018] The URL list of the target encrypted website and the corresponding website behavior operation are read, and a target URL address is read from the URL list.

[0019] The Web dirver program is started to automatically open the browser and input the read URL address.

[0020] An operation is read from the website corresponding behavior list, a script for automatically simulating the operation of the corresponding behavior is called, a tcpdump packet capturing program is started, the behavior automatic operation script is executed, and the website behavior operation is simulated.

[0021] After the operation, the packet capturing process is ended, the next operation is executed, the browser is closed when the operations in the behavior list of a website are ended, the next website in the website list is read, and the above operations are repeated until the website list is read.

[0022] Further, the behavior traffic data is preprocessed into a bidirectional burst sequence, including:

[0023] Irrelevant traffic is filtered out.

[0024] The network session packet segmentation method based on five-tuple is used to extract the flow from the packet and classify it according to the five-tuple content. The packets with the same five-tuple belong to the same uplink or downlink unidirectional data flow, the direction information of the packet is saved, +1 represents uplink traffic, and -1 represents downlink traffic. The flow with too short length due to connection establishment failure is discarded, and finally the required data flow set is obtained.

[0025] The uplink and downlink traffic are processed respectively, the uplink / downlink burst is defined as the unidirectional packet sequence corresponding to each HTTP message, and the bidirectional burst sequence is defined as the sequence of the length of the unidirectional burst in all uplink / downlink.

[0026] Further, the website content behavior recognition model comprises a basic module and a full connection module; the basic module comprises a one-dimensional convolutional neural network layer, a batch normalization layer and a maximum pooling layer; and the full connection module comprises a full connection layer.

[0027] Further, the output of the full connection module is input into a softmax classifier for classification, a cross-entropy loss function is used to calculate the loss between a predicted value and a real label, and the website content behavior recognition model is trained.

[0028] A website content behavior recognition system based on bidirectional burst sequences comprises:

[0029] A traffic acquisition module is configured to acquire behavior traffic data of an encrypted website.

[0030] A traffic preprocessing and bidirectional burst sequence extraction module is configured to preprocess the behavior traffic data into bidirectional burst sequences.

[0031] A model construction module is configured to establish a website content behavior recognition model.

[0032] A training module is configured to train the website content behavior recognition model by taking the bidirectional burst sequences as input.

[0033] An evaluation index calculation module is configured to perform website content behavior recognition of the encrypted website by using the trained website content behavior recognition model, and calculate the overall accuracy, the specified type recall rate and the specified type precision for accurate quantification.

[0034] Compared with the prior art, the website content behavior recognition system based on bidirectional burst sequences has the following beneficial effects:

[0035] The bidirectional burst sequence is selected as input, which better reflects the interaction of HTTP message requests and responses than the original data packet length sequence, and the burst sequence changes greatly due to the difference of website elements, so that the difference between website content behaviors can be better captured.

[0036] The CNNs-based convolutional neural network is adopted to construct the traffic representation model, so that automatic traffic representation and feature extraction are realized, manual feature extraction and selection are avoided, and the purpose of accurately identifying encrypted website content behavior traffic is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The figure is a flowchart of encrypted website content behavior traffic data collection.

[0038] Figure 2 The figure is a structural diagram of the website content behavior recognition model.

[0039] Figure 3A module composition schematic diagram of a website content behavior identification system based on encrypted traffic bidirectional burst sequence in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the above objectives, characteristics and advantages of the present application more apparent and easy to understand, the present application is further described in detail below with specific embodiments and drawings.

[0041] A website content behavior identification method based on encrypted traffic bidirectional burst sequence of the present application comprises the following steps:

[0042] S1: obtaining encrypted website behavior traffic data;

[0043] S2: traffic preprocessing into burst flow sequence;

[0044] S3: establishing a website content behavior identification model;

[0045] S4: encrypted website content behavior traffic identification.

[0046] Each step is described in detail below.

[0047] The encrypted website behavior traffic data of step S1 can be obtained by online capturing traffic, or offline data collected can be used, and the traffic data is saved in the form of.pcap file extension. The offline data is collected by writing an automatic behavior operation script to assist the traffic capture tool tcpdump to collect traffic data.

[0048] The specific collection process is shown in Figure 1 and comprises the following steps:

[0049] S1-1: reading the URL list of the target encrypted website and the corresponding behavior operation of the website, and reading a target URL address from the URL list. In order to exclude the influence of the browser itself on the website traffic, the cache and Cookie records in the browser are emptied before all operations, and the target website is accessed in the incognito mode of the Chrome browser. The incognito mode or traceless browsing of the Chrome browser allows the user to browse the webpage without leaving any traces of accessing the website on the computer, including cache files, cookies, history records, download records, etc., to protect the privacy and security of the user.

[0050] S1-2: starting the Web dirver program, automatically opening the browser, and inputting the URL address read in S1-1.

[0051] S1-3: Read one behavior from the behavior list corresponding to the website in S1-2, and call the script for automated simulation of the operation of the corresponding behavior. At the same time, start the tcpdump packet capture program, execute the behavior automation operation script, and simulate the website behavior operation.

[0052] S1-4: After the behavior operation is completed, end the packet capture process, and then execute the next behavior operation. When the behavior operation in the behavior list in a website is completed, close the browser, read the next website in the website list, and repeat the above operation until the website list is read completely.

[0053] Step 2: Preprocess the captured traffic data.

[0054] S2-1: Filter out irrelevant traffic. Filter out packets without actual load such as acknowledgement packets, and retransmission packets such as TCP Retransmission and Dup ACK due to network congestion.

[0055] S2-2: Aggregation by five-tuple. Use the network session packet segmentation method based on five-tuple (source IP address, destination IP address, source port, destination port, and transport layer protocol type) to extract the flow from the packet. Classify according to the five-tuple content, and the packets with the same five-tuple belong to the same uplink or downlink unidirectional data flow. Save the direction information of the packet, and use +1 to identify the uplink traffic and -1 to identify the downlink traffic. In addition, discard those flows with too short length due to connection establishment failure and the like, and finally obtain the required data flow set. The five-tuple refers to the source IP, destination IP, source port, destination port, and transport layer protocol.

[0056] S2-3: Extract bidirectional burst sequence. Process the uplink and downlink traffic obtained in S2-2 respectively. The uplink / downlink burst sequence is defined as the unidirectional packet sequence corresponding to each HTTP message. The bidirectional burst sequence is defined as a sequence composed of the uplink burst length sequence and the downlink burst length sequence.

[0057] Since TCP is a byte stream protocol, it can split the message from the upper layer of TCP (such as TLS) in any way for transmission. For larger record size, the TLS record exceeding the MSS limit will be transmitted in multiple TCP payloads, and only one of these TCP payloads contains the TLS header. The length of the one-way burst is the value of the TLS header length field. For smaller record size, the entire TLS record can be accommodated in a single TCP payload. Since the length of the TLS record is usually smaller than the MSS, TCP will intercept the next TLS record to fill the current payload as large as the MSS. Therefore, the TCP payload can not contain any TLS header, one or more TLS headers. The calculation of the uplink / downlink burst length sequence is as follows:

[0058] First, traverse the TLS header in the current TCP payload and the packet length in the one-way stream, add the value of the TLS header length field to the current packet length. Then, subtract the length of the current TCP payload from the sum to obtain the length of the remaining TLS record in the subsequent TCP payload. If it is equal to 0, the current one-way burst ends, and its length is added to the one-way burst sequence. Finally, the one-way burst sequence is obtained. The above process is performed on the uplink stream and the downlink stream respectively, and the burst order is restored according to the timestamp to obtain the bidirectional burst sequence.

[0059] Step S3 establishes a website content behavior recognition model. The input of the model is the bidirectional burst sequence output by step S2. The structure of the model is as shown in Figure 2 , which mainly consists of two parts: a basic module (basic CNNs module) and a fully connected module, and a deep network is realized by repeating the two parts, for example Figure 2 has two basic modules.

[0060] The first part is the basic module. The basic module has two, and each basic module mainly consists of a one-dimensional convolutional neural network layer (convolutional layer), a batch normalization layer, and a maximum pooling layer.

[0061] One-dimensional convolutional neural network layer: Convolutional neural network is mainly responsible for feature extraction, which mainly consists of a set of filters, which runs convolution operation on the input and transmits the operation result to the next layer. Each basic module in the model applies three one-dimensional convolutions, and their convolution kernel sizes are set to 5, 3, and 3, respectively, the input channels are set to 1, 32, and 32, respectively, the convolution kernel step is 1, the padding is set to the SAME mode, the biases are initialized to 0, and the Relu activation function is used for non-linear processing to avoid the problems of gradient explosion and gradient disappearance during training.

[0062] Batch Normalization Layer: also known as batch normalization layer. The purpose of batch normalization is to overcome the problem of difficulty in training caused by the increase of the number of layers in the neural network. Because in the neural network, the input of each layer is changed after the calculation in the layer, and the input data distribution is different from the original input data distribution, the increase of the neural network in the previous layer will be amplified by the neural network in the later layer. Using batch normalization can correct and normalize the input of the layer in time to fix the mean and variance of each layer input.

[0063] Max Pooling Layer: After the convolution process, the pooling operation is performed. The essence of pooling is sampling, which reduces the dimension of the input feature map, preserves the main features, reduces the parameters and calculation amount, prevents overfitting, and also speeds up the operation. The maximum pooling operation is selected, the pooling size is set to 2, the stride is set to 2, and the padding is set to SAME.

[0064] The second part is a full connection module. After the input is subjected to convolution operation by two identical basic modules, it enters two full connection neural networks. The number of hidden nodes of the first full connection layer is 512, and the second full connection layer is adjusted according to the number of categories of samples. After the first full connection layer, batch standard processing is used to keep the input of each layer of neural network the same distribution. Then, the Relu activation function is used for nonlinear processing, and the Dropout algorithm is used to randomly delete some hidden neurons in the network, which achieves the effect of regularization to a certain extent. The ratio of Dropout is 0.5.

[0065] The output of the full connection module is input into the softmax classifier for classification. The cross-entropy loss function is used to calculate the loss between the predicted value and the true label, so as to train the website content behavior recognition model.

[0066] Step S4 is to encrypt the website content behavior recognition. The website content behavior recognition model constructed and trained in step S4 is used to identify the encrypted website content behavior on the test set.

[0067] In one specific embodiment of the present application, a website content behavior recognition method based on encrypted traffic bidirectional burst sequence is provided, which comprises the following steps:

[0068] The traffic generated by the encrypted website content behavior is taken as the object, and the traffic is collected according to step S1.

[0069] According to the preprocessing of the traffic flow described in step S2, first, irrelevant traffic flow is filtered, and the filtered traffic flow is extracted according to the five-tuple to obtain the flow. Secondly, the traffic burst sequence in the uplink and downlink is extracted respectively, and the unidirectional burst sequence is sorted according to the timestamp to obtain the bidirectional burst sequence. In addition, the bidirectional burst sequence is divided into a training set, a validation set and a test set according to the cross-validation method.

[0070] According to step S3, the encrypted website content behavior recognition model is constructed. A convolutional network is built by using a deep learning framework (such as Pytorch, etc.). At the same time, the training set generated in step S2 is directly input into the recognition model for training.

[0071] According to step S4, the model proposed in step S3 is tested on the test set generated in step S2.

[0072] Another embodiment of the application provides a website content behavior recognition system based on bidirectional burst sequence, as shown in Figure 3 The website content behavior recognition system based on bidirectional burst sequence comprises:

[0073] A traffic acquisition module for acquiring (reading) network traffic online (offline);

[0074] A traffic preprocessing and bidirectional burst sequence extraction module for filtering non-payload traffic, five-tuple aggregation and bidirectional burst sequence extraction on the acquired original traffic;

[0075] A model construction module for constructing a behavior recognition model by using a website content behavior recognition method based on bidirectional burst sequence;

[0076] A training module for training the website behavior recognition model;

[0077] An evaluation index calculation module for performing website content behavior recognition of the encrypted website by using the trained website content behavior recognition model, calculating the overall accuracy, specified type recall rate and specified type precision, and performing accurate quantification.

[0078] Through the above technical solutions, the application provides an effective method and system to realize the identification of the content behavior in the encrypted website.

[0079] Another embodiment of the application provides a computer device (computer, server, smart phone, etc.), which comprises a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step in the method of the application.

[0080] Another embodiment of the present application provides a computer readable storage medium (such as ROM / RAM, magnetic disk, optical disk) storing a computer program, which, when executed by a computer, implements each step of the method of the present application.

[0081] The specific embodiments of the application disclosed above are intended to be illustrative only. Those of ordinary skill in the art will be able to devise numerous alternative arrangements, which, although not explicitly set forth herein, embody the principles of the application and are included within its spirit and scope. The present application is not limited to the embodiments disclosed above, but is defined by the scope of the claims.

Claims

1. A method for identifying website content behavior based on bidirectional burst sequences of encrypted traffic, characterized in that, Includes the following steps: Obtain behavioral traffic data from encrypted websites; Preprocess behavioral traffic data into bidirectional burst sequences; A website content behavior recognition model is established, and the model is trained using bidirectional burst sequences as input. Using a trained website content behavior recognition model to identify website content behavior on encrypted websites; The process of preprocessing behavioral traffic data into bidirectional burst sequences includes: Filter out irrelevant traffic; Using a network session packet segmentation method based on 5-tuples, flows are extracted from packets and classified according to the content of the 5-tuples. Packets with the same 5-tuple belong to the same unidirectional data flow in the uplink or downlink direction. The direction information of the packets is saved, with +1 indicating uplink traffic and -1 indicating downlink traffic. Flows that are too short due to connection establishment failure are discarded, and finally a set of data flows that meet the requirements is obtained. Uplink and downlink traffic are processed separately. Uplink / downlink bursts are defined as a sequence of one-way data packets corresponding to each HTTP message, and bidirectional burst sequences are defined as a sequence of one-way burst lengths in all uplink / downlink links. The website content behavior recognition model includes a basic module and a fully connected module; the basic module includes a one-dimensional convolutional neural network layer, a batch normalization layer, and a max pooling layer; the fully connected module includes a fully connected layer.

2. The method according to claim 1, characterized in that, The traffic data of the encrypted website is obtained by capturing traffic online or by using collected offline data and saving the traffic data with the .pcap file extension.

3. The method according to claim 1, characterized in that, The acquisition of behavioral traffic data from encrypted websites includes: Read the list of URLs of the target encrypted website and the corresponding actions of the website, and read a target URL address from the URL list; Start the web driver program to automatically open a browser and enter the URL address to be read; Read a behavior from the list of behaviors corresponding to the website, call the script for the automated simulation operation of the corresponding behavior, and at the same time start the tcpdump packet capture program to execute the automated operation script to simulate the website behavior operation. After the action is completed, the packet capture process ends, and the next action is executed. When the action in the action list of a website is completed, the browser is closed, the next website in the website list is read, and the above operation is repeated until the website list is completely read.

4. The method according to claim 1, characterized in that, The one-dimensional convolutional neural network layer includes three one-dimensional convolutions with kernel sizes of 5, 3, and 3, and input channels of 1, 32, and 32, respectively. The stride of each kernel is 1, padding is set to SAME mode, biases are initialized to 0, and ReLU activation function is used for non-linear processing to avoid gradient explosion and gradient vanishing problems during training. The batch normalization layer is used to correct the training samples in a timely manner, and the input of the normalization layer is used to fix the mean and variance of the input of each layer. The max pooling layer has a pooling size of 2, a stride of 2, and padding set to SAME.

5. The method according to claim 1, characterized in that, The output of the fully connected module is input into a softmax classifier for classification. The cross-entropy loss function is used to calculate the loss between the predicted value and the true label, and the website content behavior recognition model is trained.

6. A website content behavior recognition system based on bidirectional burst sequences using the method described in any one of claims 1 to 5, characterized in that, include: The traffic acquisition module is used to acquire behavioral traffic data from encrypted websites; The traffic preprocessing and bidirectional burst sequence extraction module is used to preprocess behavioral traffic data into bidirectional burst sequences; The model building module is used to build website content behavior recognition models; The training module is used to train the website content behavior recognition model with bidirectional burst sequences as input; The evaluation index calculation module is used to identify the content behavior of encrypted websites using the trained website content behavior recognition model, and to calculate the overall accuracy, recall rate of a specified type, and precision of a specified type for precise quantification.

7. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 5.

Images