Encrypted video identification method in Tor environment

Abstract

The invention discloses an encrypted video identification method in a Tor environment. The method comprises the four steps of collecting flow, extracting an AU-burst length sequence, extracting upstream features and classifying downstream tasks. Firstly, real-time Tor traffic is captured at a network information service center of a local area network entrance, then features are extracted from the real-time Tor traffic by using a corresponding TREFS i T segmentation strategy according to Tor video traffic characteristics to obtain an encrypted ADU-burst length sequence, then noise filtering and secondary feature extraction are performed by using a 1DCNN model, and finally classification is performed by using a random forest. According to the method for identifying the encrypted video in the Tor complex environment, the encrypted video does not need to be decrypted, the video played by the client through the Tor network can be identified through the video traffic characteristics so as to supervise harmful videos, and the method has wide application scenes and good supervision effects.

Description

Technical Field

[0001] The invention belongs to the technical field of network security and mainly relates to an encrypted video recognition method in a Tor environment. Background Art

[0002] In the current Internet environment, personal information is deeply integrated with the Internet. Although the popularity of encryption technologies such as TLS1.3 and HTTPS has significantly improved communication security, metadata such as users' IP addresses and handshake processes may still leak sensitive information, which has prompted more and more users to turn to anonymous communication technologies to protect their privacy. As the most widely used anonymous communication system, Tor has more than three million daily active users thanks to its unique rerouting and low-latency mechanism.

[0003] Existing security research on the Tor network mainly focuses on two directions: traffic type identification and website fingerprint attacks. However, research on the identification of encrypted video traffic in the Tor environment is relatively scarce. This research gap deserves great attention, especially considering the important position of video traffic in the total Internet traffic. According to statistics, video traffic accounted for 66% of the total traffic in the first half of 2022, and this proportion continues to grow. More importantly, compared with website visits, video viewing choices often contain richer privacy information such as personal preferences, which makes the security research of encrypted video traffic more urgent and practical. Early encrypted video recognition mainly focused on manual feature extraction and classifier selection. The most widely used method is to build a plaintext fingerprint library and collect transmission fingerprints for matching and identification.

[0004] Existing research on encrypted video recognition mainly focuses on feature extraction and classifier design. Generally speaking, there are two main approaches to video recognition: fingerprint matching based on feature construction based on expert knowledge and automatic feature extraction and classification based on deep learning.

[0005] For feature methods based on expert knowledge, the typical approach is to build a plaintext fingerprint database in advance, collect traffic and recover the length of all ADU-bursts, and then use an appropriate algorithm to match the resulting length sequence with the fingerprint database. Due to the difficulty of ADU-burst length recovery, there are also works that attempt to build the database and fingerprint matching directly at the ciphertext level, but significant errors usually occur. Therefore, the former method of using application layer length recovery is more desirable. However, in the Tor environment, interference mechanisms such as end-to-end encryption and transmission obfuscation make the existing segmentation strategies and recovery algorithms no longer applicable, making the existing fingerprint matching methods invalid.

[0006] Meanwhile, researchers also use deep learning for automatic feature extraction, filtering, and classification to enhance the robustness of the recognition process. Some methods collect aggregated flow length sequences and then adopt a Convolutional Neural Network (CNN) model; some use Siamese networks to train a Recurrent Neural Network (RNN) with a self-attention mechanism. However, existing deep learning models cannot effectively handle the highly dynamic and noisy Tor video transmission in video recognition, resulting in a significant reduction in classification performance.

[0007] In summary, traditional video recognition methods rely on length reduction methods and segmentation strategies. The existing length reduction methods are not applicable to the multi-layer encryption mode in the Tor environment, and the obfuscated environment of Tor renders the existing segmentation strategies ineffective; the feature selection and noise filtering methods of deep learning recognition methods perform poorly in the Tor environment. Therefore, this paper first proposes a video recognition attack model in the Tor environment - TorVIA: We analyze the transmission characteristics of video streams in the Tor environment, summarize two important transmission characteristics, and design an ADU segmentation strategy suitable for the Tor environment - TREFSiT based on these characteristics. Considering the multi-layer encryption mode of Tor, it is extremely difficult to accurately restore the ADU-burst length, and an accurate original video ADU-burst length sequence cannot be obtained. This paper uses a 1DCNN model for secondary feature extraction, filters the impact of obfuscated data on segmentation, retains the strong correlation between the encrypted video length sequence and video content, and finally uses the random forest algorithm for video classification. Summary of the Invention

[0008] The present invention is precisely a method for identifying and classifying encrypted videos under Tor without decryption. It includes four steps: traffic collection, ADU-burst length sequence extraction, upstream feature extraction, and downstream task classification. First, we capture real-time Tor traffic at the Network Information Service Center, then use the corresponding TREFSiT segmentation algorithm according to the Tor video traffic to convert the encrypted video stream data into an encrypted ADU-burst length sequence, then use a 1DCNN model for noise filtering and secondary feature extraction, and finally use a random forest for classification. The present invention can achieve video supervision in the Tor environment, monitor harmful videos, and alert relevant departments for further control.

[0009] To achieve the above object, the technical solution adopted by the present invention is: An encrypted video recognition method in a Tor environment, comprising the following steps:

[0010] S1, traffic collection: The client plays a video using the Tor network, and the supervisor uses tools such as wireshark to capture Tor traffic in real time at the local area network entrance;

[0011] S2. ADU-burst Length Sequence Extraction: When a client makes a resource request during video playback, it receives a group of bursty Application Data Units (ADUs), which are collectively named an ADU-burst. Based on the characteristics of video transmission in the HAS protocol, the ADU-burst length sequence of an encrypted video stream has a strong correlation with the original video content (fixed resolution) and will not change due to the playback environment or transmission background. Therefore, it can be used as a feature basis for video recognition. According to the two characteristics of encrypted video traffic in the Tor environment analyzed, "split transmission" and "request obfuscation", we set two segmentation conditions for each flow in the Tor environment: 1) The current ADU-burst has been transmitted; 2) The next ADU-burst arrives after a certain time threshold after the next request. By segmenting the TLS record stream, we obtain the length of each approximate ADU-burst, initially forming the characteristic ADU-burst length sequence, which can be used for data training and matching in the next stage. We name this ADU-burst segmentation strategy TREFSiT;

[0012] S3. Upstream Feature Extraction: The model of the present invention uses 1DCNN to perform secondary feature extraction on the ADU-burst length sequence obtained in S2. The 1DCNN model can effectively fuse sequences within a certain length through a convolutional kernel. Therefore, we use the 1DCNN model upstream to perform secondary feature extraction to mitigate the impact of complex environments on ADU-burst segmentation while retaining the strong correlation between the ADU-burst length sequence and the original video content.

[0013] S4. Downstream Task Classification: After noise filtering by the upstream feature extractor, we input the obtained feature vectors into various types of classifiers and select the one with the best classification effect as the final classifier.

[0014] As an improvement of the present invention, step S2 specifically includes the following steps:

[0015] S2-1: By analyzing the traffic transmission data in the Tor environment and comparing it with that in the non-Tor environment, we summarize two major characteristics: 1) During the video stream transmission process, it is not necessarily single-stream transmission, and sometimes it is divided into two-stream transmission; 2) During the video stream transmission process, the client will send obfuscated data that is not a request.

[0016] S2-2: Based on S2-1, we propose an ADU-burst segmentation strategy TREFSiT, mainly by setting two judgment conditions for each request: 1) the current ADU-burst has been transmitted; 2) the next ADU-burst arrives after a certain time threshold after this request. The time threshold Δtime for the second request is set to 250 ms.

[0017] S2-3: Regarding the content proposed in S2-2, we use TREFSiT to divide the actual data stream of TLS records (carried by the TCP protocol) to obtain an ADU-burst sequence. The judgment method for the first condition is to judge whether the last TLS record of a burst transmission is complete (in practice, it is necessary to compare the TCP packet sequence number interval with the length claimed by the TLS record to check whether the end of the last TCP block is the end of the last TLS block). If it is satisfied, the ADU-burst has been transmitted; for the second condition, that is, for the next request Δtime of each stream, a time threshold judgment is made, requiring the next ADU-burst data to arrive after a certain expected time threshold after the request. Only when both conditions are met will the TLS record stream be segmented, and the encrypted payloads between two requests that meet the conditions are added together as an element of the ADU-burst segmentation.

[0018] As another improvement of the present invention, the loss function for training the 1DCNN model in step S3 is as follows, where C is the number of categories, y c is the true label, is the probability predicted by the model:

[0019]

[0020] As yet another improvement of the present invention, in step S4, after the pre-training stage of the upstream task, the ADU-burst length vector after secondary extraction is obtained for specific video classification tasks. The classification models include but are not limited to logistic regression, random forest, SVM, and decision tree.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] (1) The present invention mainly innovates in the segmentation strategy. It analyzes the video transmission characteristics in the Tor environment for the first time, improves the existing traditional segmentation method, and proposes an innovative TREFSiT segmentation strategy. By setting two conditions for each stream, it almost precisely segments the ADU-burst, forming an ADU-burst length sequence with strong correlation to the original video content. Based on this, the video recognition method TorVIA in the Tor environment is proposed for the first time, achieving a breakthrough from scratch in the supervision of dark web video traffic without decryption.

[0023] (2) Compared with existing video recognition methods, this method is designed for the characteristics of the Tor environment, has higher robustness to the obfuscated environment of Tor, and stronger usability in the Tor environment. For example, in the closed-world YouTube video recognition experiment where 50 videos are each played 50 times in an environment with an i7 10700F CPU, 32 GB of RAM, and an RTX 4090 GPU, the optimal precision rate of the existing video recognition method is 67.3%, while the precision rate of this method reaches 89.4%, significantly improving the recognition accuracy and truly realizing practical Tor video traffic supervision.

[0024] (3) This method is widely applicable and can be applied to a variety of different video platforms. Experiments show that for video platforms represented by YouTube that adopt DASH streaming technology and video platforms represented by Dailymotion that adopt HLS streaming technology, this method can complete effective encrypted video recognition, so it has relatively complete usability. Brief Description of the Drawings

[0025] Figure 1 It is a schematic flow diagram of an encrypted video recognition method in a Tor environment according to the present invention. Detailed Embodiments

[0026] The present invention will be further clarified below in conjunction with the drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0027] Embodiment 1

[0028] An encrypted video recognition method in a Tor environment, as Figure 1 shown, includes four parts. The first part is video traffic collection; the second part is ADU-burst length sequence feature extraction; the third part is 1DCNN model training; the fourth part is downstream encrypted video classification testing.

[0029] The present invention selects a video dataset in a multi-platform Tor environment as the test object to test and evaluate the method of the present invention and further elaborate.

[0030] An encrypted video recognition method in a Tor environment specifically includes the following steps:

[0031] Step S1, Video traffic collection: Connect to the international network through a soft router and then use Wireshark to collect. In this embodiment, nearly 200G of video data is collected from the Internet. Among them, two data platforms YouTube and Dailymotion using different protocols are retained, and 113GB and 78GB of video data are collected respectively. The number of identified videos is 50 for each, and each video is collected 50 times, which are known videos for training and testing. At the same time, 500 unknown videos are collected respectively as an open-world data set to simulate the real network environment and test the unknown video categories and known video categories.

[0032] Step S2, Feature extraction of ADU-burst length sequence, including the following sub-steps;

[0033] Step S2-1, By analyzing the traffic transmission characteristics in the Tor environment and comparing the differences with the non-Tor environment, two major characteristics are summarized: 1) During the video stream transmission process, it is not necessarily single-stream transmission, and some will be divided into dual-stream transmission; 2) During the video stream transmission process, request confusion will occur on the client side.

[0034] In this example, the video data collected from the real Tor environment is analyzed. We collected approximately 200GB of video data on two different video platforms, YouTube and Dailymotion, and conducted statistics. It is found that the ratio of dual-stream transmission to single-stream transmission of the same video in a fixed environment on the Dailymotion platform is approximately 7:3, and the ratio on the YouTube platform is approximately 1:4. At the same time, in dual-stream transmission, the traffic in each channel approximately accounts for 1 / 2 of the entire video traffic. At the same time, in the Tor environment, the Tor proxy node sends an average of 43 times of request confusion data to the video service provider between one data packet request and the next data packet request. This request confusion transmission mode has a great impact on the original ADU-burst segmentation strategy.

[0035] Step S2-2, Based on S2-1, we propose an ADU-burst segmentation method TREFSiT, mainly by setting two conditions for each request: 1) The previous ADU-burst has been transmitted; 2) The next ADU-burst arrives after a certain time threshold for this request. The time threshold Δtime for the second request is set to 250ms.

[0036] Step S2-3. For the content proposed in S2-2, we judge the first condition of TREFSiT by determining whether the ADU-burst before the request has finished transmission, that is, verifying whether the end of the last TCP block is the end of the last TLS record. If so, for the next request Δtime of each flow, a time threshold judgment is made. Only when both conditions are met will the ADU-burst be segmented, and the encrypted payloads between the two qualified requests are added together as an element of the ADU-burst segment.

[0037] In this implementation, the TLS record header occupies 5 bytes, including 1 byte of content type, 2 bytes of TLS version, and 2 bytes of length. Let the fixed length of the header be l 0 . Each data packet starts with the control information of multiple underlying protocols including the Ethernet frame protocol, IP, and TCP, totaling 54 bytes, as shown in the following equations (1)-(2).

[0038] l 0 = 5 (1)

[0039] len ph = 54 (2)

[0040] Let the position of the first byte in the TLS record header be pos, and subtract the length len of the underlying protocol header ph to obtain len tail , that is, the remaining tail data length of the previous TLS record in front of the data packet, as shown in equation (3).

[0041] len tail = pos - len ph (3)

[0042] According to the TCP protocol design, the sequence number at the start of the payload can be read from the control information in the header, and let it be seq p . Add seq p to len tail and l 0 to obtain the sequence number seq of the first byte of the TLS encrypted data td . And adding seq td to the encrypted data length len td is the sequence number of the first byte of the next received TLS record header, denoted as seq ntd , as shown in equations (4)-(5).

[0043] seq td = seq p + len tail + l0 (4)

[0044] seq ntd = seq td + len td (5)

[0045] If P is immediately followed by a TLS record, then seq ntd should be the sequence number of the first byte of the next TLS record header; if there is no TLS record after P, i.e., the transmission has ended, then seq ntd should be equal to the TCP field value next sequence number of the last packet of the TLS record. Therefore, if seq ntd is found to be equal to the next sequence number, it means that the transmission has been completed.

[0046] Step S3, the 1DCNN model training step, includes the following sub-steps;

[0047] Step S3-1, in this step, assume that the (L-1)-th convolutional layer has N l-1 feature maps, denoted as Then, for the k-th feature map of the L-th layer, its input is the sum of the convolutions of all the feature maps in the previous layer with the corresponding convolutional kernels, as shown in Equation (6):

[0048]

[0049] Then, the hyperbolic tangent function (tanh) is used for non-linear activation: to enable the model to have the ability to express non-linear relationships and improve the discriminant performance of the model. Finally, max pooling (MaxPooling) is used to downsample the activated feature maps, denoted as: The pooled feature maps are used as the input for the next convolutional layer, and the above operations are repeated. The CNN module of the model in this paper is stacked by multiple convolutional layers and pooling layers, aiming to gradually extract the high-level features of the input data.

[0050] Video recognition is actually a multi-classification task. Therefore, we use the cross-entropy function as the loss function, as shown in Equation (7), where C is the number of classes, y c is the true label, is the probability predicted by the model:

[0051]

[0052] Step S4, Downstream Classification Test: Utilize the noise filtering of the upstream 1DCNN for feature vectors to enhance the strong correlation between the ADU-burst sequence and the original video content, and use a suitable classifier to classify the encrypted video in downstream classification.

[0053] Test Case

[0054] 1. Test Case 1:

[0055] Test Objective: The performance of the model of this method and other video recognition models in classifying and detecting videos in the Tor environment in a closed world is verified to demonstrate the superiority of this model over other models in detection performance.

[0056] Test Environment: Model training and evaluation experiments are carried out in an environment with an i7 10700F CPU, 32 GB RAM, and an RTX 4090 GPU.

[0057] Test Method: The accuracy, precision, and recall of the experimental results are evaluated according to the confusion matrix, and the calculation formulas are as follows:

[0058]

[0059]

[0060] Parameter Settings: As shown in Table 1, the ratio of the training set, test set, and validation set is 8:1:1;

[0061] Table 1 Hyperparameter Settings for Test 1

[0062] Learning rate Number of CNN layers N Convolution kernel size Dropout Epoch 0.002 3 3 0.2 50

[0063] Dataset Description: The dataset used in this test is the Tor video dataset collected in a real environment, as shown in Table 2.

[0064] Table 2 Dataset Description

[0065]

[0066] Test Description: First, the performance of the model and the baseline model in a closed world is evaluated in detail. In a closed world, users only watch the videos in the attacker's dataset, and the attacker uses the model of this article for video recognition attacks.

[0067] Test Results: The feasibility results analysis of the model in a closed world is shown in Table 3 below:

[0068] Table 3 Comparison between the Method of This Article and the Baseline Model in a Closed World

[0069]

[0070] Result analysis: The experimental results are shown in Table 3. In the closed-world test, the f1_score of the model proposed in the present invention reached 0.891 and 0.923 on the YouTube and Dailymotion datasets respectively, far higher than other baseline models, reflecting the effectiveness of the model of the present invention in video recognition in the Tor environment.

[0071] 2. Test Case 2:

[0072] Test objective: The performance of the model of this method and other baseline models in video classification detection in the Tor environment in the open world is used to verify the superiority of this model over other models in detection performance.

[0073] Test environment: Model training and evaluation experiments are carried out in an environment with an i7 10700F CPU, 32 GB RAM, and an RTX 4090 GPU.

[0074] Test method: The accuracy, precision, and recall rate of the experimental results are evaluated according to the confusion matrix, and the calculation formulas are as follows:

[0075]

[0076]

[0077] Parameter settings: As shown in Table 4:

[0078] Table 4 Hyperparameter settings for Test 2

[0079] Learning rate Number of CNN layers N Convolution kernel size Dropout Epoch 0.002 3 3 0.2 50

[0080] Dataset description: The dataset used in this test is the same as the dataset used in Test 1.

[0081] Experiment description: In actual situations, users can access any videos on the Internet, and most of these videos are not covered by the training set of the model. We call them "unknown videos". In this case, the attacker first determines whether the traffic corresponds to the target type. If so, the attacker further identifies the specific video. If not, the attacker marks it as an "unknown video" and terminates the attack. We collected the traffic of unknown videos, and the ratio of its quantity to the closed-world dataset is 10:1, thus creating an open-world dataset. We trained in the closed-world dataset and then tested in the open-world dataset.

[0082] Test results: The experimental results of the model in the open world are shown in Table 5 below:

[0083] Table 5 Comparison between the method in this paper and the baseline model in the open world

[0084]

[0085] Result analysis: We conducted experiments on the two collected datasets respectively and compared the results with the benchmark model. The performance of the model of the present invention in the open-world dataset is slightly lower than that in the closed-world. Among them, the f1_score in the YouTube dataset is 0.867, and the f1_score in the Dailymotion dataset is 0.904, both of which are much higher than the sub-optimal model, indicating that the model of the present invention has high video recognition effectiveness in the actual environment of Tor.

[0086] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches all fall within the protection scope of the claims of the present invention.

Claims

1. A method for identifying encrypted video in a Tor environment, characterized in that , including the following steps: S1, traffic collection: the client uses the Tor network to play the video, and the supervisor uses the wireshark tool to capture Tor traffic in real time at the LAN entrance; S2, ADU-burst length sequence extraction: When the video is playing, the client will receive a burst of application layer data units (Application Data Unit, ADU) for a resource request, which is named as an ADU-burst. According to the two characteristics of encrypted video traffic in the Tor environment obtained by analysis, "stream splitting transmission" and "request obfuscation", two segmentation conditions are set for each stream in the Tor environment: 1) The current ADU-burst has been transmitted; 2) The next ADU-burst arrives after a certain time threshold after the next request. The TLS record stream is segmented to obtain approximate ADU-burst lengths, and a preliminary feature ADU-burst length sequence is formed for the next stage of data training and matching. This ADU-burst segmentation strategy is named TREFSiT. S3, upstream feature extraction: The model uses 1DCNN to perform secondary feature extraction on the ADU-burst length sequence obtained in S2. The 1DCNN model effectively fuses sequences of a certain length through a convolution kernel. The 1DCNN model is used upstream to perform secondary feature extraction to alleviate the impact of complex environments on ADU-burst segmentation, while retaining the strong correlation between the ADU-burst length sequence and the original video content; S4, downstream task classification: After noise filtering by the upstream feature extractor, the obtained feature vector is input into multiple types of classifiers, and the classifier with the best classification effect is selected as the final classifier.

2. The method for identifying encrypted video in a Tor environment as claimed in claim 1, characterized in that: Step S2 specifically includes the following steps: S2-1: By analyzing the traffic transmission data in the Tor environment and comparing it with the non-Tor environment, S2-2: Based on S2-1, an ADU-burst segmentation strategy TREFSiT is proposed. It mainly sets two judgment conditions for each request: 1) the current ADU-burst has been transmitted; 2) the next ADU-burst arrives after a certain time threshold after the current request. The time threshold Δtime of the second request is set to 250ms. S2-3: In response to the content proposed in S2-2, TREFSiT is used to divide the actual data flow of TLS records (carried by the TCP protocol) to obtain the ADU-burst sequence. The judgment method for the first condition is to determine whether the last TLS record of a burst transmission is complete. If it is met, the ADU-burst has been transmitted. For the second condition, the time threshold of the next request Δtime of each stream is judged, requiring the next ADU-burst data to arrive after a certain time threshold expected after the request. Only when both conditions are met will the TLS record stream be segmented, and the encrypted payload between two qualified requests is added as an element of the ADU-burst segment.

3. The encrypted video recognition method in a Tor environment as claimed in claim 2, characterized in that: The TLS record header occupies 5 bytes, including 1 byte of content type, 2 bytes of TLS version and 2 bytes of length. Assume that the fixed length of the header is l0. Each data packet begins with control information of multiple underlying protocols including Ethernet frame protocol, IP and TCP, totaling 54 bytes, as shown in the following equations (1)-(2): l0＝5 (1) only ph =54 (2) Let the position of the first byte in the TLS record header be pos, minus the length of the underlying protocol header, len ph Get len tail , that is, the remaining tail data length of the previous TLS record in front of the data packet, as shown in formula (3), only tail =message only ph (3) According to the TCP protocol design, the sequence number of the payload start is read from the control information in the header and set to seq p , seq p with len tail Add it to l0 to get the sequence number seq of the first byte of the TLS encrypted data td , while seq td Add the encrypted data length len td It is the sequence number of the first byte of the next received TLS record header, recorded as seq ntd , as shown in formula (4)-(5), seq td =seq p +len tail +l0 (4) seq ntd =seq td +len td (5) If P is followed by a TLS record, then seq ntd It should be the sequence number of the first byte of the next TLS record header; if there is no TLS record after P, that is, the transmission has ended, then seq ntd It should be equal to the TCP field value next sequence number of the last packet in the TLS record. Therefore, if seq ntd If it is equal to the next sequence number, it means that the transmission is complete.

4. The method for identifying encrypted video in a Tor environment as claimed in claim 3, characterized in that: Step S3, 1DCNN model training step, includes the following sub-steps; Step S3-1, suppose the L-1th convolutional layer has N l-1 feature map, represented as Then, for the kth feature map of the Lth layer, its input is the sum of all feature maps of the previous layer convolved with the corresponding convolution kernel, as shown in formula (6): Then the hyperbolic tangent function (tanh) is used for nonlinear activation: The model is able to express nonlinear relationships and improve the discriminative performance of the model. Finally, the activated feature map is downsampled using the maximum pooling (MaxPooling), which is expressed as: The pooled feature map is used as the input of the next convolutional layer, and the above operation is repeated. The loss function of 1DCNN model training is as follows, as shown in formula (7), where C is the number of categories, y c is the true label, The probability predicted by the model is:

5. The method for identifying encrypted video in a Tor environment as claimed in claim 1, characterized in that: In step S4, after the pre-training stage of the upstream task, the ADU-burst length vector after secondary extraction is obtained to perform a specific video classification task, and the classification models include logistic regression, random forest, SVM and decision tree.

Images