A method for encrypted video recognition in a Tor environment

Abstract

This invention discloses an encrypted video recognition method in a Tor environment, comprising four steps: traffic acquisition, ADU-burst length sequence extraction, upstream feature extraction, and downstream task classification. First, real-time Tor traffic is captured from the network information service center at the local area network entry point. Then, based on the characteristics of Tor video traffic, features are extracted using the corresponding TREFS iT segmentation strategy to obtain the encrypted ADU-burst length sequence. Next, a 1DCNN model is used for noise filtering and secondary feature extraction. Finally, random forest is used for classification. This method proposes an encrypted video recognition approach for the complex environment of Tor. It does not require decryption of the encrypted video; it can identify videos played by clients through the Tor network based on video traffic characteristics for the purpose of monitoring harmful videos. It has wide application scenarios and good monitoring effects.

Description

Technical Field

[0001] This invention belongs to the technical field of network security and mainly relates to an encrypted video recognition method in the Tor environment. Background Technology

[0002] In today's internet environment, personal information is deeply integrated with the internet. Although the widespread adoption of encryption technologies such as TLS 1.3 and HTTPS has significantly improved communication security, metadata such as users' IP addresses and handshake processes can still leak sensitive information. This has prompted more and more users to turn to anonymous communication technologies to protect their personal privacy. Tor, as the most widely used anonymous communication system, boasts over three million daily active users thanks to its unique rerouting and low-latency mechanism.

[0003] Existing security research on the Tor network mainly focuses on traffic type identification and website fingerprinting attacks. However, research on encrypted video traffic identification in the Tor environment is relatively scarce. This research gap deserves serious attention, especially considering the significant role of video traffic in total internet traffic—statistics show that in the first half of 2022, video traffic accounted for 66% of total traffic, and this proportion continues to grow. More importantly, compared to website visits, video viewing choices often contain richer personal preferences and other privacy information, making the security research of encrypted video traffic even more urgent and relevant. Early encrypted video identification mainly focused on manual feature extraction and classifier selection. The most widely used method was to construct a plaintext fingerprint database and collect transmitted fingerprints for matching and identification.

[0004] Existing research on encrypted video recognition mainly focuses on feature extraction and classifier design. Generally, there are two main methods for video recognition: fingerprint matching based on feature construction using expert knowledge and automatic feature extraction and classification based on deep learning.

[0005] For feature-based methods relying on expert knowledge, a typical approach is to pre-build a plaintext fingerprint database, collect traffic data, recover the lengths of all ADU-bursts, and then use an appropriate algorithm to match the resulting length sequences against the fingerprint database. Due to the difficulty of ADU-burst length recovery, some works have attempted to directly construct the database and perform fingerprint matching at the ciphertext level, but this usually results in significant errors. Therefore, the application-layer length recovery method is more preferable. However, in the Tor environment, end-to-end encryption, transmission obfuscation, and other interference mechanisms render existing segmentation strategies and recovery algorithms inapplicable, thus causing existing fingerprint matching methods to fail.

[0006] Meanwhile, researchers are also using deep learning for automated feature extraction, filtering, and classification to enhance the robustness of the recognition process. Some methods collect aggregated traffic length sequences and then employ convolutional neural network (CNN) models; others use conjoined networks to train recurrent neural networks (RNNs) with self-attention mechanisms. However, existing deep learning models are unable to effectively handle the highly dynamic and noisy Tor video transmissions in video recognition, resulting in a significant decrease in classification performance.

[0007] In summary, traditional video recognition methods rely on length restoration methods and segmentation strategies. Existing length restoration methods are not suitable for the multi-layered encryption modes in the Tor environment, and Tor's obfuscated environment renders existing segmentation strategies ineffective. Deep learning-based recognition methods, particularly feature and noise filtering, perform poorly in the Tor environment. Therefore, this paper proposes for the first time a video recognition attack model for the Tor environment—TorVIA. We analyze the transmission characteristics of video streams in the Tor environment, summarize two important transmission features, and design an ADU segmentation strategy suitable for the Tor environment—TREFSiT. Considering Tor's multi-layered encryption mode, accurately restoring the ADU-burst length is extremely difficult, making it impossible to obtain an accurate original video ADU-burst length sequence. This paper uses a 1DCNN model for secondary feature extraction, filters out the influence of obfuscated data on segmentation, preserves the strong correlation between the encrypted video length sequence and the video content, and finally uses the random forest algorithm for video classification. Summary of the Invention

[0008] This invention addresses the identification and classification of encrypted videos under Tor conditions without decryption. It comprises four steps: traffic acquisition, ADU-burst length sequence extraction, upstream feature extraction, and downstream task classification. First, real-time Tor traffic is captured from a network information service center. Then, based on the Tor video traffic, the corresponding TREFSiT segmentation algorithm is used to convert the encrypted video stream data into an encrypted ADU-burst length sequence. Next, a 1DCNN model is used for noise filtering and secondary feature extraction. Finally, a random forest is used for classification. This invention enables video surveillance in a Tor environment, monitoring harmful videos to alert relevant departments for further regulation.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is: an encrypted video recognition method in a Tor environment, comprising the following steps:

[0010] S1, Traffic Collection: The client uses the Tor network to play videos, and the supervisor uses tools such as Wireshark to capture Tor traffic in real time at the local area network entry point;

[0011] S2, ADU-burst length sequence extraction: During video playback, a client's resource request will receive a burst of Application Data Units (ADUs), which are collectively named an ADU-burst. Based on the characteristics of HAS protocol video transmission, the ADU-burst length sequence of an encrypted video stream is strongly correlated with the original video content (fixed resolution) and does not change due to the playback environment or transmission background; therefore, it can be used as a feature for video recognition. Based on the two characteristics of encrypted video traffic under the Tor environment—"split transmission" and "request obfuscation"—we set two segmentation conditions for each stream under the Tor environment: 1) the current ADU-burst has finished transmitting; 2) the next ADU-burst arrives after a certain time threshold following the next request. Segmenting the TLS record stream yields approximate ADU-burst lengths, initially forming a feature ADU-burst length sequence, which can be used for data training and matching in the next stage. We name this ADU-burst segmentation strategy TREFSiT.

[0012] S3, Upstream Feature Extraction: The model in this invention uses a 1DCNN to perform secondary feature extraction on the ADU-burst length sequence obtained in S2. The 1DCNN model can effectively fuse sequences of a certain length using a single convolutional kernel. Therefore, we use a 1DCNN model upstream for secondary feature extraction to mitigate the impact of complex environments on ADU-burst segmentation, while preserving the strong correlation between the ADU-burst length sequence and the original video content.

[0013] S4, Downstream Task Classification: After noise filtering by the upstream feature extractor, we input the obtained feature vectors into various types of classifiers and select the one with the best classification performance as the final classifier.

[0014] As an improvement to the present invention, step S2 specifically includes the following steps:

[0015] S2-1: By analyzing the traffic transmission data in the Tor environment and comparing the differences with the non-Tor environment, we have summarized two major characteristics: 1) Video streaming is not necessarily transmitted as a single stream; sometimes it is transmitted as a dual stream. 2) During video streaming, the client will send obfuscated data that is not requested.

[0016] S2-2: Based on S2-1, we propose an ADU-burst segmentation strategy, TREFSiT, which mainly sets two judgment conditions for each request: 1) the current ADU-burst has finished transmitting; 2) the next ADU-burst arrives after a certain time threshold following this request. We set the time threshold Δtime for the second request to 250ms.

[0017] S2-3: Regarding the points raised in S2-2, we use TREFSiT to divide the actual data stream of TLS records (carried by the TCP protocol) into ADU-burst sequences. The first condition is determined by checking if the last TLS record in a burst transmission is complete (in practice, this requires comparing the TCP sequence number interval with the claimed length of the TLS record to see if the end of the last TCP block is the end of the last TLS block). If this condition is met, the ADU-burst transmission is complete. For the second condition, a time threshold is applied to the next request Δtime for each stream, requiring the next ADU-burst data to arrive after a certain expected time threshold following the request. Only when both conditions are met will TLS record stream segmentation occur, and the encrypted payloads between two qualifying requests are added together as one element of the ADU-burst segment.

[0018] As another improvement of the present invention, the loss function for training the 1DCNN model in step S3 is as follows, where C is the number of classes, and y c For real labels, The probability predicted by the model:

[0019]

[0020] As another improvement of the present invention, in step S4, after the pre-training stage of the upstream task, the ADU-burst length vector after secondary extraction is obtained for a specific video classification task. The classification model includes, but is not limited to, logistic regression, random forest, SVM and decision tree.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] (1) This invention mainly innovates in terms of segmentation strategy. It analyzes the characteristics of video transmission under the Tor environment for the first time and improves upon existing traditional segmentation methods by proposing the innovative TREFSiT segmentation strategy. This strategy uses two conditions to almost precisely segment the ADU-burst for each stream, forming an ADU-burst length sequence that is strongly correlated with the original video content. Based on this, it proposes TorVIA, a video recognition method under the Tor environment, achieving a breakthrough in monitoring video traffic on the dark web without decryption.

[0023] (2) Compared with existing video recognition methods, this method is designed specifically for the characteristics of the Tor environment, exhibiting higher robustness to Tor's obfuscated environments and stronger usability in the Tor environment. For example, in a closed-world YouTube video recognition experiment with 50 videos played 50 times each in an environment with an i7 10700F CPU, 32GB RAM, and an RTX 4090 GPU, the best accuracy rate of existing video recognition methods was 67.3%, while the accuracy rate of this method reached 89.4%, significantly improving recognition accuracy and truly realizing practical Tor video traffic monitoring.

[0024] (3) This method has wide applicability and can be applied to a variety of different video platforms. Experiments show that this method can effectively identify encrypted videos for video platforms that use DASH streaming technology, such as YouTube, and video platforms that use HLS streaming technology, such as Dailymotion, thus demonstrating its comprehensive usability. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating an encrypted video recognition method in a Tor environment according to the present invention. Detailed Implementation

[0026] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0027] Example 1

[0028] A method for encrypted video recognition in a Tor environment, such as Figure 1 As shown, it consists of four parts: the first part is video traffic collection; the second part is ADU-burst length sequence feature extraction; the third part is 1DCNN model training; and the fourth part is downstream encrypted video classification testing.

[0029] This invention selects a video dataset in a multi-platform Tor environment as the test object to test and evaluate the method of this invention, and further describes it in detail.

[0030] A method for recognizing encrypted videos in a Tor environment, specifically including the following steps:

[0031] Step S1, Video Traffic Collection: Connect to the international network via a soft router, and then collect data using Wirshark. In this embodiment, nearly 200GB of video data was collected from the internet, including data from two data platforms using different protocols, YouTube and Dailymotion, with 113GB and 78GB of video data collected respectively. The number of videos identified was 50 for each platform, and each video was collected 50 times. These were known videos used for training and testing. At the same time, 500 unknown videos were collected for each platform as an open-world dataset to simulate a real network environment and test the unknown video categories against the known video categories.

[0032] Step S2, ADU-burst length sequence feature extraction, includes the following sub-steps;

[0033] Step S2-1: By analyzing the characteristics of traffic transmission in the Tor environment and comparing the differences with the non-Tor environment, two main characteristics are summarized: 1) Video stream transmission is not necessarily a single-stream transmission; some may be divided into dual-stream transmission. 2) Request obfuscation will occur on the client side during video stream transmission.

[0034] This example analyzes video data collected from a real Tor environment. We collected approximately 200GB of video data from two different video platforms, YouTube and Dailymotion, and performed statistical analysis. We found that on the Dailymotion platform, under a fixed environment, the ratio of dual-stream to single-stream transmission for the same video was approximately 7:3, while on the YouTube platform, the ratio was approximately 1:4. Furthermore, in dual-stream transmission, the traffic in each channel accounted for approximately half of the total video traffic. Additionally, in the Tor environment, Tor proxy nodes sent an average of 43 requests for obfuscated data to the video service provider between one packet request and the next. This request obfuscation transmission pattern significantly impacts the original ADU-burst segmentation strategy.

[0035] Step S2-2, based on S2-1, proposes an ADU-burst segmentation method, TREFSiT, which mainly sets two conditions for each request: 1) the previous ADU-burst has finished transmitting; 2) the next ADU-burst arrives only after a certain time threshold of the previous request. The time threshold Δtime for the second request is set to 250ms.

[0036] Step S2-3: Regarding the content proposed in S2-2, we determine the first condition of TREFSiT by checking whether the ADU-burst transmission before the request has ended, i.e., verifying whether the end of the last TCP block is the end of the last TLS record. If so, we check the time threshold for the next request Δtime of each stream. Only if both conditions are met will ADU-burst segmentation be performed. The encrypted payloads between the two qualified requests are added together as an element of the ADU-burst segment.

[0037] In this implementation, the TLS record header occupies 5 bytes, including 1 byte for content type, 2 bytes for TLS version, and 2 bytes for length. The fixed length of the header is set to l0. Each data packet begins with control information for multiple underlying protocols, including Ethernet frame protocol, IP, and TCP, totaling 54 bytes, as shown in equations (1)-(2) below.

[0038] l0=5 (1)

[0039] len ph =54 (2)

[0040] Let pos be the position of the first byte in the TLS record header, and subtract len ​​from the length of the underlying protocol header. ph Get len tail That is, the length of the remaining tail data of the previous TLS record before the data packet, as shown in Equation (3).

[0041] len tail =pos-len ph (3)

[0042] According to the TCP protocol design, the sequence number at the start of the payload can be read from the control information in the header and set as seq. p . seq p with len tail Adding l0 to l0 gives the sequence number seq of the first byte of the TLS encrypted data. td And seq td Add the length of the encrypted data (len) td It is the sequence number of the first byte of the next received TLS record header, denoted as seq. ntd As shown in equations (4)-(5).

[0043] seq td =seq p +len tail +l0 (4)

[0044] seqntd =seq td +len td (5)

[0045] If P is immediately followed by a TLS record, then seq ntd It should be the sequence number of the first byte of the next TLS record header; if there are no TLS records after P, meaning the transmission has ended, then seq ntd It should be equal to the next sequence number, which is the TCP field value of the last packet recorded in the TLS record. Therefore, if the sequence number is found... ntd If the number is equal to the next sequence number, it means that the transmission is complete.

[0046] Step S3, 1DCNN model training steps, includes the following sub-steps;

[0047] Step S3-1, in this step, let the (L-1)th convolutional layer have N l-1 A feature mapping, denoted as Next, for the k-th feature map of the L-th layer, its input is the sum of the convolutions of all feature maps of the previous layer with their corresponding convolution kernels, as shown in Equation (6):

[0048]

[0049] Next, a hyperbolic tangent function (tanh) is used for nonlinear activation: This enables the model to express nonlinear relationships, improving its discriminative performance. Finally, max pooling is used to downsample the activated feature maps, as shown below: The pooled feature maps are used as input to the next convolutional layer, and the above operation is repeated. The CNN module of the model in this paper consists of multiple stacked convolutional and pooling layers, aiming to progressively extract high-level features from the input data.

[0050] Video recognition is actually a multi-class classification task, so we use the cross-entropy function as the loss function, as shown in equation (7), where C is the number of classes and y c For real labels, The probability predicted by the model:

[0051]

[0052] Step S4, Downstream Classification Test: Utilize the noise filtering of the feature vector by the upstream 1DCNN to enhance the strong correlation between the ADU-burst sequence and the original video content, and use an appropriate classifier to classify the encrypted video in the downstream classification.

[0053] Test case

[0054] 1. Test Example 1:

[0055] Test objective: To evaluate the performance of our proposed model in video classification and detection within a closed world environment (Tor) compared to other video recognition models, thereby verifying the superiority of our model in detection performance.

[0056] Test environment: Model training and evaluation experiments were conducted in an environment with an i7 10700F CPU, 32GB RAM, and an RTX 4090 GPU.

[0057] Testing method: The accuracy, precision, and recall of the experimental results were evaluated based on the confusion matrix. The calculation formulas are as follows:

[0058]

[0059]

[0060] Parameter settings: As shown in Table 1, the ratio of training set, test set, and validation set is 8:1:1;

[0061] Table 1 Hyperparameter Settings for Test 1

[0062] Learning rate CNN layers N kernel size Dropout Epoch 0.002 3 3 0.2 50

[0063] Dataset Description: The dataset used in this test is a Tor video dataset collected in a real-world environment, as shown in Table 2.

[0064] Table 2 Dataset Description

[0065]

[0066] Test Description: First, the performance of the model and the baseline model was evaluated in detail in a closed world. In this closed world, users only watch videos from the attacker's dataset. The attacker uses the model in this paper to perform video recognition attacks.

[0067] Test Results: The feasibility analysis results of the model in a closed world are shown in Table 3 below:

[0068] Table 3 Comparison of our method and benchmark models in closed worlds.

[0069]

[0070] Results Analysis: The experimental results are shown in Table 3. In the closed-world test, the model proposed in this invention achieved f1 scores of 0.891 and 0.923 on the YouTube and Dailymotion datasets, respectively, which are far higher than other benchmark models, reflecting the effectiveness of the model in video recognition in the Tor environment.

[0071] 2. Test Example 2:

[0072] Test objective: To evaluate the performance of this method's model in video classification and detection within the Tor environment in an open world, comparing it with other baseline models, in order to verify the superiority of this model in detection performance compared to other models.

[0073] Test environment: Model training and evaluation experiments were conducted in an environment with an i7 10700F CPU, 32GB RAM, and an RTX 4090 GPU.

[0074] Testing method: The accuracy, precision, and recall of the experimental results were evaluated based on the confusion matrix. The calculation formulas are as follows:

[0075]

[0076]

[0077] Parameter settings: as shown in Table 4:

[0078] Table 4 Test 2 Hyperparameter Settings

[0079] Learning rate CNN layers N kernel size Dropout Epoch 0.002 3 3 0.2 50

[0080] Dataset description: The dataset used in this test is the same as the dataset used in Test 1.

[0081] Experimental Description: In real-world scenarios, users can access any video on the internet, most of which are not covered in the model's training set; we refer to these as "unknown videos." In this case, the attacker first determines whether the traffic corresponds to the target type. If so, the attacker further identifies the specific video. If not, the attacker marks it as an "unknown video" and terminates the attack. We collected traffic from unknown videos at a ratio of 10:1 to the closed-world dataset, thus creating the open-world dataset. We trained on the closed-world dataset and then tested on the open-world dataset.

[0082] Test results: The experimental performance of the model in the open world is shown in Table 5 below:

[0083] Table 5 Comparison of our method and benchmark models in open-world scenarios.

[0084]

[0085] Results Analysis: We conducted experiments on two collected datasets and compared the results with the benchmark model. The performance of the model of this invention is slightly lower than that of the closed world dataset in the open world dataset. Specifically, the f1_score is 0.867 in the YouTube dataset and 0.904 in the Dailymotion dataset, both of which are much higher than the suboptimal model. This indicates that the model of this invention has high video recognition effectiveness in the real environment of Tor.

[0086] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A method for recognizing encrypted video in a Tor environment, characterized in that... It includes the following steps: S1, Traffic Collection: The client uses the Tor network to play videos, and the supervisor uses the Wireshark tool at the local area network entry point to capture Tor traffic in real time; S2, ADU-burst length sequence extraction: During video playback, a client's resource request will receive a burst of Application Data Units (ADUs), which are collectively named an ADU-burst. Based on the two characteristics of encrypted video traffic in the Tor environment, namely "split transmission" and "request obfuscation," two segmentation conditions are set for each stream in the Tor environment: 1) The current ADU-burst has finished transmitting; 2) The next ADU-burst arrives after a certain time threshold after the next request. Segmenting the TLS record stream yields an approximate ADU-burst length, initially forming a feature ADU-burst length sequence for the next stage of data training and matching. This ADU-burst segmentation strategy is named TREFSiT. S3, upstream feature extraction: The model uses 1DCNN to perform secondary feature extraction on the ADU-burst length sequence obtained in S2. The 1DCNN model effectively fuses sequences of a certain length through a convolutional kernel. The 1DCNN model is used upstream to perform secondary feature extraction to alleviate the impact of complex environment on ADU-burst segmentation, while preserving the strong correlation between ADU-burst length sequence and original video content. S4, Downstream Task Classification: After noise filtering by the upstream feature extractor, the obtained feature vectors are input into various types of classifiers, and the classifier with the best classification effect is selected as the final classifier.

2. The encrypted video recognition method in a Tor environment as described in claim 1, characterized in that: Step S2 specifically includes the following steps: S2-1: By analyzing traffic transmission data in a Tor environment, compare the differences with those in a non-Tor environment. S2-2: Based on S2-1, an ADU-burst segmentation strategy, TREFSiT, is proposed. This strategy sets two conditions for each request: 1) the current ADU-burst has finished transmitting; 2) the next ADU-burst arrives after a certain time threshold following this request. The time threshold for the second request is... Set to 250ms S2-3: Based on the content proposed in S2-2, TREFSiT is used to divide the actual data stream of TLS records into ADU-burst sequences. The first condition is determined by checking if the last TLS record of a burst transmission is complete; if so, the ADU-burst transmission has ended. The second condition is the next request for each stream. A time threshold is determined, requiring the next ADU-burst data to arrive after a certain expected time threshold following the request. Only if both conditions are met will TLS record stream segmentation be performed, and the encrypted payloads between the two qualified requests will be added as an element of the ADU-burst segment.

3. The encrypted video recognition method in a Tor environment as described in claim 2, characterized in that: The TLS record header occupies 5 bytes, including a 1-byte content type, a 2-byte TLS version, and a 2-byte length. Assume the header has a fixed length. Each data packet begins with control information from multiple underlying protocols, including Ethernet frame protocol, IP, and TCP, totaling 54 bytes, as shown in equations (1)-(2) below. Let pos be the position of the first byte in the TLS record header, and subtract the length of the underlying protocol header. get That is, the length of the remaining tail data of the previous TLS record before the data packet, as shown in equation (3). According to the TCP protocol design, the sequence number at the start of the payload is read from the control information in the header and set to... ,Will and and Add them together to get the sequence number of the first byte of the TLS encrypted data. ,and Add the length of encrypted data It is the sequence number of the first byte of the next received TLS record header, denoted as... As shown in equations (4)-(5), If P is immediately followed by a TLS record, then It should be the sequence number of the first byte of the next TLS record header; if there is no TLS record after P, meaning the transmission has ended, then... It should be equal to the next sequence number, the TCP field value of the last packet in the TLS record. Therefore, if it is found... If the number is equal to the next sequence number, it means that the transmission is complete.

4. The encrypted video recognition method in a Tor environment as described in claim 3, characterized in that: Step S3, 1DCNN model training steps, includes the following sub-steps; Step S3-1, assuming the (L-1)th convolutional layer has A feature mapping, denoted as Next, for the k-th feature map of the L-th layer, its input is the sum of the convolutions of all feature maps of the previous layer with their corresponding convolution kernels, as shown in equation (6): Next, a hyperbolic tangent function (tanh) is used for nonlinear activation: This enables the model to express nonlinear relationships, improving its discriminative performance. Finally, max pooling is used to downsample the activated feature maps, as shown below: The pooled feature maps are used as input to the next convolutional layer, and the above operation is repeated. The loss function for training the 1DCNN model is as follows, as shown in equation (7), where C is the number of classes. For real labels, The probability predicted by the model: 。 5. The encrypted video recognition method in a Tor environment as described in claim 1, characterized in that: In step S4, after the pre-training stage of the upstream task, the ADU-burst length vector after secondary extraction is obtained for the specific video classification task. The classification models include logistic regression, random forest, SVM and decision tree.

Images