A method for identifying real-time applications of RTC media streams based on encrypted traffic analysis

Abstract

The present invention discloses a real-time application identification method for real-time communication (RTC) media stream based on encrypted traffic analysis. The method first divides the traffic into traffic blocks with the help of a time window so as to perform classification work according to the traffic of any block; then, in the traffic blocks divided by each time window, the data packets within the unit time are aggregated, and features are extracted from the aggregated data packets, so as to characterize the original traffic data into feature vectors; in order to reduce memory and power consumption, the present invention designs a lightweight one-dimensional convolutional neural network classification model, which automatically learns the deep high-order features of the traffic from the shallow features, and realizes accurate real-time classification of the traffic. The method provided by the present invention is based on a unidirectional data flow, so it is suitable for asymmetric network structures. The method can identify the application to which the media traffic belongs in real time from the media traffic mixed with various applications, and can be used for network traffic analysis and user service quality assurance.

Description

Technical Field

[0001] The present invention relates to a method for identifying real-time application of RTC media stream based on encrypted traffic analysis, and belongs to the technical field of network management. Background Art

[0002] In recent years, under the influence of globalization, remote and online communication has become increasingly popular. Social software for real-time communication (RTC), such as video conferencing and Internet telephony, has been widely adopted.

[0003] RTC media streams are usually generated by voice or video services. They need to transmit a large amount of data in real time, so the requirements for network bandwidth and transmission delay are more stringent. Therefore, many RTC application providers pay higher fees to Internet service providers to ensure that their users receive higher service levels. In this case, real-time and accurate identification of RTC applications is a prerequisite for Internet services to provide differentiated service quality. However, network traffic is often mixed with media streams from various applications, which are very similar due to traffic encryption and service crossover. Therefore, it is difficult to accurately identify the applications carrying these media streams.

[0004] Traffic classification is used to identify the type of traffic. In the early days of the Internet, many scholars devoted themselves to classifying traffic using traditional deep packet inspection techniques and port-based methods. However, traffic encryption and dynamic port allocation have gradually restricted this approach. Current research mainly relies on machine learning or deep learning methods. Machine learning-based methods rely on statistical features of traffic and require feature engineering. The feature acquisition process is complex and time-consuming, and the features have poor stability and short validity period. Although deep learning-based methods avoid complex feature engineering, there are still some problems. First, the features used by many methods are homogeneous. Second, most methods require the entire or most of the data stream as input. In addition, most deep learning models require high computing memory and power, which are not suitable for high-speed networks and real-time scenarios. In addition, many flow classification methods are based on bidirectional data flows. However, the existence of asymmetric routing causes upstream and downstream traffic to pass through different paths, resulting in the possibility that only unidirectional traffic can be captured at a specific node. Therefore, in an asymmetric network environment, flow classification methods based on bidirectional data flows may be damaged or even fail.

[0005] Existing classification methods for RTC traffic are usually limited to one protocol (such as RTP) or application (such as Skype), and cannot be applied to traffic containing multiple protocols or applications. In addition, most of the work only identifies the service type of the RTC media stream and cannot further identify the corresponding application from the media stream. Moreover, few studies can identify different RTC media streams from a large amount of background traffic. For RTC traffic, it is crucial to identify the application type of the media stream as early as possible, but it is still a challenge to identify RTC traffic in real time in a real environment. Summary of the invention

[0006] In order to achieve accurate real-time classification of traffic in a high-speed network environment within a reasonable time and limited memory, the present invention divides the traffic by means of time windows, and can classify traffic only based on the traffic in an arbitrary time window, and combines the time unit to quickly extract stable features from the aggregated data packets within the unit time, and finally uses a lightweight convolutional neural network model to achieve real-time classification of RTC traffic.

[0007] In order to realize the real-time identification of the application to which the RTC media stream belongs in a high-speed network environment, the present invention first collects the RTC media traffic from various applications, and then mixes the RTC traffic with a large amount of background traffic in the high-speed network to simulate the high-speed network scenario. Secondly, the present invention proposes a feature extraction method based on time windows and aggregated data packets in unit time through encrypted traffic analysis. The features are simple and stable, and flexible and fast traffic classification can be achieved. Finally, the present invention uses a supervised deep learning method to train the feature vectors obtained by feature extraction to obtain a classification model, which can be used to classify subsequent high-speed network traffic.

[0008] A method for identifying real-time applications of RTC media streams based on encrypted traffic analysis, the method comprising the following steps:

[0009] Step (1) obtaining the RTC traffic data of the communication between the two parties, and counting the unidirectional data flow according to the five-tuple;

[0010] Step (2) using a sliding time window, divide the traffic into traffic blocks of the same time length;

[0011] Step (3) based on the result of step (2), in each traffic block, the data packets within the unit time are aggregated according to the data flow, the features of the aggregated data packets are extracted, and the feature sequence within the time window is obtained;

[0012] Step (4) A lightweight 1DCNN neural network model is designed, and the feature sequence of the aggregated data packet is used as input for model training, and supervised deep learning is performed to obtain a trained classification model;

[0013] Step (5) uses the classification model in step (4) to classify the test data that has not participated in the training.

[0014] Furthermore, in step (1), the method for counting RTC data stream is as follows:

[0015] (1.1) Select the social software to be classified and capture the data of specific media streams to obtain a labeled media stream dataset of social software, and manually divide the dataset into a training set and a test set;

[0016] (1.2) Count all five-tuple data flows (source IP, source port, destination IP, destination port, and transport protocol) involved in the duration of the user behavior. In order to be applicable to asymmetric network structures, this step only requires counting unidirectional data flows; then add labels based on the type of media flow and the application to which it belongs;

[0017] Furthermore, in step (2), the traffic segmentation method based on the time window is as follows:

[0018] (2.1) First, you need to set the size of the time window and set the time length to T s , starting from the first data packet transmitted during the user's active time and in steps of T s Slide until the last data packet is transmitted and divide the traffic into traffic blocks of equal length. Each block may contain multiple data flows. The time window converts the classification object from the entire flow to the data within a time window, simplifying the classification object and making the classification more flexible. In the i-th traffic block, as shown in formula (1), the set of all data flows counted according to the five-tuple is F i . Flow Set F i The jth flow in is denoted as f ij , represents the jth flow of the i-th traffic block. In formula (2), represents the jth data flow f in the i-th traffic block ij The first packet of ij It is composed of all the data packets of the data flow in the corresponding traffic block;

[0019] F i ={f i1 ,f i2 ,...f ij ,...} (1)

[0020]

[0021] Furthermore, the specific implementation steps of extracting the features of aggregated data packets in the traffic block in step (3) are as follows:

[0022] (3.1) Step (2) divides the traffic into traffic blocks of equal time length, each block may contain multiple data flows. ij , for its unit time T u Aggregate all the data packets in the data flow and calculate the sum of the data packet sizes. Aggregate data packets avoid the instability of the characteristics caused by the change of the maximum transmission unit of the device. The data volume of the aggregated data packet is the characteristic value of the unit time. As shown in formula (3), the data flow f ij The mth eigenvalue of is the mth unit time T u The sum of all packet sizes within and Respectively represent the mth time unit T u Internal data flow ij The first and last packets of

[0023]

[0024] (3.2) For each data flow in the traffic block, count the number of units of time T in chronological order. u The characteristic values ​​of the aggregated data packets, Figure 3 The shaded part indicates that the data flow is active, and the striped part indicates that the data has not yet started to be transmitted or has been transmitted. In order to ensure that the obtained feature sequences are of equal length, Figure 3 The stripe part in is regarded as the transmission of zero-length data packets, and finally the characteristic sequence in formula (4) can be obtained It is composed of the characteristic values ​​of each unit time. In addition, the number of characteristic sequences that can be obtained for each traffic block is equal to the number of data flows in the flow set.

[0025]

[0026] Furthermore, in step (4), a lightweight 1DCNN neural network model is designed and used to train the feature sequence of the aggregated data packets with application and media stream type labels obtained in step (3.2), and the method for obtaining the classification model is as follows:

[0027] (4.1) Divide the labeled feature sequence data obtained in step (3.2) into a training set and a test set for model training;

[0028] (4.2) A lightweight 1DCNN classifier is designed based on the transmission characteristics of RTC traffic, and the model is trained on the training set to obtain a classification model for real-time application classification of RTC media streams.

[0029] Furthermore, in step (5), the method of using the classification model to classify the data of the test set is as follows:

[0030] (5.1) Use the classification model obtained in step (4) to classify the data in the test set.

[0031] Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects.

[0032] (1) The present invention is aimed at social applications of the real-time communication type and proposes a method for real-time identification of applications from media streams based on encrypted traffic analysis. The method is applicable to unidirectional network structures and high-speed network scenarios and has strong practicality and innovation.

[0033] (2) The method of the present invention only needs to detect a traffic block in an arbitrary time window to achieve a high recognition accuracy, the classification is flexible, the feature extraction depends on the data volume of the aggregated data packet, the memory consumption is small and the features are more stable.

[0034] (3) The neural network model designed by the present invention is lightweight enough and can perform application recognition of RTC media streams by automatically learning high-order deep features of feature sequences. The recognition speed is fast and real-time recognition can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Provides an overall framework diagram for real-time application identification of media streams based on encrypted traffic analysis;

[0036] Figure 2 It is a schematic diagram of flow mixing;

[0037] Figure 3 It is a flow chart of feature extraction based on time window and time unit;

[0038] Figure 4 A lightweight one-dimensional convolutional neural network model is constructed;

[0039] Figure 5 The accuracy of model training and testing;

[0040] Figure 6 This is the confusion matrix of the classification results of the test set;

[0041] Figure 7 Confusion matrix of the classification results for the test set with different background stream mixtures. DETAILED DESCRIPTION

[0042] The technical solution provided by the present invention will be described in detail below in conjunction with specific embodiments. It should be understood that the following specific implementation methods are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0043] Specific embodiment: The method for real-time application identification of media streams based on encrypted traffic analysis provided by the present invention can realize feature extraction and application identification of RTC media streams in high-speed network scenarios. The process architecture is shown in the attached Figure 1 shown.

[0044] The following will introduce the implementation methods of each step:

[0045] Step (1) obtaining the RTC traffic data of the communication between the two parties, and counting the unidirectional data flow according to the five-tuple;

[0046] In one embodiment of the present invention, collecting and counting RTC data streams specifically includes the following process:

[0047] (1.1) First, collect the media stream data of audio and video calls / conferences of QQ, WeChat, Telegram, WhatsApp, and Zoom. The types of media streams and applications in this example are shown in Table 1. In order to make the collected traffic data closer to the traffic generated in actual applications, the collection equipment covers different operating systems and application versions. Use Wireshark to capture various types of data. In order to maintain the balance of the data set, each type of media stream is collected for about 9 hours. Finally, the total PCAP file is 32GB, of which 31GB is manually divided into training sets and the rest is test sets.

[0048] Table 1 Data type description table

[0049]

[0050]

[0051] (1.2) In order to evaluate the effectiveness of the method in a real network scenario with a large amount of background traffic, the present invention uses 15 minutes of packet traces of the WIDE backbone network collected by the MAWI working group as background traffic, where the scale of the background traffic used in the training set and the test set is 19.2GB and 21.2GB respectively. The previously collected RTC media traffic is randomly inserted into the background traffic, and the mixing process is achieved by changing the arrival time of the data packets to be inserted. As shown in the attached figure, Figure 2 As shown, a time is randomly generated from the duration of the background traffic, and the arrival time of the RTC data packet to be inserted is uniformly increased by this time value. Unless the duration of the data packet to be inserted is already greater than the duration of the background traffic, this time value should not lengthen the duration of the mixed traffic.

[0052] (1.3) Collect statistics on all five-tuple data flows (source IP, source port, destination IP, destination port, and transport protocol) involved in the duration of the user behavior, remove data flows with less than 50 packets, and add labels based on the type of media flow and the application to which it belongs.

[0053] Step (2) using a sliding time window, divide the traffic into traffic blocks of the same time length;

[0054] In one embodiment of the present invention, the size of the time window T s Set to 10s. Starting from the first data packet transmitted during the user activity time, the number of packets transmitted is incremented by a step of T s Slide until the last data packet is transmitted and divide the traffic into traffic blocks of equal length. Each block may contain multiple data flows. In the i-th traffic block, as shown in formula (1), the set of all data flows counted according to the five-tuple is F i . Flow Set F i The jth flow in is denoted as f ij , represents the jth flow of the i-th traffic block. In formula (2), represents the jth data flow f in the i-th traffic block ij The first packet of ij It is composed of all the data packets of the data flow in the corresponding traffic block;

[0055] F i ={f i1 ,f i2 ,...f ij ,...} (1)

[0056]

[0057] Step (3) in each traffic block, the data packets within the unit time are aggregated according to the data flow, the features of the aggregated data packets are extracted, and the feature sequence within the time window is obtained;

[0058] This step specifically includes the following processes:

[0059] (3.1) According to the time window, the traffic has been divided into traffic blocks of equal time length. Each block may contain multiple data flows. Figure 3 The second flow block includes two data flows, flow1 and flow2. In one embodiment of the present invention, the unit time T is set u The size of is 10ms. For data stream f ij , for unit time T uAggregate all the data packets in the data flow, calculate the sum of the data packet sizes, and the data volume of the aggregated data packet is the characteristic value of the unit time. As shown in formula (3), the data flow f ij The mth eigenvalue of is the mth unit time T u The sum of all packet sizes within and Respectively represent the mth time unit T u Internal data flow ij The first and last packets of

[0060]

[0061] (3.2) For each data flow in the traffic block, count the number of units of time T in chronological order. u The characteristic value of the aggregated data packet. In a traffic block, such as the attached Figure 3 As shown in the striped part, some data streams start transmission later or end transmission earlier, resulting in their duration being shorter than the size of the time window. In order to ensure that the obtained feature sequences are of equal length, Figure 3 The stripe part transmits zero-length data packets, and finally the characteristic sequence in formula (4) can be obtained It is composed of the characteristic values ​​of each unit time. In addition, the number of characteristic sequences that can be obtained for each traffic block is equal to the number of data flows in the flow set.

[0062]

[0063] (3.3) Finally, the number of samples of each data type obtained after feature extraction is shown in Table 2. The last column of Table 2 shows the proportion of each category in the test set to the total test samples. More than 99% of the samples are background traffic, and less than 1% of the samples are various types of RTC media traffic. This provides scenario support for identifying RTC traffic from network traffic containing a large amount of background traffic in this case. In addition, after feature extraction, the amount of RTC data in the training set is converted from the original 31GB to 120MB, which greatly reduces memory overhead.

[0064] Table 2 Dataset overview

[0065]

[0066]

[0067] In step (4), a lightweight 1DCNN neural network model is used to train the aggregated data packet feature sequence with media streams and application types obtained in step (3.2), and the method for obtaining the classification model is as follows:

[0068] (4.1) The feature sequence data of the labeled training set obtained in step (3.2) is used for model training.

[0069] (4.2) Design and use a lightweight 1DCNN classifier for model training to obtain a classification model.

[0070] (4.2.1) The structure of the model is shown in the attached Figure 4 As shown in the figure. In order to automatically extract the feature information contained in the feature sequence of aggregated data packets, we draw on the idea of ​​1DCNN network to process text, regard each length value as the index value of the word in the text vector, use the Embedding layer in Keras to map the length sequence to a fixed-dimensional space, and then use the 1DCNN network to process the mapped feature sequence. 1DCNN uses one CNN layer to extract the features of the output sequence of the Embedding layer, and after global pooling, uses two fully connected layers to complete dimensionality reduction and classification respectively, making the model lightweight enough.

[0071] In step (5), the method of using the classification model to classify the data of the test set is as follows:

[0072] (5.1) Use the classification model obtained in step (4) to classify the data in the test set.

[0073] (5.1.1) This case deploys the model on Intel(R) Xeon(R) Gold 5220R CPU@2.20GHz and NVIDIA RTX A5000 GPU, using Ubuntu 20.04.3LTS as the operating system and 125G memory. In addition, this case uses the TensorFlow environment and Keras library as the background to run the neural network model. The batch size of the model is set to 128, and it is run for 25 rounds, with 1220 batches per round. Figure 5 As shown in the figure, the model converges in about 10 rounds.

[0074] (5.1.2) This case conducted ten experiments, and the overall precision, recall rate and F1 score index can reach more than 99%. Figure 6The confusion matrix shows that the models used can almost completely separate all categories. In order to avoid randomness, this case mixed the application data of the test set with the background flow data collected by the MAWI working group on June 10, 2020, and repeated the above experiment, and obtained similar classification results. The results are shown in the attached figure. Figure 7 shown.

[0075] (5.1.3) Actual campus network data is used for verification. Since the MAWI public dataset data does not carry a payload, it cannot be analyzed in depth. Therefore, this case captures data from the campus network boundary as a verification dataset. The dataset is a 32.6GB 100s packet trace captured at the campus network boundary on November 23, 2021. The bandwidth of the capture interface is 10Gbps, and the network throughput is 1.8Gbps-2.6Gbps.

[0076] This case selected WeChat video call data for further analysis to explore whether the background traffic data predicted to be WeChat video calls actually contains this type of data. First, the original PACP data (with load) of the background traffic predicted to be WeChat video calls was extracted and compared with the known WeChat video call data in the training set. Finally, through deep packet inspection technology, it was found that three data streams were WeChat video call types, which confirmed the availability and effectiveness of this method in actual network environments. In addition, it was successfully verified in other categories.

[0077] (5.1.4) Real-time performance. Table 3 shows that the average prediction speed of the test set is 9.79 Gbps, while the maximum throughput of the test set MAWI background traffic does not exceed 3.4 Gbps. The average recognition speed of the validation data set is 6.12 Gbps, which also exceeds the maximum throughput of the campus network of 2.6 Gbps. This shows that the recognition speed of the model is much faster than the data transmission speed. In addition, for a 10-second traffic block, the feature extraction method of the present invention only takes an average of 0.30 seconds. In addition, feature extraction and model prediction can be performed in parallel, indicating that the method proposed in the present invention can achieve real-time recognition.

[0078] Table 3 Data processing of test set and validation set

[0079] Dataset Data volume Average prediction time Average prediction speed Test Set 21.5GB 17.57s 9.79Gbps Validation set 3.02GB 3.95s 6.12Gbps

[0080] (5.2) The influence of different time window sizes on the recognition results. This case conducts experiments and analysis on the size of the time window. s The experiments were conducted with the time settings of 1s, 3s, 5s, and 10s, and the accuracy results of each category are shown in Table 4 below.

[0081] Table 4 Classification performance of LS-CNN model under different sequence lengths

[0082]

[0083]

[0084] Table 4 shows that when the time window is 5 seconds, 7 seconds and 10 seconds, the accuracy of almost all categories can reach more than 95%. Generally speaking, traffic from the same service or application is more easily confused, but the results in the table show that the present invention overcomes this difficulty. In addition, the data volume of voice streams is smaller than that of video streams, but they can also be well identified from background traffic. In addition, even when the time window is 3s, except for the WhatsApp video call with an accuracy of 90%, the accuracy of the remaining categories can reach more than 94%, which shows that any 3-second traffic block can complete traffic identification. When the time window is 1s, it can be seen from Table 4 that only 37% of the data of qq-voicecall is correctly classified, and as much as 53% of the remaining data is predicted as qq-videocall. Only 58% of the data in zoom-voicemeeting is predicted correctly, and the remaining 25% is classified into zoom-videomeeting. This result shows that confusion mainly occurs in the same application, so without distinguishing the media streams carried by the application, the 1s time window can still achieve a high classification accuracy.

[0085] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A method for identifying real-time applications of RTC media streams based on encrypted traffic analysis, characterized in that: The method comprises the following steps: Step (1) obtaining the RTC traffic data of the communication between the two parties, and counting the unidirectional data flow according to the five-tuple; Step (2) using a sliding time window, divide the traffic into traffic blocks of the same time length; Step (3) based on the result of step (2), in each traffic block, the data packets within the unit time are aggregated according to the data flow, the features of the aggregated data packets are extracted, and the feature sequence within the time window is obtained; Step (4) A lightweight 1DCNN neural network model is designed, and the feature sequence of the aggregated data packet is used as input for model training, and supervised deep learning is performed to obtain a trained classification model; Step (5) uses the classification model in step (4) to classify the test data that has not participated in the training; In step (1), the method for counting RTC data stream is as follows: (1.1) Select the social software to be classified and capture the data of specific media streams to obtain a labeled media stream dataset of social software, and manually divide the dataset into a training set and a test set; (1.2) Collect statistics on all five-tuple data flows involved in the duration of the user behavior, namely, source IP, source port, destination IP, destination port, and transport protocol, and add labels based on the type of media flow and the application to which it belongs. In step (2), the traffic segmentation method based on the time window is as follows: (2.1) First, you need to set the size of the time window and set the time length to T s , starting from the first data packet transmitted during the user's active time, with a step length of T s Slide until the last data packet is transmitted and divide the traffic into traffic blocks of equal length. Each block contains multiple data flows. In the i-th traffic block, as shown in formula (1), the set of all data flows counted by the five-tuple is F i , flow set F i The jth flow in is denoted as f ij , represents the jth flow of the i-th traffic block. In formula (2), represents the jth data flow f in the i-th traffic block ij The first packet of ij It is composed of all the data packets of the data flow in the corresponding traffic block; F i ={f i1 ,f i2 ,...f ij ,...} (1) The specific implementation steps of extracting the features of aggregated data packets within the traffic block in step (3) are as follows: (3.1) Step (2) divides the traffic into traffic blocks of equal time length. Each block contains multiple data flows. For data flow f ij , for its unit time T u Aggregate all the data packets in the data flow, calculate the sum of the data packet sizes, and the data volume of the aggregated data packet is the characteristic value of the unit time, as shown in formula (3). ij The mth eigenvalue of is the mth unit time T u The sum of all packet sizes within and Respectively represent the mth time unit T u Internal data flow ij The first and last packets of (3.2) For each data flow in the traffic block, count the number of units of time T in chronological order. u The characteristic values ​​of the internally aggregated data packets are finally obtained as the characteristic sequence in formula (4): It is composed of the characteristic values ​​of each unit time. In addition, the number of characteristic sequences obtained for each traffic block is equal to the number of data flows in the flow set.

2. According to claim 1, a method for identifying real-time applications of RTC media streams based on encrypted traffic analysis is characterized in that: In step (4), a lightweight 1DCNN neural network model is designed and used to train the feature sequence of the aggregated data packets with application and media stream type labels obtained in step (3.2). The method for obtaining the classification model is as follows: (4.1) Divide the labeled feature sequence data obtained in step (3.2) into a training set and a test set for model training; (4.2) Use the designed lightweight 1DCNN classifier to train the model on the training set and obtain the classification model.

3. The method for identifying real-time applications of RTC media streams based on encrypted traffic analysis according to claim 2, characterized in that: In step (5), (5.1) classifies the data of the test set using the classification model obtained in step (4).

Images