A traffic identification method, device and electronic equipment

Abstract

The application provides a traffic identification method and device and electronic equipment, which can train gradient boosting decision tree models and logistic regression algorithm models capable of identifying anonymous traffic by using the collected derived features of the traffic, and process the data packets of encrypted traffic to be identified by using the gradient boosting decision tree models and logistic regression algorithm models capable of identifying anonymous traffic, so as to determine whether the encrypted traffic is anonymous traffic, thereby quickly and accurately identifying whether the traffic is anonymous communication traffic.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a traffic identification method, apparatus, and electronic device. Background Technology

[0002] Currently, with the continuous development of information technology, various security problems such as data breaches and rampant malware are becoming increasingly serious. To improve network security, it is necessary to identify anonymous communication traffic on the internet.

[0003] If anonymous communication traffic is encrypted during the communication process, then conventional network traffic identification technologies will be unable to identify the encrypted anonymous communication traffic. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a traffic identification method, apparatus, and electronic device.

[0005] In a first aspect, embodiments of the present invention provide a traffic identification method, including:

[0006] Collect data packets from both anonymous and non-anonymous traffic.

[0007] The anonymous traffic data packets are parsed to obtain the first data packet information of the anonymous traffic data packets, and the anonymous traffic data packets are parsed to obtain the second data packet information of the anonymous traffic data packets;

[0008] The first data packet information and the second data packet information are subjected to feature derivation to obtain the traffic derived feature information; the traffic derived feature information includes: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total number of downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, total data packet length variance, total uplink data packet length variance, and total downlink data packet length variance;

[0009] The gradient boosting decision tree model and the logistic regression algorithm model are trained using the derived feature information of the traffic to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

[0010] When a data packet of encrypted traffic to be identified is obtained, a gradient boosting decision tree model and a logistic regression algorithm model, which are capable of identifying anonymous traffic, are used to identify whether the data packet of encrypted traffic is anonymous traffic.

[0011] Secondly, embodiments of the present invention also provide a traffic flow identification device, comprising:

[0012] The acquisition module is used to collect data packets from both anonymous and non-anonymous traffic.

[0013] The parsing module is used to parse the data packets of the anonymous traffic to obtain the first data packet information of the anonymous traffic data packets, and to parse the data packets of the anonymous traffic to obtain the second data packet information of the anonymous traffic data packets;

[0014] The derivation module is used to perform feature derivation on the first data packet information and the second data packet information to obtain derived feature information of the traffic; the derived feature information of the traffic includes: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total number of downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, variance of total data packet length, variance of total uplink data packet length, and variance of total downlink data packet length;

[0015] The training module is used to train the gradient boosting decision tree model and the logistic regression algorithm model using the derived feature information of the traffic, so as to obtain the gradient boosting decision tree model and the logistic regression algorithm model that can identify anonymous traffic.

[0016] The identification module is used to identify whether the encrypted traffic data packet is anonymous traffic when the encrypted traffic data packet is acquired, using a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

[0017] Thirdly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the method described in the first aspect.

[0018] Fourthly, embodiments of the present invention also provide an electronic device, the electronic device including a memory, a processor and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor using the steps of the method described in the first aspect above.

[0019] In the solutions provided by the first to fourth aspects of the embodiments of the present invention, the collected anonymous traffic data packets and non-anonymous traffic data packets are parsed and feature derived to obtain traffic derived feature information including the total number of uplink data packets, the total number of downlink data packets, the total number of uplink payload data packets, the total number of downlink payload data packets, the total uplink load, the total downlink load, the flow duration, the average uplink load, the average downlink load, the minimum uplink load, the minimum downlink load, the maximum uplink load, the maximum downlink load, the total number of packets, the variance of the total data packet length, the variance of the total uplink data packet length, and the variance of the total downlink data packet length. The above-mentioned traffic derived feature information is then used to develop a gradient boosting decision tree model capable of identifying anonymous traffic. The algorithm trains a gradient boosting decision tree model and a logistic regression algorithm model to identify encrypted traffic. When a data packet of encrypted traffic to be identified is acquired, the algorithm uses a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic to identify whether the data packet of encrypted traffic is anonymous traffic. Compared with the methods in related technologies that cannot identify anonymous traffic, the algorithm can use the derived features of the collected traffic to train a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic. The algorithm then uses the gradient boosting decision tree model and the logistic regression algorithm model that can identify anonymous traffic to process the data packet of encrypted traffic to be identified in order to determine whether the encrypted traffic is anonymous traffic. This allows for fast and accurate identification of whether the traffic is anonymous communication traffic.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of a traffic identification method provided in Embodiment 1 of the present invention is shown;

[0023] Figure 2 A schematic diagram of the structure of a flow identification device provided in Embodiment 2 of the present invention is shown;

[0024] Figure 3 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention is shown. Detailed Implementation

[0025] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0026] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0028] Currently, with the continuous development of information technology, various security problems such as data breaches and rampant malware are becoming increasingly serious. To improve network security, it is necessary to identify anonymous communication traffic on the internet.

[0029] Current network traffic identification technologies mainly include port-based identification, application protocol feature-based identification, and behavior-based identification. Port-based traffic identification primarily uses simple port mapping to identify traffic. With technological advancements and to prevent interference from port spoofing techniques, Deep Packet Inspection (DPI) technology has emerged. DPI uses payload feature strings within traffic packets for traffic detection. Both port-based and DPI methods are packet-based traffic identification techniques; therefore, they become ineffective when communication is encrypted.

[0030] In other words, if anonymous communication traffic is encrypted during the communication process, then conventional network traffic identification technologies will be unable to identify the encrypted anonymous communication traffic.

[0031] Based on this, this embodiment proposes a traffic identification method, device, and electronic device. By analyzing and derivation of data packets from both anonymous and non-anonymous traffic, derived traffic feature information is obtained, including the total number of uplink data packets, the total number of downlink data packets, the total number of uplink payload data packets, the total number of downlink payload data packets, the total uplink load, the total downlink load, the flow duration, the average uplink load, the average downlink load, the minimum uplink load, the minimum downlink load, the maximum uplink load, the maximum downlink load, the total number of packets, the variance of the total data packet length, the variance of the total uplink data packet length, and the variance of the total downlink data packet length. This traffic feature information is then used to train a gradient boosting decision tree model and a logistic regression algorithm model capable of identifying anonymous traffic. When a data packet of encrypted traffic to be identified is obtained, the gradient boosting decision tree model and the logistic regression algorithm model are used to identify whether the encrypted traffic data packet is anonymous traffic, thereby enabling rapid and accurate identification of whether traffic is anonymous communication traffic.

[0032] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Example 1

[0034] This embodiment proposes a traffic identification method, in which the server is the executing entity.

[0035] See Figure 1 The flowchart shown illustrates a traffic identification method. This embodiment proposes a traffic identification method, which includes the following specific steps:

[0036] Step 100: Collect data packets of anonymous traffic and data packets of non-anonymous traffic.

[0037] In step 100 above, the tcpdump tool or the Wireshark tool is used to collect data packets of anonymous traffic and non-anonymous traffic from the Internet.

[0038] The specific process by which the tcpdump tool or the Wireshark tool collects anonymous and non-anonymous communication traffic is existing technology and will not be described in detail here.

[0039] Both the anonymous traffic and the non-anonymous communication traffic are encrypted traffic.

[0040] The anonymous traffic refers to traffic that has been encrypted and obfuscated during communication.

[0041] The non-anonymous traffic refers to traffic that is not encrypted during communication.

[0042] Step 102: Parse the data packets of the anonymous traffic to obtain the first data packet information of the anonymous traffic data packets, and parse the data packets of the anonymous traffic to obtain the second data packet information of the anonymous traffic data packets.

[0043] In step 102 above, the first data packet information includes, but is not limited to: the uplink and downlink identifiers of the anonymous traffic data packet, the data packet length of the anonymous traffic data packet, the data packet start time of the anonymous traffic data packet, and the data packet payload length of the anonymous traffic data packet.

[0044] The second data packet information includes, but is not limited to: the uplink and downlink identifiers of the non-anonymous traffic data packet, the data packet length of the non-anonymous traffic data packet, the data packet start time of the non-anonymous traffic data packet, and the data packet payload length of the non-anonymous traffic data packet.

[0045] Step 104: Perform feature derivation on the first data packet information and the second data packet information to obtain derived feature information of the traffic; the derived feature information of the traffic includes: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total number of downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, variance of total data packet length, variance of total uplink data packet length, and variance of total downlink data packet length.

[0046] In step 104 above, in order to obtain the derived feature information of the traffic, the following steps (1) to (11) can be performed:

[0047] (1) Count the total number of data packets of the anonymous traffic and the data packets of the non-anonymous traffic;

[0048] (2) Count the number of first empty data in the uplink and downlink identifiers of anonymous traffic data packets and the uplink and downlink identifiers of non-anonymous traffic data packets, count the number of second empty data in the data packet length of anonymous traffic data packets and the data packet length of non-anonymous traffic data packets, count the number of third empty data in the data packet start time of anonymous traffic data packets and the data packet start time of non-anonymous traffic data packets, and count the number of fourth empty data in the data packet payload length of anonymous traffic data packets and the data packet payload length of non-anonymous traffic data packets.

[0049] (3) When the ratio of the number of the first empty data to the total number of data packets is less than or equal to the ratio threshold, the first empty data in the uplink and downlink identifiers of the anonymous traffic data packets and the uplink and downlink identifiers of the non-anonymous traffic data packets in the first data packet information is filled using the uplink and downlink identifiers.

[0050] (4) When the ratio of the number of the second empty data to the total number of data packets is less than or equal to the ratio threshold, calculate the average data packet length of the anonymous traffic data packet and the average data packet length of the non-anonymous traffic data packet, and use the calculated average data packet length to fill the second empty data in the data packet length of the anonymous traffic data packet and the data packet length of the non-anonymous traffic data packet.

[0051] (5) When the ratio of the number of the third empty data to the total number of data packets is less than or equal to the ratio threshold, calculate the average data packet start time of the anonymous traffic data packet and the data packet start time of the non-anonymous traffic data packet, and use the calculated average data packet start time to fill the third empty data in the data packet start time of the anonymous traffic data packet and the data packet start time of the non-anonymous traffic data packet.

[0052] (6) When the ratio of the number of fourth empty data to the total number of data packets is less than or equal to the ratio threshold, calculate the average data packet payload length of anonymous traffic data packets and the average data packet payload length of non-anonymous traffic data packets, and use the calculated average data packet payload length to fill the fourth empty data in the data packet payload length of anonymous traffic data packets and the data packet payload length of non-anonymous traffic data packets.

[0053] (7) Use the NumPy tool and the Pandas tool to perform feature derivation on the padded first data packet information and the padded second data packet information to obtain traffic derived feature information; wherein, the traffic derived feature information includes: derived feature information of anonymous traffic data packets and derived feature information of non-anonymous traffic data packets.

[0054] (8) When the ratio of the number of the first empty data to the total number of data packets is greater than the ratio threshold, the uplink and downlink identifiers of the anonymous traffic data packets in the first data packet information are discarded, and the uplink and downlink identifiers of the non-anonymous traffic data packets in the second data packet information are discarded.

[0055] (9) When the ratio of the number of the second empty data to the total number of data packets is greater than the ratio threshold, the data packet length of the anonymous traffic data packet in the first data packet information is discarded, and the data packet length of the non-anonymous traffic data packet in the second data packet information is discarded.

[0056] (10) When the ratio of the number of the third empty data to the total number of data packets is greater than the ratio threshold, the start time of the anonymous traffic data packets in the first data packet information is discarded, and the start time of the non-anonymous traffic data packets in the second data packet information is discarded.

[0057] (11) When the ratio of the number of fourth empty data to the total number of data packets is greater than the ratio threshold, the data packet payload length of the anonymous traffic data packet in the first data packet information is discarded, and the data packet payload length of the non-anonymous traffic data packet in the second data packet information is discarded.

[0058] In step (2) above, the first empty data refers to the case where the uplink and downlink identifiers of anonymous traffic data packets and the uplink and downlink identifiers of non-anonymous traffic data packets are missing values.

[0059] The second empty data refers to the case where the length of the anonymous traffic data packet and the length of the non-anonymous traffic data packet are missing values.

[0060] The third blank data refers to the case where the start time of anonymous traffic data packets and the start time of non-anonymous traffic data packets are missing values.

[0061] The fourth blank data refers to the case where the payload length of anonymous traffic data packets and the payload length of non-anonymous traffic data packets are missing values.

[0062] In step (3) above, the ratio threshold is pre-stored in the server.

[0063] The ratio threshold can be set to any value between 0.45 and 0.6, which will not be elaborated here.

[0064] Filling the first empty data in the uplink and downlink identifiers of anonymous traffic data packets and non-anonymous traffic data packets in the first data packet information using uplink and downlink identifiers means filling the missing values ​​in the uplink and downlink identifiers of anonymous traffic data packets and non-anonymous traffic data packets in the first data packet information using known uplink and downlink identifiers.

[0065] The known uplink and downlink identifiers are pre-stored on the server.

[0066] In step (4) above, filling the second empty data in the data packet length of anonymous traffic data packets and the data packet length of non-anonymous traffic data packets with the calculated average data packet length means filling the missing values ​​in the data packet length of anonymous traffic data packets and the data packet length of non-anonymous traffic data packets with the calculated average data packet length.

[0067] In step (5) above, filling the third empty data in the packet start time of anonymous traffic packets and the packet start time of non-anonymous traffic packets with the calculated average packet start time means filling the missing values ​​in the packet start time of anonymous traffic packets and the packet start time of non-anonymous traffic packets with the calculated average packet start time.

[0068] In step (6) above, filling the fourth empty data in the packet payload length of anonymous traffic packets and the packet payload length of non-anonymous traffic packets with the calculated average packet payload length means filling the missing values ​​in the packet payload length of anonymous traffic packets and the packet payload length of non-anonymous traffic packets with the calculated average packet payload length.

[0069] In step (7) above, using the combination of NumPy and Pandas tools to perform feature derivation on the padded first data packet information and the padded second data packet information means first using NumPy tools to process the padded first data packet information and the padded second data packet information to obtain intermediate results of feature derivation, and then using Pandas tools to process the intermediate results of feature derivation to obtain the derived feature information of traffic.

[0070] The specific process of using a combination of NumPy and Pandas tools to perform feature derivation on the padded first and second data packet information to obtain the derived feature information of the traffic is existing technology and will not be elaborated here.

[0071] In step (8) above, uplink and downlink identifiers are discarded, which means that uplink and downlink identifiers are not used in the feature derivation process.

[0072] In step (9) above, the data packet length is discarded, which means that uplink and downlink identifiers are not used in the feature derivation process.

[0073] In step (10) above, the packet start time is discarded, which means that the packet start time is not used in the feature derivation process.

[0074] In step (11) above, the packet payload length is discarded, which means that the packet payload length is not used in the feature derivation process.

[0075] After obtaining the derived feature information of the traffic through step 104 above, the following step 106 can be performed to train the gradient boosting decision tree model and the logistic regression algorithm model using the derived feature information of the traffic, so as to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

[0076] Step 106: Use the derived feature information of the traffic to train the gradient boosting decision tree model and the logistic regression algorithm model to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

[0077] In step 106 above, the gradient boosting decision tree model and logistic regression algorithm model that can identify anonymous traffic is a model that combines the logistic regression algorithm model (LR) and the gradient boosting decision tree model (GBDT) to identify anonymous traffic.

[0078] The idea behind training a gradient boosting decision tree model and a logistic regression model capable of identifying anonymous traffic is to use GBDT (Gradient Boosting Decision Tree) to generate feature values, and then use LR (Logistic Regression) to fit these feature values. GBDT consists of multiple trees; a decision tree is constructed, and another decision tree is built based on the residuals of the existing model and the actual sample output, iterating continuously. Each iteration produces a classification feature with a large gain; therefore, the feature space is as large as the number of leaf nodes in the GBDT tree. The feature values ​​generated by GBDT are used as input to the LR model, ultimately training a model to identify anonymous traffic. Since the final output of GBDT is a high-dimensional sparse vector, not a dense vector, the high-dimensional sparse vector fed into LR from the numerous trees and leaf nodes offers the following advantages:

[0079] 1. Sparse vector inner product multiplication is fast, easy to calculate and store;

[0080] 2. Discretized features have a strong ability to generalize to outlier data;

[0081] 3. The LR model is a generalized linear model with limited expressive power. Discretizing the single variable into n variables is equivalent to introducing nonlinearity into the model, which can improve the model's expressive power.

[0082] 4. After discretization, feature crossing can be performed;

[0083] 5. The trained recognition model will be more stable.

[0084] The specific process of training the gradient boosting decision tree model and the logistic regression algorithm model using the derived feature information of the traffic to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic is existing technology and will not be elaborated here.

[0085] Step 108: When the data packet of the encrypted traffic to be identified is obtained, the gradient boosting decision tree model and logistic regression algorithm model, which can identify anonymous traffic, are used to identify whether the encrypted traffic is anonymous traffic.

[0086] Here, in order to identify whether the traffic is anonymous, the following steps (1) to (5) can be performed:

[0087] (1) Parse the data packets of the encrypted traffic to obtain the encrypted data packet information of the encrypted traffic data packets;

[0088] (2) Perform feature derivation on the encrypted data packet information to obtain the derived feature information of the encrypted traffic;

[0089] (3) In the gradient boosting decision tree model that uses the derived feature information of the encrypted traffic to identify the anonymous traffic, a high-dimensional sparse vector of the encrypted traffic is obtained;

[0090] (4) Input the high-dimensional sparse vector of the encrypted traffic into a logistic regression algorithm model that can identify anonymous traffic to obtain the probability value that the encrypted traffic is anonymous traffic.

[0091] (5) When the probability value is greater than the probability threshold, the encrypted traffic is determined to be anonymous traffic.

[0092] In step (1), the encrypted data packet information includes, but is not limited to: the uplink and downlink identifiers of the encrypted traffic data packets, the data packet length, the data packet start time, and the data packet payload length.

[0093] In step (2), the process of obtaining the derived feature information of the encrypted traffic is similar to the process of performing feature derivation on the first data packet information and the second data packet information to obtain the derived feature information of the traffic in step 104 above, and will not be described again here.

[0094] The content described in steps (3) to (4) above is the process by which the gradient boosting decision tree model and the logistic regression algorithm model, which are capable of identifying anonymous traffic, process the encrypted traffic.

[0095] The specific process of obtaining the high-dimensional sparse vector of the encrypted traffic in the gradient boosting decision tree model that identifies anonymous traffic by using the derived feature information of the encrypted traffic is existing technology and will not be described in detail here.

[0096] The specific process of inputting the high-dimensional sparse vector of the encrypted traffic into a logistic regression algorithm model that can identify anonymous traffic to obtain the probability value of the encrypted traffic being anonymous traffic is existing technology and will not be elaborated here.

[0097] In step (5), the probability threshold is pre-stored in the server. The probability threshold can be set to any value between 0.6 and 0.85, which will not be elaborated here.

[0098] In summary, this embodiment proposes a traffic identification method. By parsing and derivation of data packets from both anonymous and non-anonymous traffic, derived traffic features are obtained, including: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, variance of total data packet length, variance of total uplink data packet length, and variance of total downlink data packet length. This derived traffic feature information is then used to develop a gradient boosting decision tree model and logical... The algorithm model is trained using a logistic regression model. When a data packet of encrypted traffic to be identified is acquired, a gradient boosting decision tree model and a logistic regression algorithm model, which can identify anonymous traffic, are used to identify whether the data packet of encrypted traffic is anonymous traffic. Compared with the methods in related technologies that cannot identify anonymous traffic, the gradient boosting decision tree model and the logistic regression algorithm model, which can identify anonymous traffic, can be trained using the derived features of the collected traffic. The gradient boosting decision tree model and the logistic regression algorithm model, which can identify anonymous traffic, are then used to process the data packet of encrypted traffic to be identified in order to determine whether the encrypted traffic is anonymous traffic. This allows for fast and accurate identification of whether the traffic is anonymous communication traffic.

[0099] Example 2

[0100] This embodiment proposes a traffic identification device for executing the traffic identification method proposed in Embodiment 1 above.

[0101] See Figure 2 The diagram shown illustrates the structure of a flow identification device. This embodiment proposes a flow identification device, comprising:

[0102] The acquisition module 200 is used to acquire data packets of both anonymous and non-anonymous traffic.

[0103] The parsing module 202 is used to parse the data packets of the anonymous traffic to obtain the first data packet information of the anonymous traffic data packets, and to parse the data packets of the anonymous traffic to obtain the second data packet information of the anonymous traffic data packets;

[0104] The derivation module 204 is used to perform feature derivation on the first data packet information and the second data packet information to obtain derived feature information of the traffic; the derived feature information of the traffic includes: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total number of downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, variance of total data packet length, variance of total uplink data packet length, and variance of total downlink data packet length;

[0105] Training module 206 is used to train the gradient boosting decision tree model and the logistic regression algorithm model using the derived feature information of the traffic, so as to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

[0106] The identification module 208 is used to identify whether the encrypted traffic data packet is anonymous traffic when the encrypted traffic data packet is obtained, by using a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

[0107] Specifically, the acquisition module is used for:

[0108] Use tools like tcpdump or Wireshark to collect packets of both anonymous and non-anonymous traffic from the internet.

[0109] Specifically, the first data packet information includes: the uplink and downlink identifiers of the anonymous traffic data packet, the data packet length of the anonymous traffic data packet, the data packet start time of the anonymous traffic data packet, and the data packet payload length of the anonymous traffic data packet.

[0110] The second data packet information includes: the uplink and downlink identifiers of the non-anonymous traffic data packet, the data packet length of the non-anonymous traffic data packet, the data packet start time of the non-anonymous traffic data packet, and the data packet payload length of the non-anonymous traffic data packet.

[0111] The derived module is specifically used for:

[0112] Count the total number of data packets in the anonymous traffic and the non-anonymous traffic;

[0113] The number of first empty data in the uplink / downlink identifiers of anonymous traffic data packets and the uplink / downlink identifiers of non-anonymous traffic data packets is counted; the number of second empty data in the data packet length of anonymous traffic data packets and the data packet length of non-anonymous traffic data packets is counted; the number of third empty data in the data packet start time of anonymous traffic data packets and the data packet start time of non-anonymous traffic data packets is counted; and the number of fourth empty data in the data packet payload length of anonymous traffic data packets and the data packet payload length of non-anonymous traffic data packets is counted.

[0114] When the ratio of the number of the first empty data to the total number of data packets is less than or equal to the ratio threshold, the first empty data in the uplink and downlink identifiers of the anonymous traffic data packets and the uplink and downlink identifiers of the non-anonymous traffic data packets in the first data packet information are filled using uplink and downlink identifiers.

[0115] When the ratio of the number of the second empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet length of the anonymous traffic data packet and the average data packet length of the non-anonymous traffic data packet are calculated, and the second empty data in the data packet length of the anonymous traffic data packet and the data packet length of the non-anonymous traffic data packet are filled with the calculated average data packet length.

[0116] When the ratio of the number of the third empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet start time of the anonymous traffic data packets and the data packet start time of the non-anonymous traffic data packets is calculated, and the calculated average data packet start time is used to fill the third empty data in the data packet start time of the anonymous traffic data packets and the data packet start time of the non-anonymous traffic data packets.

[0117] When the ratio of the number of fourth empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet payload length of anonymous traffic data packets and non-anonymous traffic data packets is calculated, and the fourth empty data in the data packet payload length of anonymous traffic data packets and non-anonymous traffic data packets is filled using the calculated average data packet payload length.

[0118] The NumPy and Pandas tools are combined to perform feature derivation on the padded first data packet information and the padded second data packet information to obtain traffic derived feature information; wherein, the traffic derived feature information includes: derived feature information of anonymous traffic data packets and derived feature information of non-anonymous traffic data packets.

[0119] In summary, this embodiment proposes a traffic identification device that analyzes and derives features from collected anonymous and non-anonymous traffic data packets to obtain derived traffic feature information, including the total number of uplink data packets, the total number of downlink data packets, the total number of uplink payload data packets, the total number of downlink payload data packets, the total uplink load, the total downlink load, the flow duration, the average uplink load, the average downlink load, the minimum uplink load, the minimum downlink load, the maximum uplink load, the maximum downlink load, the total number of packets, the variance of the total data packet length, the variance of the total uplink data packet length, and the variance of the total downlink data packet length. This derived traffic feature information is then used to develop a gradient boosting decision tree model and logical logic for identifying anonymous traffic. The algorithm model is trained using a logistic regression model. When a data packet of encrypted traffic to be identified is acquired, a gradient boosting decision tree model and a logistic regression algorithm model, which can identify anonymous traffic, are used to identify whether the data packet of encrypted traffic is anonymous traffic. Compared with the methods in related technologies that cannot identify anonymous traffic, the gradient boosting decision tree model and the logistic regression algorithm model, which can identify anonymous traffic, can be trained using the derived features of the collected traffic. The gradient boosting decision tree model and the logistic regression algorithm model, which can identify anonymous traffic, are then used to process the data packet of encrypted traffic to be identified in order to determine whether the encrypted traffic is anonymous traffic. This allows for fast and accurate identification of whether the traffic is anonymous communication traffic.

[0120] Example 3

[0121] This embodiment proposes a computer-readable storage medium storing a computer program. When the computer program is run by a processor, it executes the steps of the traffic identification method described in Embodiment 1 above. For a detailed implementation, please refer to Method Embodiment 1, which will not be repeated here.

[0122] In addition, see Figure 3 The diagram shows the structure of an electronic device. This embodiment also proposes an electronic device, which includes a bus 51, a processor 52, a transceiver 53, a bus interface 54, a memory 55, and a user interface 56. The electronic device includes a memory 55.

[0123] In this embodiment, the electronic device further includes: one or more programs stored in memory 55 and executable on processor 52, configured to be executed by the processor to perform the one or more programs for steps (1) to (5):

[0124] (1) Collect data packets of anonymous traffic and data packets of non-anonymous traffic;

[0125] (2) Parse the data packets of the anonymous traffic to obtain the first data packet information of the anonymous traffic data packets, and parse the data packets of the anonymous traffic to obtain the second data packet information of the anonymous traffic data packets;

[0126] (3) Perform feature derivation on the first data packet information and the second data packet information to obtain the derived feature information of the traffic; the derived feature information of the traffic includes: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total number of downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, total data packet length variance, total uplink data packet length variance, and total downlink data packet length variance;

[0127] (4) The gradient boosting decision tree model and the logistic regression algorithm model are trained using the derived feature information of the traffic to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

[0128] (5) When the data packet of the encrypted traffic to be identified is obtained, the gradient boosting decision tree model and logistic regression algorithm model, which can identify anonymous traffic, are used to identify whether the data packet of the encrypted traffic is anonymous traffic.

[0129] Transceiver 53 is used to receive and send data under the control of processor 52.

[0130] The bus architecture (represented by bus 51) can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 52 and memory represented by memory 55. Bus 51 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be further described in this embodiment. Bus interface 54 provides an interface between bus 51 and transceiver 53. Transceiver 53 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. For example, transceiver 53 receives external data from other devices. Transceiver 53 is used to transmit data processed by processor 52 to other devices. Depending on the nature of the computing system, a user interface 56 may also be provided, such as a keypad, display, speaker, microphone, or joystick.

[0131] Processor 52 is responsible for managing bus 51 and general processing, such as running a general-purpose operating system as described above. Memory 55 can be used to store data used by processor 52 during operation.

[0132] Optionally, the processor 52 may be, but is not limited to, a central processing unit, a microcontroller, a microprocessor, or a programmable logic device.

[0133] It is understood that the memory 55 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 55 of the systems and methods described in this embodiment is intended to include, but is not limited to, these and any other suitable types of memory.

[0134] In some implementations, memory 55 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof: operating system 551 and application programs 552.

[0135] The operating system 551 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 552 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 552.

[0136] In summary, this embodiment proposes a computer-readable storage medium and electronic device. By parsing and derivation of data packets from both anonymous and non-anonymous traffic, derived traffic feature information is obtained, including the total number of uplink data packets, the total number of downlink data packets, the total number of uplink payload data packets, the total number of downlink payload data packets, the total uplink load, the total downlink load, the flow duration, the average uplink load, the average downlink load, the minimum uplink load, the minimum downlink load, the maximum uplink load, the maximum downlink load, the total number of packets, the variance of the total data packet length, the variance of the total uplink data packet length, and the variance of the total downlink data packet length. This derived traffic feature information is then used to build a gradient boosting decision tree capable of identifying anonymous traffic. The model and logistic regression algorithm are trained. When a data packet of encrypted traffic to be identified is obtained, the gradient boosting decision tree model and logistic regression algorithm model, which can identify anonymous traffic, are used to identify whether the data packet of encrypted traffic is anonymous traffic. Compared with the methods in related technologies that cannot identify anonymous traffic, the gradient boosting decision tree model and logistic regression algorithm model that can identify anonymous traffic can be trained using the derived features of the collected traffic. The gradient boosting decision tree model and logistic regression algorithm model that can identify anonymous traffic are then used to process the data packet of encrypted traffic to be identified in order to determine whether the encrypted traffic is anonymous traffic. This allows for fast and accurate identification of whether traffic is anonymous communication traffic.

[0137] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A traffic identification method, characterized by, include: Collect data packets from both anonymous and non-anonymous traffic. The anonymous traffic data packets are parsed to obtain the first data packet information of the anonymous traffic data packets, and the anonymous traffic data packets are parsed to obtain the second data packet information of the anonymous traffic data packets; Feature derivation is performed on the first data packet information and the second data packet information to obtain derived feature information of the traffic; The derived characteristic information of the traffic includes: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total number of downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, variance of total data packet length, variance of total uplink data packet length, and variance of total downlink data packet length. The gradient boosting decision tree model and the logistic regression algorithm model are trained using the derived feature information of the traffic to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic. When a data packet of encrypted traffic to be identified is obtained, a gradient boosting decision tree model and a logistic regression algorithm model, which are capable of identifying anonymous traffic, are used to identify whether the data packet of encrypted traffic is anonymous traffic.

2. The method of claim 1, wherein, The data packets for collecting anonymous traffic and the data packets for collecting non-anonymous traffic include: Use tools like tcpdump or Wireshark to collect packets of both anonymous and non-anonymous traffic from the internet.

3. The method according to claim 1, characterized in that, The first data packet information includes: the uplink and downlink identifiers of the anonymous traffic data packet, the data packet length of the anonymous traffic data packet, the data packet start time of the anonymous traffic data packet, and the data packet payload length of the anonymous traffic data packet; The second data packet information includes: the uplink and downlink identifiers of the non-anonymous traffic data packet, the data packet length of the non-anonymous traffic data packet, the data packet start time of the non-anonymous traffic data packet, and the data packet payload length of the non-anonymous traffic data packet; The step of performing feature derivation on the first data packet information and the second data packet information to obtain derived feature information of the traffic includes: Count the total number of data packets in the anonymous traffic and the non-anonymous traffic; The number of first empty data in the uplink / downlink identifiers of anonymous traffic data packets and the uplink / downlink identifiers of non-anonymous traffic data packets is counted; the number of second empty data in the data packet length of anonymous traffic data packets and the data packet length of non-anonymous traffic data packets is counted; the number of third empty data in the data packet start time of anonymous traffic data packets and the data packet start time of non-anonymous traffic data packets is counted; and the number of fourth empty data in the data packet payload length of anonymous traffic data packets and the data packet payload length of non-anonymous traffic data packets is counted. When the ratio of the number of the first empty data to the total number of data packets is less than or equal to the ratio threshold, the first empty data in the uplink and downlink identifiers of the anonymous traffic data packets and the uplink and downlink identifiers of the non-anonymous traffic data packets in the first data packet information are filled using uplink and downlink identifiers. When the ratio of the number of the second empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet length of the anonymous traffic data packet and the average data packet length of the non-anonymous traffic data packet are calculated, and the second empty data in the data packet length of the anonymous traffic data packet and the data packet length of the non-anonymous traffic data packet are filled with the calculated average data packet length. When the ratio of the number of the third empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet start time of the anonymous traffic data packets and the data packet start time of the non-anonymous traffic data packets is calculated, and the calculated average data packet start time is used to fill the third empty data in the data packet start time of the anonymous traffic data packets and the data packet start time of the non-anonymous traffic data packets. When the ratio of the number of fourth empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet payload length of anonymous traffic data packets and non-anonymous traffic data packets is calculated, and the fourth empty data in the data packet payload length of anonymous traffic data packets and non-anonymous traffic data packets is filled using the calculated average data packet payload length. The NumPy and Pandas tools are combined to perform feature derivation on the padded first data packet information and the padded second data packet information to obtain traffic derived feature information; wherein, the traffic derived feature information includes: derived feature information of anonymous traffic data packets and derived feature information of non-anonymous traffic data packets.

4. The method of claim 3, wherein, The step of performing feature derivation on the first data packet information and the second data packet information to obtain derived feature information of the traffic further includes: When the ratio of the number of the first empty data to the total number of data packets is greater than the ratio threshold, the uplink and downlink identifiers of the anonymous traffic data packets in the first data packet information are discarded, and the uplink and downlink identifiers of the non-anonymous traffic data packets in the second data packet information are discarded. When the ratio of the number of the second empty data to the total number of data packets is greater than the ratio threshold, the data packet length of the anonymous traffic data packet in the first data packet information is discarded, and the data packet length of the non-anonymous traffic data packet in the second data packet information is discarded. When the ratio of the number of the third empty data to the total number of data packets is greater than the ratio threshold, the start time of the anonymous traffic data packets in the first data packet information is discarded, and the start time of the non-anonymous traffic data packets in the second data packet information is discarded. When the ratio of the number of fourth empty data to the total number of data packets is greater than the ratio threshold, the data packet payload length of the anonymous traffic data packets in the first data packet information is discarded, and the data packet payload length of the non-anonymous traffic data packets in the second data packet information is discarded.

5. The method of claim 1, wherein, When a data packet of encrypted traffic to be identified is obtained, the algorithm uses a gradient boosting decision tree model and a logistic regression algorithm model, which are capable of identifying anonymous traffic, to identify whether the encrypted traffic is anonymous traffic, including: The encrypted traffic data packets are parsed to obtain the encrypted data packet information of the encrypted traffic data packets; The encrypted data packet information is subjected to feature derivation to obtain the derived feature information of the encrypted traffic; In the gradient boosting decision tree model that uses the derived feature information of the encrypted traffic to identify anonymous traffic, a high-dimensional sparse vector of the encrypted traffic is obtained. The high-dimensional sparse vector of the encrypted traffic is input into a logistic regression algorithm model that can identify anonymous traffic to obtain the probability value that the encrypted traffic is anonymous traffic. When the probability value is greater than the probability threshold, the encrypted traffic is determined to be anonymous traffic.

6. A traffic identification device, characterized by include: The acquisition module is used to collect data packets from both anonymous and non-anonymous traffic. The parsing module is used to parse the data packets of the anonymous traffic to obtain the first data packet information of the anonymous traffic data packets, and to parse the data packets of the anonymous traffic to obtain the second data packet information of the anonymous traffic data packets; The derivation module is used to perform feature derivation on the first data packet information and the second data packet information to obtain derived feature information of the traffic; The derived characteristic information of the traffic includes: total number of uplink data packets, total number of downlink data packets, total number of uplink payload data packets, total number of downlink payload data packets, total uplink load, total downlink load, flow duration, average uplink load, average downlink load, minimum uplink load, minimum downlink load, maximum uplink load, maximum downlink load, total number of packets, variance of total data packet length, variance of total uplink data packet length, and variance of total downlink data packet length. The training module is used to train the gradient boosting decision tree model and the logistic regression algorithm model using the derived feature information of the traffic, so as to obtain a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic. The identification module is used to identify whether the encrypted traffic data packet is anonymous traffic when the encrypted traffic data packet is acquired, using a gradient boosting decision tree model and a logistic regression algorithm model that can identify anonymous traffic.

7. The apparatus of claim 6, wherein, The acquisition module is specifically used for: Use tools like tcpdump or Wireshark to collect packets of both anonymous and non-anonymous traffic from the internet.

8. The apparatus of claim 6, wherein, The first data packet information includes: the uplink and downlink identifiers of the anonymous traffic data packet, the data packet length of the anonymous traffic data packet, the data packet start time of the anonymous traffic data packet, and the data packet payload length of the anonymous traffic data packet; The second data packet information includes: the uplink and downlink identifiers of the non-anonymous traffic data packet, the data packet length of the non-anonymous traffic data packet, the data packet start time of the non-anonymous traffic data packet, and the data packet payload length of the non-anonymous traffic data packet; The derived module is specifically used for: Count the total number of data packets in the anonymous traffic and the non-anonymous traffic; The number of first empty data in the uplink / downlink identifiers of anonymous traffic data packets and the uplink / downlink identifiers of non-anonymous traffic data packets is counted; the number of second empty data in the data packet length of anonymous traffic data packets and the data packet length of non-anonymous traffic data packets is counted; the number of third empty data in the data packet start time of anonymous traffic data packets and the data packet start time of non-anonymous traffic data packets is counted; and the number of fourth empty data in the data packet payload length of anonymous traffic data packets and the data packet payload length of non-anonymous traffic data packets is counted. When the ratio of the number of the first empty data to the total number of data packets is less than or equal to the ratio threshold, the first empty data in the uplink and downlink identifiers of the anonymous traffic data packets and the uplink and downlink identifiers of the non-anonymous traffic data packets in the first data packet information are filled using uplink and downlink identifiers. When the ratio of the number of the second empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet length of the anonymous traffic data packet and the average data packet length of the non-anonymous traffic data packet are calculated, and the second empty data in the data packet length of the anonymous traffic data packet and the data packet length of the non-anonymous traffic data packet are filled with the calculated average data packet length. When the ratio of the number of the third empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet start time of the anonymous traffic data packets and the data packet start time of the non-anonymous traffic data packets is calculated, and the calculated average data packet start time is used to fill the third empty data in the data packet start time of the anonymous traffic data packets and the data packet start time of the non-anonymous traffic data packets. When the ratio of the number of fourth empty data to the total number of data packets is less than or equal to the ratio threshold, the average data packet payload length of anonymous traffic data packets and non-anonymous traffic data packets is calculated, and the fourth empty data in the data packet payload length of anonymous traffic data packets and non-anonymous traffic data packets is filled using the calculated average data packet payload length. The NumPy and Pandas tools are combined to perform feature derivation on the padded first data packet information and the padded second data packet information to obtain traffic derived feature information; wherein, the traffic derived feature information includes: derived feature information of anonymous traffic data packets and derived feature information of non-anonymous traffic data packets.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program, when run by a processor, performs the steps of the method described in any one of claims 1-5.

10. An electronic device, comprising: The electronic device includes a memory, a processor, and one or more programs, wherein the one or more programs are stored in the memory and are configured to, with the processor, perform the steps of the method of any of claims 1-5.

Images