Domain name preposition flow detection method and system based on throughput fluctuation

Abstract

The invention discloses a domain name preposition flow detection method and system based on throughput fluctuation, and belongs to the technical field of network security. In order to solve the problems of low precision and high false alarm in the existing detection technology, domain name preposition flow detection is mainly carried out by combining network flow capturing, data preprocessing, feature extraction and a machine learning algorithm. By mining the throughput fluctuation characteristics of the domain name preposition technology at the session level, the domain name preposition flow and the normal flow can be accurately distinguished, the false alarm rate is reduced, and the detection precision is improved.

Description

Technical Field

[0001] The invention provides a domain name front flow detection method and system based on throughput fluctuation, belonging to the technical field of network security. Background Art

[0002] Domain name prepending is a typical network covert channel technology. Due to its strong concealment, domain name prepending is widely used to hide various malicious network activities. For example, in the APT29 attack, the attacker used domain name prepending technology to secretly access the victim environment for more than two years. In addition, domain name prepending technology is also integrated into spyware to circumvent network traffic inspection. Take meek as an example. As the first mature tool developed based on domain name prepending technology, it is widely used to build network attack channels and spread malicious information. Therefore, shortly after the domain name prepending technology was proposed, it was considered one of the most dangerous attack technologies.

[0003] The concealment of domain name front traffic is closely related to the misuse of content distribution networks (CDNs). CDNs are one of the important components of cloud services, and are mainly used to accelerate the user experience on the Internet. In order to improve the user experience, many websites host their domain names on CDNs. In this case, if the malicious service is hosted on the same CDN, the attacker can access the malicious service under the name of a normal domain name, thereby hiding the real network behavior. For example, when using meek to build a network attack, the entry node of the network may be hosted on the Microsoft CDN service. At this time, the attacker can first establish a TLS connection with Microsoft's well-known domain name microsoft.com, and then within the TLS connection, set the HOST field of the HTTP protocol to the domain name of the network entry to inform the CDN of the real network destination address. Since the HTTP protocol is encrypted by TLS and the TLS connection is only related to benign domain names, malicious traffic can bypass network security checks.

[0004] By establishing a connection with a benign domain name, domain name pre-flow can achieve the effect of disguising normal network traffic. Therefore, finding the difference between domain name pre-flow and normal network traffic is a prerequisite for detecting domain name pre-flow covert channels. Existing studies have found a variety of packet-level camouflage defects. These camouflage defects can distinguish domain name pre-flow from normal network traffic to a certain extent. For example, Field et al. used meek as the research object and found that domain name pre-flow usually contains many short data packets with a length close to 0 bytes, and long data packets with a length of 1418 bytes account for more than 40%. The above-mentioned packet-level camouflage defects can effectively detect camouflaged traffic, but are limited by the fluctuations in statistical characteristics caused by different network environments, causing the method to fail during field deployment.

[0005] Existing methods have made some progress in the research work of identifying domain name prefixes by calculating statistical features of data packet length, time, direction, etc. However, the camouflage defects used in these methods are easily affected by the network environment, resulting in poor robustness, high false alarm rate, and difficulty in deployment and use in actual environments. Existing detection technologies are mainly based on packet-level statistical features, which are easily affected by the network environment and change. The throughput characteristics are related to the business functions of the upper layer. Even if the network status changes, it is difficult to change the business of the application layer because the structure of the network protocol stack determines that the network layer and the application layer are independent. In addition, attackers can bypass detection by adjusting the sending status of the data packet. Therefore, existing detection methods cannot meet the actual needs of network security. Summary of the invention

[0006] The purpose of the present invention is to propose a domain name pre-placement traffic detection method and system based on throughput fluctuation, which can solve the problems of low accuracy and high false alarm of existing detection technology by exploring the camouflage defects of domain name pre-placement technology at the session level and using the defects and machine learning algorithms to detect domain name pre-placement traffic.

[0007] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0008] A method for detecting domain name front-end traffic based on throughput fluctuation comprises the following steps:

[0009] 1) Capture network traffic;

[0010] 2) Preprocess the captured network traffic;

[0011] 3) Extract the direction information and payload data of the network session, calculate the inbound and outbound throughput based on the packet direction, and generate the throughput fluctuation characteristics;

[0012] 4) Build and train a domain name front-end traffic detector based on throughput fluctuation characteristics;

[0013] 5) Deploy a trained domain name front traffic detector to detect unknown attribute traffic in the network, classify and judge the traffic throughput fluctuation characteristics, and identify potential domain name front traffic.

[0014] Furthermore, in step 1), the network traffic is captured in real time online or by loading a saved traffic file offline, and saved in a standard pcap or pcapng format.

[0015] Furthermore, the pre-processing step in step 2) includes:

[0016] Filter data packets without payload and data packets affected by network status;

[0017] Aggregate traffic into independent sessions based on the five-tuple in the data packet, which includes source IP, destination IP, source port, destination port, and transport layer protocol;

[0018] Filter TLS sessions by port filtering and handshake length.

[0019] Furthermore, the filtered no-load data packets include TCP ACK packets used to maintain the network transmission status and data packets unrelated to application layer data; the data packets affected by the network status include retransmission packets and repeated ACK packets.

[0020] Furthermore, filtering TLS sessions through port filtering and handshake length judgment specifically includes: filtering data packets with a target port of 443, eliminating irrelevant traffic, and retaining only sessions that complete the TLS handshake; discarding traffic with a session length shorter than 20 data packets.

[0021] Furthermore, after extracting the direction information and payload data in step 3), a signed integer sequence in bytes is constructed, where the absolute value of the integer in the sequence represents the original data packet length, and the positive and negative signs are used to identify the direction, with negative values ​​representing inbound data packets and positive values ​​representing outbound data packets.

[0022] Furthermore, in step 3), the direction of the data packet is determined according to the source IP prefix and marked as inbound or outbound; then the inbound and outbound throughputs are calculated by accumulating the sizes of data packets in the same direction; and the throughput fluctuation characteristics are extracted based on the changes in the inbound and outbound throughputs.

[0023] Furthermore, in step 4), the domain name pre-traffic detector is a binary classifier, which is trained using the random forest algorithm. The domain name pre-traffic detector labels the network traffic with positive and negative labels based on the throughput fluctuation characteristics, where the positive label represents the domain name pre-traffic and the negative label represents the normal traffic. The training data used consists of normal traffic with labeled data and domain name pre-traffic, where the normal traffic is used as a negative sample and the domain name pre-traffic is used as a positive sample.

[0024] Furthermore, in step 5), after the domain name front traffic detector identifies the domain name front traffic, it takes measures such as reporting, blocking or backing up the traffic to facilitate subsequent tracing and evidence collection.

[0025] A domain name front flow detection system based on throughput fluctuation comprises a memory and a processor. The memory stores a computer program. The processor implements the steps of the above method when executing the computer program.

[0026] The beneficial effects achieved by the present invention are as follows:

[0027] 1. The throughput fluctuation characteristics proposed in the present invention can more clearly distinguish between domain name pre-flow and normal network flow, thereby improving the accuracy of flow detection.

[0028] 2. The present invention adopts a more discriminating throughput fluctuation feature to enhance the sensitivity and accuracy of the detector.

[0029] 3. The present invention performs detection based on a random forest model and has strong fault tolerance. By aggregating the prediction results of multiple decision trees, it can reduce the erroneous impact of a single decision tree.

[0030] 4. The model of the present invention shows strong robustness when the network state changes along with the multi-decision process of the random forest algorithm, ensuring stability and adaptability in the actual network environment.

[0031] 5. The detection method of the present invention has a low false alarm rate for normal traffic, has good practicality and deployment potential, and can effectively improve the supervision capabilities of existing network security facilities.

[0032] 6. The present invention detects the domain name front traffic in real time and can perform analysis without waiting for the traffic to end, thus having strong real-time defense capabilities.

[0033] 7. The present invention performs feature design targeting the camouflage defects of domain name front traffic, which can effectively identify malicious traffic and normal traffic and improve the detection effect.

[0034] 8. The present invention not only focuses on the detection effect, but also pays attention to the interpretability and performance optimization of the model, ensuring high flexibility and adaptability, and can be applied in different network environments.

[0035] 9. The present invention supports the use of other machine learning models for implementation, further improving the customizability and application scope of the model and having strong scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of domain name front flow detection based on throughput fluctuation of the present invention.

[0037] Figure 2 It is a throughput comparison chart of meek traffic and normal network traffic in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the various technical features and advantages or technical effects in the above technical solutions of the present invention more obvious and easy to understand, they are described in detail below with reference to the accompanying drawings.

[0039] The present invention discloses a domain name front flow detection method based on throughput fluctuation, and its process is as follows: Figure 1 As shown, the main steps include:

[0040] S1: Capture network traffic

[0041] Collect network traffic in the actual network environment where the detector is deployed, without limiting the traffic type. You can choose to capture it online in real time or load the saved traffic file offline. To improve the versatility of the detector, it is recommended to save the captured traffic in the standardized pcap or pcapng format.

[0042] S2: Network traffic preprocessing

[0043] S2-1: Filters data packets without payload, including TCP ACK packets used to maintain network transmission status and other data packets unrelated to application layer data. At the same time, removes retransmission packets and duplicate ACK packets caused by network status to reduce the computational overhead in the subsequent detection process and improve the distinguishability of features.

[0044] S2-2: Aggregate traffic into independent sessions based on the five-tuple in the data packet. Since the captured network traffic is usually a chaotic collection of data packets, in order to calculate the throughput in each session, the data packets need to be classified according to the five-tuple. Data packets with the same five-tuple (including source IP, destination IP, source port, destination port and transport layer protocol) are considered to be unidirectional data flows in the same direction.

[0045] S2-3: Filter TLS sessions. Based on the characteristics of domain name fronting technology, only network sessions established through the standard TLS protocol are processed. To this end, irrelevant traffic is removed by filtering packets with a target port of 443. At the same time, only sessions that have completed the TLS handshake are retained, because malicious communication requires a complete TLS handshake as a prerequisite. A complete TLS handshake usually requires about 20 packets, so traffic with a session length that is too short due to reasons such as connection establishment failure is discarded.

[0046] S3: Build throughput fluctuation characteristics for each network session

[0047] S3-1: Extract network session meta information. The throughput calculation depends on the payload size and packet direction information of the TCP stream. When parsing the TLS session, extract this data and construct a sequence of signed integers in bytes. The absolute value of the integer in the sequence represents the original packet length, and the positive and negative signs are used to identify the direction: negative values ​​represent inbound packets, and positive values ​​represent outbound packets.

[0048] S3-2: Data packet direction determination. Determine the direction of the data packet based on the prefix of the source IP address. v4 addresses, if the prefix matches "10.", "172." or "192.", it is marked as an outbound packet, otherwise it is marked as an inbound packet. These prefixes belong to LAN IP addresses and are usually assigned to clients, so they can be used as a basis for direction determination. v 6 address, a similar prefix matching method is used to determine the direction of the data packet.

[0049] S3-3: Throughput calculation. The present invention defines throughput as the amount of data transmitted over a network connection. Even if the domain name is pre-pended or the normal network connection is encrypted, the TCP payload size (i.e., packet length) is still visible. Therefore, the throughput value can be calculated by accumulating the size of all packets in the same direction before the current packet.

[0050] The throughput calculation includes the data volume in the TLS handshake phase and the subsequent data transmission phase. Since the data volume of the TLS handshake is small and has little impact on the overall throughput, it is not excluded separately. For any TLS session, the inbound and outbound throughput values ​​are calculated separately according to the direction of the data packet, thereby generating two throughput values ​​with opposite signs. Specifically, the throughput in the inbound direction is recorded as tput in , the corresponding calculation formula is tput in =tput in +s i ; The throughput in the outbound direction is recorded as tput out , the corresponding calculation formula is tput out =tput out -s i , where s i It has nothing to do with direction. In the above two formulas, it represents the amount of data carried by a single packet. The throughput in the inbound direction increases as the number of packets increases, while the outbound direction shows the opposite trend.

[0051] S3-4: Throughput fluctuation characteristics. The present invention robustly extracts traffic characteristics by analyzing the throughput changes during packet sending and receiving. Specifically, two set variables are defined to record the throughput values ​​of the inbound and outbound directions respectively. After calculating the throughput in each direction, the final throughput fluctuation characteristics are extracted, including the inbound throughput and the outbound throughput. This characteristic is calculated by the formula flct in ||flct out , which is used to comprehensively reflect the changes in traffic in different directions.

[0052] S4: Building a domain name front traffic detector based on machine learning model

[0053] S4-1: Training detector. The domain name front traffic detector proposed in the present invention is a binary classifier used to label network traffic as positive or negative. The positive label represents the domain name front traffic, and the negative label represents normal network traffic. Since domain name front detection involves security issues, the model needs to have strong interpretability, and due to the large scale of network traffic, the model is required to have high execution efficiency. To meet these requirements, the present invention uses a random forest algorithm for model training. Random forest is a classic supervised learning algorithm with good interpretability and is more lightweight than deep learning models, showing good performance in traffic classification tasks. Training the detector requires two types of labeled data: negative samples (normal network traffic) and positive samples (domain name front traffic). Negative samples come from normal network access traffic, and positive samples collect domain name front traffic according to different detection targets. The feature used in the training process is throughput fluctuation.

[0054] S4-2: Deploy the detector. After the detector is trained, it is deployed in the actual network environment to detect the domain name front traffic.

[0055] S5: Use the domain name fronting traffic detector to identify potential domain name fronting traffic in the network

[0056] The deployed detector is used to identify domain name pre-traffic that may exist in the network. When encountering unknown traffic, the throughput fluctuation characteristics of the traffic are first extracted and input into the detector for judgment, thereby identifying domain name pre-traffic. When certain traffic is detected and determined to be domain name pre-traffic, the detector can take the following actions: report the network attributes of the traffic to the defender, immediately block the relevant network connection, or back up the traffic for subsequent tracing and forensics.

[0057] A specific embodiment is listed below:

[0058] The detection object of this embodiment is the meek proxy software. Meek is selected as the detection object mainly because the software is open source and free, has been widely used on the Internet, and has about 100,000 active users per day. In addition, meek is the first network covert communication tool based on domain name pre-positioning technology. Its operating environment is complex and diverse, and detection is difficult. Therefore, the selection of meek as an embodiment is representative and also has reference significance for detecting other covert communication software based on domain name pre-positioning.

[0059] First, relevant network traffic is captured. To capture meek traffic, the present invention downloads and installs multiple versions of meek tools on multiple operating systems. In order to capture normal network traffic, the present invention also installs common browsers such as Chrome and Firefox to initiate requests to normal websites. In this embodiment, the operation of all software is manual operation to avoid frequent network requests affecting the normal access of other users. When running these software, the present invention synchronously uses the Wireshark tool to capture relevant data packets.

[0060] After the traffic capture is completed, the present invention pre-processes the traffic. The XOR value is calculated by the five-tuple information in the data packet, and the data packets with the same XOR value are aggregated into the same session. Sessions with a session length greater than or equal to 20 data packets are considered valid sessions. In this embodiment, a total of 1812 valid positive sample traffic (i.e., meek sessions) and 4100 valid negative sample traffic (i.e., normal network access traffic) were captured. In the experiment, the present invention selected five common network request types, including audio, video, image, web page, and text.

[0061] After the data set is constructed, the present invention randomly selects several sessions from the positive and negative samples and analyzes the camouflage defects of the domain name front traffic. Figure 2 As shown in the figure, the experiment found that the domain name front traffic has obvious camouflage defects in throughput. Figure 2 The throughput of the meek session and five common network requests in the first 50 packets is shown, where "in" represents the inbound throughput and "out" represents the outbound throughput. It can be observed that the throughput of the meek session in the inbound direction is significantly smaller than that of normal network requests, indicating that this difference can be used to distinguish domain name front traffic from normal traffic. For example, meek only receives about 10.7KB of data, while the image request receives 47KB of data. However, in the opposite outbound direction, the throughput of meek is greater than that of normal network requests.

[0062] The present invention notes that the throughput difference is related to the session mode. Normal network requests usually follow a request-response mode, that is, the server will try its best to respond after receiving the client request, resulting in the client receiving a large number of data packets in a short period of time. Meek, on the other hand, needs to poll the CDN to see if the original data has been obtained, resulting in an increase in outbound throughput. In addition, in the initial stage of the session, the increase in outbound data packets may squeeze the data packets in the opposite direction, thereby reducing the inbound throughput.

[0063] After proving that throughput is discriminative, the present invention extracts the throughput fluctuation characteristics of each session and uses them to construct a data set. Specifically, the present invention uses the first 30 packets in each session to calculate the throughput fluctuation characteristics and saves the results as a JSON format file. Since only the first 30 packets are used, the calculated throughput fluctuation feature vector is usually 30-dimensional; if the length of a feature vector is less than 30, -1 is padded at the end of the feature vector.

[0064] Afterwards, the present invention uses these features to train a random forest-based detector. Specifically, training data is randomly selected from the dataset, 70% of the traffic is used for training, and the remaining 30% is used to evaluate the model performance. Each detector is trained for 10 rounds, and the performance is evaluated using the average metric. During this process, the best performing model is selected and saved for subsequent testing. The testing phase is similar to the training phase, and the output of the model is of two types, positive and negative, where "positive" represents domain name front traffic and "negative" represents normal network traffic.

[0065] Finally, the present invention deployed the model and compared the robustness with existing detection technologies. To compare the robustness, the present invention re-downloaded and installed seven different versions of meek software from the training phase, and captured 50 network traffic for each version. The test results show that the detection method proposed by the present invention is significantly better than the four existing detection methods. For example, the recall rate of the present invention for version 11.5.7 reached 100%, while the other detection methods were all below 87%. These test results show that the technology proposed by the present invention can accurately detect domain name front traffic, and has a low false alarm rate for normal traffic. It has the potential to be deployed in actual network environments, which helps to improve the regulatory capabilities of existing network security facilities.

[0066] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions of the technical solutions of the present invention made by ordinary technicians in the field should all be included in the protection scope of the present invention. The protection scope of the present invention shall be based on what is defined in the claims.

Claims

1. A domain name front traffic detection method based on throughput fluctuation, characterized in that: The following steps are involved: 1) Capture network traffic; 2) Preprocess the captured network traffic; 3) Extract the direction information and payload data of the network session, calculate the inbound and outbound throughput based on the packet direction, and generate the throughput fluctuation characteristics; 4) Build and train a domain name front-end traffic detector based on throughput fluctuation characteristics; 5) Deploy a trained domain name front traffic detector to detect unknown attribute traffic in the network, classify and judge the traffic throughput fluctuation characteristics, and identify potential domain name front traffic.

2. The method according to claim 1, characterized in that In step 1), the network traffic is captured by online real-time capture or by loading a saved traffic file offline and saved in standard pcap or pcapng format.

3. The method according to claim 1, characterized in that The pre-processing steps in step 2) include: Filter data packets without payload and data packets affected by network status; Aggregate traffic into independent sessions based on the five-tuple in the data packet, which includes source IP, destination IP, source port, destination port, and transport layer protocol; Filter TLS sessions by port filtering and handshake length.

4. The method according to claim 3, characterized in that The filtered no-payload data packets include TCP ACK packets used to maintain network transmission status and data packets unrelated to application layer data; the data packets affected by network status include retransmission packets and repeated ACK packets.

5. The method according to claim 3, characterized in that Screening TLS sessions through port filtering and handshake length judgment specifically includes: filtering data packets with a target port of 443, eliminating irrelevant traffic, and retaining only sessions that have completed the TLS handshake; discarding traffic with a session length shorter than 20 packets.

6. The method according to claim 1, characterized in that After extracting the direction information and payload data in step 3), a signed integer sequence in bytes is constructed. The absolute value of the integer in the sequence represents the length of the original data packet. The positive and negative signs are used to identify the direction. Negative values ​​represent inbound data packets, and positive values ​​represent outbound data packets.

7. The method according to claim 1, characterized in that In step 3), the direction of the data packet is determined according to the source IP prefix and marked as inbound or outbound; then the inbound and outbound throughputs are calculated by accumulating the sizes of data packets in the same direction; and the throughput fluctuation characteristics are extracted based on the changes in the inbound and outbound throughputs.

8. The method according to claim 1, characterized in that In step 4), the domain name front traffic detector is a binary classifier, which is trained using the random forest algorithm. The domain name front traffic detector marks the network traffic with positive and negative labels based on the throughput fluctuation characteristics, where the positive label represents the domain name front traffic and the negative label represents the normal traffic. The training data used consists of normal traffic and domain name prefix traffic with labeled data, normal traffic is used as negative samples, and domain name prefix traffic is used as positive samples.

9. The method according to claim 1, characterized in that In step 5), after the domain name front traffic detector identifies the domain name front traffic, it takes measures such as reporting, blocking or backing up the traffic to facilitate subsequent tracing and evidence collection.

10. A domain name front-end flow detection system based on throughput fluctuation, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 9 when executing the computer program.

Images