A DNS log-based P2P content distribution network node identification method and system

Abstract

The application discloses a DNS log-based P2P content distribution network node identification method and system, and belongs to the technical field of network security and traffic analysis.The application solves the problem that the prior art cannot accurately identify P2P content distribution network nodes.The application collects DNS resolution logs from recursive resolvers of Internet service providers, extracts key information of DNS queries from the DNS resolution logs, analyzes domain name access preference features, content provider aggregation features and external service behavior features of the nodes according to the extracted key information, obtains abstract features of the nodes, and finally inputs the abstract features of the nodes into a P2P content distribution network node identification model corresponding to network connection time lengths, so as to accurately identify the P2P content distribution network nodes through the identification model.The method can be applied to P2P content distribution network node identification.

Description

Technical Field

[0001] This invention belongs to the field of network security and traffic analysis technology, specifically relating to a method and system for identifying P2P content delivery network (PCDN) nodes based on DNS logs. Background Technology

[0002] With the development of the internet, P2P content delivery network technology has gradually emerged. This technology integrates the content acceleration capabilities of traditional CDNs with the distributed architecture of P2P networks, utilizing user terminal devices as content distribution nodes to achieve efficient resource sharing and content distribution. Based on this technology, traffic monetization (RPT) services have emerged, allowing users to become PCDN nodes by installing PCDN boxes and earn income by renting out their home bandwidth.

[0003] However, the large-scale deployment of PCDN nodes in the network has brought serious network problems. First, PCDN nodes consume a significant amount of additional network resources because they not only need to handle normal network requests from individual users but also provide content distribution services to other users, thus consuming substantial bandwidth. Second, by utilizing low-cost home broadband traffic for content distribution, PCDN technology weakens the market competitiveness of ISPs' high-priced enterprise broadband services and disrupts the traditional bandwidth usage and pricing system. Finally, PCDN technology interferes with the normal operation of shared bandwidth, leading to a decline in the network experience for other users and affecting applications with high real-time network requirements, such as online gaming and video conferencing.

[0004] While the PCDN ecosystem has attracted considerable attention from researchers, existing research primarily focuses on issues arising from PCDN deployment and implementation, such as security vulnerabilities like video clip contamination and IP address leaks. Currently, there is a lack of research on methods to accurately identify PCDN nodes within a network. This makes it difficult for network operators and administrators to effectively monitor and manage these nodes, hindering their ability to take targeted measures to maintain network stability and security. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that existing technologies cannot accurately identify P2P content delivery network nodes, and to propose a P2P content delivery network node identification method based on DNS logs.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] A method for identifying P2P content delivery network nodes based on DNS logs, the method specifically includes the following steps:

[0008] Step 1: Collect DNS resolution logs for each DNS query from the recursive resolver of the Internet service provider, and extract information for each DNS query based on the collected DNS resolution logs;

[0009] DNS query information includes timestamp, source IP address, source port, query record, fully qualified domain name, and query type;

[0010] Step 2: Classify the fully qualified domain names for each DNS query, and the classification result is whether the queried domain name belongs to a PCDN domain or a non-PCDN domain.

[0011] Based on the source IP address and source port, obtain the node to which each DNS resolution log belongs;

[0012] Step 3: For any node a, based on the domain name classification results, obtain the absolute number of queries for PCDN domain names and the total number of queries for PCDN and non-PCDN domain names. Then calculate the PCDN domain name query ratio of node a and use the calculated PCDN domain name query ratio as the domain name access preference feature of node a.

[0013] For any DNS resolution log corresponding to node a, determine whether the node accesses a content provider based on the query records in the current resolution log. After traversing each DNS resolution log corresponding to node a, obtain the number of content providers accessed by node a and the total number of times node a accesses content providers, thus obtaining the content provider aggregation feature of node a.

[0014] Obtain the external service behavior characteristics of node a based on the source IP address and query type;

[0015] The feature vector of node a is composed of the domain name access preference features, content provider aggregation features, and external service behavior features of node a;

[0016] Step 4: Input the feature vectors of each node into the XGBoost recognition model to identify whether each node is a P2P content delivery network node.

[0017] Furthermore, the identification of the fully qualified domain name for each DNS query uses the BERT model.

[0018] Furthermore, the PCDN domain name query ratio of node a is: the ratio of the absolute number of queries of node a for PCDN domain names to the total number of queries of node a for both PCDN and non-PCDN domain names.

[0019] Furthermore, the external service behavior characteristics of node a include whether node a has been queried, the number of times node a has received queries, and the ratio of the number of times node a has queried other nodes to the number of times node a has received queries.

[0020] Furthermore, the specific process of step four is as follows:

[0021] Step 41: Divide the node's online time into 24-hour intervals, and denote each hour interval as (z, z+1], where z is a positive integer;

[0022] Train an XGBoost recognition model for each hourly interval;

[0023] Step 42: For node a, determine the network connection duration of node a based on the timestamps of all DNS resolution logs corresponding to node a, and select the corresponding XGBoost identification model based on the network connection duration of node a.

[0024] Input the feature vector of node a into the selected XGBoost recognition model, and use the XGBoost recognition model to identify whether node a is a PCDN node.

[0025] A P2P content delivery network node identification system based on DNS logs, the system comprising a DNS resolution log collection module, a DNS query information extraction module, a fully qualified domain name classification module, a node feature vector extraction module, and a P2P content delivery network node identification module, wherein:

[0026] The DNS resolution log collection module is used to collect DNS resolution logs for each DNS query from the recursive resolver of the Internet service provider;

[0027] The DNS query information extraction module is used to extract information from the DNS resolution log of each DNS query. The information includes timestamp, source IP address, source port, query record, fully qualified domain name, and query type.

[0028] The fully qualified domain name classification module is used to classify the fully qualified domain names in each DNS query, and the classification result is whether the queried domain name belongs to a PCDN domain name or a non-PCDN domain name.

[0029] The node feature vector extraction module is used to extract the feature vectors of each node based on the information extracted by the DNS query information extraction module and the classification results of the fully qualified domain name classification module.

[0030] The P2P content delivery network node identification module is used to identify whether a node is a P2P content delivery network node based on the node's feature vector.

[0031] Furthermore, the fully qualified domain name classification module is equipped with a BERT model, which is used to classify the fully qualified domain names for each DNS query.

[0032] Furthermore, the working process of the node feature vector extraction module is as follows:

[0033] Step 1: Determine the node to which each DNS resolution log belongs based on the source IP address and source port;

[0034] Step 2: For any node a, based on the domain name classification results, obtain the absolute number of queries for PCDN domain names and the total number of queries for both PCDN and non-PCDN domain names. Then, calculate the ratio of the absolute number of queries for PCDN domain names to the total number of queries for both PCDN and non-PCDN domain names. Use the obtained ratio as the domain name access preference feature of node a.

[0035] Step 3: Determine whether each query record in the DNS resolution log accesses the domain name of the content provider, and then obtain the number of content providers accessed by node a and the total number of times node a accesses the content providers, thus obtaining the content provider aggregation characteristics of node a.

[0036] Step 4: Obtain the external service behavior characteristics of node a based on the source IP address and query type;

[0037] Step 5: Use the domain name access preference features, content provider aggregation features, and external service behavior features of node a to form the feature vector of node a.

[0038] Furthermore, the external service behavior characteristics of node a include whether node a has been queried, the number of times node a has received queries, and the ratio of the number of times node a has queried other nodes to the number of times node a has received queries.

[0039] Furthermore, the working process of the P2P content delivery network node identification module is as follows:

[0040] The online duration of nodes is divided into 24-hour intervals, and each hour interval is denoted as (z, z+1], where z is a positive integer; then an XGBoost recognition model is trained for each hour interval.

[0041] The network connection duration of node a is determined based on the timestamps of all DNS resolution logs corresponding to node a, and the corresponding XGBoost identification model is selected based on the network connection duration of node a.

[0042] Input the feature vector of node a into the selected XGBoost recognition model, and use the XGBoost recognition model to identify whether node a is a PCDN node.

[0043] The beneficial effects of this invention are:

[0044] This invention proposes a method for identifying P2P content distribution network nodes based on DNS logs. It collects DNS resolution logs from the recursive resolvers of Internet service providers, extracts key information about DNS queries from these logs, and analyzes the node's domain name access preferences, content provider aggregation characteristics, and external service behavior based on this extracted information. This yields abstract characteristics of the node, which are then input into a P2P content distribution network node identification model corresponding to the network connection duration. This model enables accurate identification of P2P content distribution network nodes. This method can improve the accuracy and efficiency of network management for network operators and administrators, reduce the negative impact of P2P content distribution network nodes on network stability, and enhance network stability and security. Attached Figure Description

[0045] Figure 1 This is a flowchart of a P2P content delivery network node identification method based on DNS logs according to the present invention;

[0046] Figure 2 This is a schematic diagram of node relationships when extracting external service behavior features. Detailed Implementation

[0047] Specific implementation method one: Combining Figure 1 This embodiment describes a method for identifying P2P content delivery network nodes based on DNS logs. The method specifically includes the following steps:

[0048] Step 1: Collect DNS resolution logs for each DNS query from the Internet Service Provider's (ISP) recursive resolver, and extract key information for each DNS query based on the collected DNS resolution logs;

[0049] Key information for a DNS query includes timestamp, source IP address, source port, query record, fully qualified domain name (FQDN), and query type (QTYPE).

[0050] Step 2: Classify the fully qualified domain names for each DNS query, and the classification result is whether the queried domain name belongs to a PCDN domain or a non-PCDN domain.

[0051] Based on the source IP address and source port, obtain the node to which each DNS resolution log belongs;

[0052] It should be noted that due to the source IP address exhaustion problem, operators in real-world environments typically employ IP address reuse strategies. Therefore, relying solely on IP addresses to distinguish users is insufficient. This invention utilizes the mapping rules and node-IP mapping provided by the ISP to obtain the dynamic port range (i.e., source port) assigned to each node after the IP address is set. It then creates a unique identifier for each node by combining the IP address with its corresponding dynamic port range, and finally obtains the DNS resolution logs belonging to each node.

[0053] Step 3: For any node a, based on the domain name classification results, obtain the absolute number of queries for PCDN domain names and the total number of queries for PCDN and non-PCDN domain names. Then calculate the PCDN domain name query ratio of node a and use the calculated PCDN domain name query ratio as the domain name access preference feature of node a.

[0054] For any DNS resolution log corresponding to node a, determine whether the node accesses a content provider based on the query records in the current resolution log. After traversing each DNS resolution log corresponding to node a, obtain the number of content providers accessed by node a and the total number of times node a accesses content providers, thus obtaining the content provider aggregation feature of node a.

[0055] Obtain the external service behavior characteristics of node a based on the source IP address and query type;

[0056] The feature vector of node a is composed of the domain name access preference features, content provider aggregation features, and external service behavior features of node a;

[0057] Step 4: Input the feature vectors of each node into the XGBoost recognition model to identify whether each node is a P2P content delivery network node.

[0058] This invention extracts features that can distinguish whether a node is equipped with a PCDN box, including three parts: "domain access preference", "content provider aggregation" and "external service behavior".

[0059] Domain Name Access Preferences: A significant difference between PCDN and non-PCDN nodes lies in the frequency of access to PCDN domain names. PCDN domain names are used by the PCDN box to support content distribution, resource localization, and node communication. PCDN nodes typically access more PCDN domain names to perform tasks such as resource caching or resource scheduling. Therefore, analyzing PCDN domain name access patterns can assess a node's domain name access preferences, which helps in effectively identifying PCDN nodes. For any given node, after obtaining all DNS log entries belonging to node 'a', we obtain the number of queries the node makes to PCDN domain names and the total number of queries to both PCDN and non-PCDN domain names based on the domain name classification results of each DNS log entry. Then, we calculate the ratio of the number of queries to PCDN domain names to the total number of queries to both PCDN and non-PCDN domain names.

[0060] Content Provider Aggregation: PCDN nodes differ from non-PCDN nodes in the range of content providers they access. The internet has various types of content providers; ordinary users typically access only a small number of fixed providers, while PCDN nodes, due to business needs, usually access a large number of different providers. Furthermore, to cache resources, PCDN nodes generate far more domain name accesses than ordinary users. By analyzing the content providers behind the domains accessed by each node, the range of content providers accessed and the number of domain names requested can be assessed, thus identifying PCDN nodes. ISPs provide a database of content provider domain names; based on this database, it can be determined whether each query record accesses a content provider's domain name and the corresponding content provider can be identified. This allows calculation of the number of content providers accessed and the total number of domain name queries for each node in a single day.

[0061] External service behavior: Another prominent feature of PCDN nodes is that they often act as service providers, offering services to other nodes, which requires them to be accessed frequently. For example... Figure 2 As shown, each IP address represents a node, and each DNS query represents a directed edge, with the source IP address of the DNS query as the starting point and the A record of the DNS query as the ending point. Due to business requirements, PCDN nodes typically have high out-degree and in-degree. In contrast, users without deployed PCDN boxes typically have a higher in-degree than out-degree. Estimating the service provision frequency of each node by calculating the number of times it is resolved helps in identifying PCDN nodes. A records and CNAME records from query types are extracted from the DNS logs as features. If a node's IP address appears in node a's A record or resolved CNAME record, then this node is considered to have been visited once by node a. Since it is impossible to precisely determine which specific node corresponding to an IP address was visited, this invention assumes that all nodes corresponding to this IP address were visited once by node a. For example, forFigure 2 In the DNS log entry for node 7, the A and CNAME records within the query type are extracted. If the IP address of node 5 appears in these extracted A and CNAME records, then node 5 was accessed once by node 7. If other nodes exist under the IP address of node 5, then those other nodes under that IP address are also considered to have been accessed once by node 7. Therefore, it is possible to determine whether a node has been accessed (queried), the number of times it has been queried, and the total number of times this node has queried other nodes.

[0062] Finally, three types of features are extracted from all DNS logs of each node: domain name access preference features, content provider aggregation features, and external service behavior features. The final features obtained by fusion are shown in Table 1.

[0063] Table 1

[0064]

[0065] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the classification of fully qualified domain names for each DNS query uses the BERT model.

[0066] The other steps and parameters are the same as in Specific Implementation Method 1.

[0067] This invention utilizes the rich language representation and superior feature extraction capabilities of the BERT model to identify PCDN and non-PCDN domain names using the BERT-based PCDN domain name identification method (PDIM).

[0068] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that the PCDN domain name query ratio of node a is: the ratio of the absolute number of queries of node a for PCDN domain names to the total number of queries of node a for both PCDN and non-PCDN domain names.

[0069] Other steps and parameters are the same as in specific implementation method one or two.

[0070] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the external service behavior characteristics of node a include whether node a has been queried, the number of times node a has received queries, and the ratio of the number of times node a has queried other nodes to the number of times node a has received queries.

[0071] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0072] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the specific process of step four is as follows:

[0073] Step 4: 1. Divide the node's online time into 24-hour intervals, and denote each hour interval as (z, z+1], where z is a positive integer; the value of z ranges from 0 to 23.

[0074] Train an XGBoost recognition model for each hourly interval;

[0075] Step 42: For node a, determine the network connection duration of node a based on the timestamps of all DNS resolution logs corresponding to node a, and select the corresponding XGBoost identification model based on the network connection duration of node a.

[0076] Input the feature vector of node a into the selected XGBoost recognition model, and use the XGBoost recognition model to identify whether node a is a PCDN node.

[0077] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0078] For node 'a' in the network, its various feature values ​​gradually increase with the duration of the network connection, but the change in feature values ​​is relatively small within a continuous hour. Therefore, this invention divides the network connection duration into hourly intervals from 0 to 24 hours, and trains a separate recognition model for each hourly interval, resulting in 24 independent recognition models. An appropriate model is then selected for node recognition based on the user's online duration. This strategy enhances the model's adaptability and accuracy under different network connection durations. This invention selects the XGBoost model to perform the recognition task. The feature vector generated through "feature fusion" is input into the XGBoost model corresponding to the user's online duration. The XGBoost model ultimately outputs whether a node is a PCDN node.

[0079] Specific Implementation Method Six: This implementation method describes a P2P content delivery network node identification system based on DNS logs. The system includes a DNS resolution log collection module, a DNS query information extraction module, a fully qualified domain name classification module, a node feature vector extraction module, and a P2P content delivery network node identification module, wherein:

[0080] The DNS resolution log collection module is used to collect DNS resolution logs for each DNS query from the recursive resolver of the Internet service provider;

[0081] The DNS query information extraction module is used to extract information from the DNS resolution log of each DNS query. The information includes timestamp, source IP address, source port, query record, fully qualified domain name, and query type.

[0082] The fully qualified domain name classification module is used to classify the fully qualified domain names in each DNS query, and the classification result is whether the queried domain name belongs to a PCDN domain name or a non-PCDN domain name.

[0083] The node feature vector extraction module is used to extract the feature vectors of each node based on the information extracted by the DNS query information extraction module and the classification results of the fully qualified domain name classification module.

[0084] The P2P content delivery network node identification module is used to identify whether a node is a P2P content delivery network node based on the node's feature vector.

[0085] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Method Six in that the fully qualified domain name classification module is equipped with a BERT model, which is used to classify the fully qualified domain names for each DNS query.

[0086] The other steps and parameters are the same as in Specific Implementation Method Six.

[0087] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods Six or Seven in that the working process of the node feature vector extraction module is as follows:

[0088] Step 1: Determine the node to which each DNS resolution log belongs based on the source IP address and source port;

[0089] Step 2: For any node a, based on the domain name classification results, obtain the absolute number of queries for PCDN domain names and the total number of queries for both PCDN and non-PCDN domain names. Then, calculate the ratio of the absolute number of queries for PCDN domain names to the total number of queries for both PCDN and non-PCDN domain names. Use the obtained ratio as the domain name access preference feature of node a.

[0090] Step 3: Determine whether each query record in the DNS resolution log accesses the domain name of the content provider, and then obtain the number of content providers accessed by node a and the total number of times node a accesses the content providers, thus obtaining the content provider aggregation characteristics of node a.

[0091] Step 4: Obtain the external service behavior characteristics of node a based on the source IP address and query type;

[0092] Step 5: Use the domain name access preference features, content provider aggregation features, and external service behavior features of node a to form the feature vector of node a.

[0093] The other steps and parameters are the same as in specific implementation methods six or seven.

[0094] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods Six to Eight in that the external service behavior characteristics of node a include whether node a has been queried, the number of times node a has received queries, and the ratio of the number of times node a has queried other nodes to the number of times node a has received queries.

[0095] The other steps and parameters are the same as those in specific implementation methods six to eight.

[0096] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods Six to Nine in that the working process of the P2P content distribution network node identification module is as follows:

[0097] The online duration of nodes is divided into 24-hour intervals, and each hour interval is denoted as (z, z+1], where z is a positive integer; then an XGBoost recognition model is trained for each hour interval.

[0098] The network connection duration of node a is determined based on the timestamps of all DNS resolution logs corresponding to node a, and the corresponding XGBoost identification model is selected based on the network connection duration of node a.

[0099] Input the feature vector of node a into the selected XGBoost recognition model, and use the XGBoost recognition model to identify whether node a is a PCDN node.

[0100] The other steps and parameters are the same as those in one of the specific implementation methods six to nine.

[0101] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for identifying P2P content delivery network nodes based on DNS logs, characterized in that, The method specifically includes the following steps: Step 1: Collect DNS resolution logs for each DNS query from the recursive resolver of the Internet service provider, and extract information for each DNS query based on the collected DNS resolution logs; DNS query information includes timestamp, source IP address, source port, query record, fully qualified domain name, and query type; Step 2: Classify the fully qualified domain names for each DNS query, and the classification result is whether the queried domain name belongs to a PCDN domain or a non-PCDN domain. Based on the source IP address and source port, obtain the node to which each DNS resolution log belongs; Step 3: For any node a, based on the domain name classification results, obtain the absolute number of queries for PCDN domain names and the total number of queries for PCDN and non-PCDN domain names. Then calculate the PCDN domain name query ratio of node a and use the calculated PCDN domain name query ratio as the domain name access preference feature of node a. For any DNS resolution log corresponding to node a, determine whether the node accesses a content provider based on the query records in the current resolution log. After traversing each DNS resolution log corresponding to node a, obtain the number of content providers accessed by node a and the total number of times node a accesses content providers, thus obtaining the content provider aggregation feature of node a. Obtain the external service behavior characteristics of node a based on the source IP address and query type; The feature vector of node a is composed of the domain name access preference features, content provider aggregation features, and external service behavior features of node a; Step 4: Input the feature vectors of each node into the XGBoost recognition model to identify whether each node is a P2P content delivery network node.

2. The method for identifying P2P content delivery network nodes based on DNS logs according to claim 1, characterized in that, The BERT model is used to classify the fully qualified domain names for each DNS query.

3. The method for identifying P2P content delivery network nodes based on DNS logs according to claim 2, characterized in that, The PCDN domain name query ratio of node a is: the ratio of the absolute number of queries of node a for PCDN domain names to the total number of queries of node a for both PCDN and non-PCDN domain names.

4. The method for identifying P2P content delivery network nodes based on DNS logs according to claim 3, characterized in that, The external service behavior characteristics of node a include whether node a has been queried, the number of times node a has received queries, and the ratio of the number of times node a has queried other nodes to the number of times node a has received queries.

5. The method for identifying P2P content delivery network nodes based on DNS logs according to claim 4, characterized in that, The specific process of step four is as follows: Step 41: Divide the node's online time into 24-hour intervals, and denote each hour interval as (z, z+1], where z is a positive integer; Train an XGBoost recognition model for each hourly interval; Step 42: For node a, determine the network connection duration of node a based on the timestamps of all DNS resolution logs corresponding to node a, and select the corresponding XGBoost identification model based on the network connection duration of node a. Input the feature vector of node a into the selected XGBoost recognition model, and use the XGBoost recognition model to identify whether node a is a PCDN node.

6. A P2P content delivery network node identification system based on DNS logs, the system being used to execute the P2P content delivery network node identification method based on DNS logs as described in any one of claims 1 to 5, characterized in that, The system includes a DNS resolution log collection module, a DNS query information extraction module, a fully qualified domain name classification module, a node feature vector extraction module, and a P2P content delivery network node identification module, wherein: The DNS resolution log collection module is used to collect DNS resolution logs for each DNS query from the recursive resolver of the Internet service provider; The DNS query information extraction module is used to extract information from the DNS resolution log of each DNS query. The information includes timestamp, source IP address, source port, query record, fully qualified domain name, and query type. The fully qualified domain name classification module is used to classify the fully qualified domain names in each DNS query, and the classification result is whether the queried domain name belongs to a PCDN domain name or a non-PCDN domain name. The node feature vector extraction module is used to extract the feature vectors of each node based on the information extracted by the DNS query information extraction module and the classification results of the fully qualified domain name classification module. The P2P content delivery network node identification module is used to identify whether a node is a P2P content delivery network node based on the node's feature vector.

Images