A real-time perception method for DoH services in high-speed networks

Abstract

The present invention discloses a real-time perception method for DoH services for high-speed networks, which is divided into an offline training stage, an online update stage, and a real-time identification stage. In the offline training stage, DoH data of domestic and foreign encrypted DNS service providers are collected, and according to the characteristics of the encrypted DNS protocol, several representative one-way traffic features are extracted; the collected data is sampled and stored in combination with the DoH-Sketch technology; and a DoH service identification model is obtained through training. In the online update stage, the model is iteratively optimized using open world data, and online classification, active detection, and online learning technologies are adopted to achieve iterative updates of the DoH classification and identification model for real network environments. In the real-time identification stage, based on the DoH service list in the real network obtained in the offline training and online update stages, the DoH grouping is quickly identified for the traffic captured in the high-speed network. The present invention is used for DoH service detection and early warning in massive traffic scenarios such as high-speed networks, providing a basis for network security supervision.

Description

Technical Field

[0001] The present invention relates to a real-time perception method for DoH services in high-speed networks, belonging to the field of cyberspace security technology. Background Art

[0002] DoH (DNS over HTTPS) is a widely adopted emerging DNS resolution method. By encrypting DNS queries, it effectively prevents third parties from eavesdropping and tampering with DNS packets. Legitimate use of DoH can effectively protect user privacy. However, widespread use of DoH may also introduce new challenges. Because DoH encapsulates DNS queries within HTTPS traffic, third parties such as network operators and cybersecurity departments cannot directly view the contents of DNS packets or analyze encrypted traffic, complicating cybersecurity monitoring and protection. Network operators are unable to obtain information about users' online activities and access habits, potentially impacting their advertising and user behavior analysis services, thus impacting their business models and revenue. Furthermore, DoH's encryption limits traditional network policing methods. Malicious users or malware can conceal malicious activities within DoH traffic, thereby circumventing traditional network monitoring measures. This poses new challenges to network security governance and threatens national cybersecurity. While DoH technology provides greater protection for user privacy, identifying DoH services remains a necessary measure to maintain social stability and security. By identifying DoH services, regulators can better understand and monitor the dissemination of information on social media platforms, promptly detect the spread of harmful information and potential activities that threaten social security, and take necessary intervention measures to maintain social order and public safety.

[0003] Currently, research on DoH traffic is mainly divided into two directions: DoH tunnel traffic identification and DoH service identification.

[0004] DoH tunneling traffic refers to malicious traffic that uses the DoH service as a tunnel. Researchers are continuously exploring new methods to distinguish between benign and malicious DoH tunneling traffic. Existing methods measure the TLS fingerprint of the DoH service, establish a fingerprint library, and detect DoH tunneling traffic by comparing the TLS fingerprint of the traffic under test with the fingerprints in the fingerprint library. However, DoH tunneling traffic detection methods based on TLS fingerprints lack accuracy and flexibility. Attackers can evade TLS fingerprint-based detection by modifying parameters in the TLS fingerprint extension or information such as the cipher suite. Therefore, TLS fingerprint-based detection methods have poor fault tolerance. Recent methods have used machine learning algorithms to conduct DoH tunneling traffic detection experiments based on the CIRA-CIC-DoHBrw-2020 dataset. These experiments show that random forests, decision trees, support vector machines, and K-nearest neighbor algorithms achieve accuracy rates approaching 99.9%. However, support vector machines and K-nearest neighbor algorithms are the slowest algorithms during the training phase. The Gaussian Naive Bayes algorithm is the fastest at identifying traffic types, but performs the worst during detection.

[0005] Currently, there are invention patents for identifying DoH tunnel traffic. The existing invention patent, "Method, Device, and Electronic Device for Detecting Malicious DoH Traffic," uses an autoencoder trained with normal DoH traffic to detect DoH traffic. This is because it can accurately output an output vector with an extremely low mean square error with the input vector by processing normal DoH traffic, effectively identifying traffic that deviates significantly from the normal pattern. However, this type of method cannot be updated and adjusted to adapt to new threat patterns when attackers use highly covert tunneling technology or dynamically changing attack strategies. In addition, "A DoH Malicious Tunnel Traffic Detection Method Based on a Two-Layer Random Forest" designs a DoH malicious tunnel traffic detection method based on a two-layer random forest. By analyzing traffic characteristics and passing it through a two-layer random forest classifier, it can extract DoH traffic from normal HTTPS traffic and deeply identify DoH malicious tunnel traffic. However, this type of method has high computational overhead during training and inference, and the model cannot adapt to new threat patterns, making it impossible to directly deploy and apply it in real-time monitoring systems.

[0006] Although research on DoH tunnel traffic identification has achieved certain results, these studies mainly focus on distinguishing benign DoH traffic from malicious DoH tunnel traffic. The goal of DoH service identification is to identify all traffic packets belonging to the DoH protocol in the traffic, rather than distinguishing whether the traffic has malicious characteristics. This is inconsistent with the goal of DoH tunnel traffic identification. Therefore, the DoH tunnel traffic identification method cannot be directly applied to the identification of DoH services. DoH service identification requires identifying DoH traffic packets in the traffic. Since DoH traffic and general encrypted HTTPS application traffic both use TCP port 443 and the packet payload is encrypted, DoH traffic cannot be directly identified by parsing the packet payload.

[0007] Current research methods for identifying DoH services can be categorized into three main categories: fingerprint-based, supervised machine learning-based, and deep learning-based methods. However, all have limitations. Fingerprint-based identification methods exploit the fact that DoH servers always respond identically when establishing TLS connections with the same client. Fingerprint recognition is used to detect the TLS negotiation between the client and the DoH server, thereby determining the DoH server's identity. However, while theoretically highly accurate and reliable, this approach relies heavily on the quality and integrity of the fingerprint database in practice. Furthermore, updating and maintaining the fingerprint database requires significant human and material resources, making it difficult to quickly and accurately analyze and identify TLS fingerprints in highly concurrent network environments. Therefore, this approach is unsuitable for high-speed network scenarios. Supervised machine learning-based identification methods select appropriate flow-based statistical and temporal features, such as packet length statistics, number of packets, flow duration, and inter-packet intervals, and then employ supervised machine learning to train a classifier to identify DoH service traffic within the network. This method generally selects statistical features based on bidirectional five-tuple flows. However, asymmetric routing is widely deployed in backbone networks, and the uplink and downlink packets of the same flow may pass through different network paths. Therefore, only traffic in a certain direction can be observed on the same network node. Therefore, the statistical features based on bidirectional five-tuple flows selected by the above research are not suitable for DoH service identification in backbone networks; the recognition method based on deep learning automatically selects and extracts features by learning the inherent laws of sample data, trains detection models, and identifies DoH service traffic. However, this method requires training and prediction based on a large amount of data, and only uses sample data collected by itself for training and testing during implementation. The traffic in high-speed networks is huge. If deep learning methods are used, not only will the computational complexity be high, but the storage space requirements for the system will also be very large. Therefore, this method cannot be applied to high-speed network scenarios.

[0008] Currently, there are patents for identifying DoH traffic. The existing patent, "DoH Service Identification Method and Apparatus, Electronic Device, and Storage Medium," performs DoH-based domain name resolution requests for detected domain names at a preset detection frequency, receives resolution results, and updates the network address in the database based on these results. When responding to a domain name resolution service type query request, the system compares the network address in the query request with the network address in the database to determine whether the domain name resolution service is a DoH-based service. Due to the dynamic and complex nature of networks, domain name resolution servers may respond to a large number of resolution requests, resulting in delays in identifying DoH-based resolution services and affecting the real-time nature of database updates. Therefore, this method is not suitable for large-scale, high-speed network scenarios. Furthermore, "A DoH Detection Method Based on Self-Attention BiLSTM" utilizes the self-attention mechanism and bidirectional long short-term memory network in deep learning to build a complex model capable of processing and analyzing encrypted HTTPS traffic. This approach aims to address the data imbalance, difficulty, and low accuracy challenges faced by traditional DoH traffic detection. While this approach has made some progress in improving detection accuracy, the diversity and dynamic nature of network traffic limit the model's generalization in real-world applications. Moreover, in a high-speed network environment that requires real-time response, the time and computing storage resources required for deep learning are very large, which will seriously affect the overall protection effect. Therefore, it cannot be applied to actual high-speed network scenarios.

[0009] Furthermore, existing methods fail to account for the data drift that exists in real networks during model training. Data collection and labeling are typically performed within a limited timeframe, resulting in datasets that only reflect the characteristics of DoH services within a specific timeframe. However, in real-world network environments, the characteristics of DoH services may change over time, resulting in the original training data not being able to represent the mapping relationships in the current network environment. This can affect the performance of DoH service identification, increasing the probability of false positives and false negatives, and thus reducing identification accuracy.

[0010] In summary, although the existing DoH traffic identification research has achieved certain results, the following problems exist, which make the methods unsuitable for high-speed networks: (1) The existing methods use bidirectional flow data and do not fully consider the real network scenarios where asymmetric routing is widely present; (2) The time and resource costs required for the existing methods to extract features from the traffic are too high. In a high-speed network environment, the time and computing and storage resources required for grouping flows are very large, which leads to delays and efficiency problems in the identification process; (3) Although the existing methods have achieved high recognition accuracy, they do not pay attention to the recognition speed, resulting in the existing research results cannot be applied in practice; (4) The existing methods do not take into account the data drift phenomenon in the real network during the model training process, which affects the performance of DoH service identification and reduces the accuracy of identification.

[0011] The real-time DoH service perception method provided by the present invention can solve the difficulty in real-time identification of DoH services in high-speed networks in the current network security field. Summary of the Invention

[0012] In order to solve the above problems, the present invention discloses a real-time perception method of DoH (DNS over HTTPS, DoH) services for high-speed networks. The identification of DoH services can be completed by the method proposed by the present invention. The identification method in the present invention is divided into an offline training stage, an online update stage and a real-time identification stage. In the offline training stage, DoH data of encrypted DNS service providers at home and abroad are collected. According to the characteristics of the full ciphertext information and the encrypted DNS protocol, several representative one-way traffic feature data are extracted and corresponding feature groups are constructed; the collected data are systematically sampled and stored in combination with the DoH-Sketch technology; finally, a specific machine learning method is used for training to obtain a DoH service identification model. In the online update stage, the model is iteratively optimized using open world data, and online classification, active detection, and online learning technologies are used to design an automatic update mechanism to continuously improve the discrimination performance of the model and realize the iterative update of the DoH classification and identification model for the real network environment. During the real-time identification phase, data preprocessing is performed on traffic captured from actual high-speed networks based on the list of DoH services in real-world network environments obtained during the offline training and online update phases. A list matching algorithm based on hash function mapping is then used to quickly identify DoH packets, thereby detecting whether DoH traffic exists within the current flow. This invention can be used for DoH service detection and early warning in high-volume traffic scenarios such as high-speed networks, providing a basis for network security supervision.

[0013] To achieve the purpose of the present invention, the specific technical steps of this solution are as follows: A real-time perception method for DoH services on high-speed networks, the method comprising the following steps:

[0014] Step (1) Use distributed active detection to achieve efficient and automatic collection of DoH datasets in parallel, obtaining traffic data from domestic and foreign encrypted DNS service providers. Based on the IP address of the encrypted DNS server, obtain sample labels, add labels to the traffic data, and form a training dataset;

[0015] Step (2) Based on the full ciphertext information and the traffic characteristics of the high-speed network, several feature representation methods are constructed using indicators based on proportion and distribution rules to extract several feature representations that can effectively identify DoH traffic, such as the number of TLS fragments in the data packet, the size of the TLS fragments, the proportion of the number of data packets, and the size distribution of the payload;

[0016] Step (3) Design a custom DoH-Sketch massive traffic processing structure and use the HASH algorithm to determine the appropriate sampling rate based on the performance of the DoH-Sketch structure. Based on the sampled traffic, store the feature data described in step (2) and achieve rapid extraction;

[0017] Step (4) To further improve the processing speed, a machine learning algorithm with lower complexity is selected, and the feature vector obtained in step (3) is input into the machine learning algorithm for training to obtain a recognition model for DoH traffic;

[0018] Step (5) uses online classification, active detection, and online learning technologies to design an automatic update mechanism for the model in step (4) using open world data, continuously improving the model's discrimination performance and achieving iterative updates of the DoH recognition model for high-speed networks;

[0019] Step (6) uses a list matching algorithm based on hash function mapping to quickly identify DoH groups based on the triple information of the real-time data in the high-speed network according to the DoH service list in the real network environment obtained in step (5).

[0020] Furthermore, in step (1), the steps of constructing the data set used for training are as follows:

[0021] (1.1) Design and develop a sample automatic collection program for global DoH services. Use automated tools to control the browser and sequentially access the URL list corresponding to the domestic and foreign DoH service supply list, and collect large-scale network traffic of global encrypted DNS service providers as a training dataset;

[0022] (1.2) According to the IP addresses of domestic and foreign encrypted DNS servers and the traffic data of the training data set in step (1.1), labels are added to the training data set in step (1.1).

[0023] Furthermore, in step (2), the steps of constructing corresponding feature groups according to traffic characteristics are as follows:

[0024] (2.1) Analyze the principles of the encrypted DNS protocol and extract several payload-related feature data that can enhance the recognition of the encrypted DNS protocol, such as the proportion of packets containing payload in the connection and the size of the payload;

[0025] (2.2) Using indicators based on proportion and distribution rules, we extract several characteristic data related to the length of TLS fragments in data packets, which can enhance the recognition of encrypted DNS protocols, such as the length of TLS fragments in data packets and the packet length;

[0026] (2.3) In order to reduce the processing delay of packets, the data packet feature extraction is optimized.

[0027] Furthermore, in step (3), the steps of efficiently extracting features and processing them are as follows:

[0028] (3.1) There are multiple storage units in the custom DoH-Sketch structure, each of which contains multiple counters to simultaneously record multiple traffic feature data. The DoH-Sketch structure is updated every time a data packet is obtained. Figure 2 Shows a custom DoH-Sketch structure storing DoH feature statistics;

[0029] (3.2) In order to reduce the impact of hash conflicts on statistical results, the training data set collected in step (1) is Systematic sampling is carried out in proportion to obtain the flow data after sampling;

[0030] (3.3) Since DoH uses TCP port 443, when the source or destination address of a packet uses TCP port 443, the corresponding IP address, port 443, and TCP protocol are used as key values ​​for hash calculation. To increase processing speed and reduce packet processing overhead, DoH-Sketch uses only one hash function to locate the bucket, divides the hash value into multiple parts, and increments the counter after locating the response location.

[0031] (3.4) When the total counter of the number of recorded packets in a bucket reaches the threshold, a saturation event occurs and the operation of extracting feature vectors, i.e., packet accumulation features, is performed.

[0032] Furthermore, in step (4), during the machine learning training of feature data, the algorithm's processing speed, classification accuracy, and ability to resist overfitting need to be comprehensively considered. This method selects the random forest algorithm to train the sample features in step (3) to obtain a recognition model for DoH traffic. This algorithm is known for its excellent classification performance and robustness.

[0033] Furthermore, in step (5), the steps of performing online update iteration on the offline model are as follows:

[0034] (5.1) Capture real-time traffic from real backbone network nodes, perform traffic sampling and feature data storage statistics according to the process of step (3), and extract feature data after reaching the threshold;

[0035] (5.2) Input the features extracted in step (5.1) into the DoH traffic recognition model trained in step (4) to obtain the prediction results;

[0036] (5.3) For the group obtained in step (5.1), use the distributed active detection mechanism to send a request to it, and combine the prediction results obtained in step (5.2) and the active detection response traffic type to determine whether it is a real DoH service provider. If it is a real DoH service provider, the DoH service list information is updated to the DoH database in the format of a triple (ip, port, protocol); if it is not a DoH service provider, its group is re-marked to generate updated sample data;

[0037] (5.4) For the verification results obtained in step (5.3), the recognition model in step (4) is updated and iterated online using an incremental learning method based on machine learning to adapt to the new data continuously generated in the real network environment. Figure 3 Demonstrated the construction and optimization of DoH traffic identification model for massive data.

[0038] Furthermore, in step (6), the steps of real-time identification and supervision of DoH services for data in the high-speed network are as follows:

[0039] (6.1) Initialize a bit vector of length m with all bits set to 0. Read all DoH lists from the DoH database. For each DoH service entry to be added, use k independent hash functions h1,h2,…,hk to calculate k positions in the range of 0 to m-1, and set the bits at these positions to 1;

[0040] (6.2) When a traffic sample S is captured from a high-speed network, the k hash functions in step (6.1) are used to calculate the triplet information of S, and the bits of the bit vector corresponding to m in step (6.1) are checked to see if they are all 1. If any of the bits is 0, it can be determined that the sample is definitely not in the set; if all bits are 1, the sample exists in the DoH list, and regulatory measures are implemented for the sample.

[0041] An electronic device includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the method for real-time perception of DoH services for high-speed networks is implemented.

[0042] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the real-time perception method of DoH services for high-speed networks.

[0043] Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0044] (1) The present invention proposes a method for real-time perception of DoH services in high-speed network scenarios. The method can extract efficient DoH traffic features, design an efficient data processing structure for massive encrypted traffic data, and construct and optimize a real-time DoH traffic identification model for massive data, thereby quickly identifying DoH groups of massive data and providing data basis for network security personnel.

[0045] (2) In response to the current lack of online real-time extraction methods for group features, the present invention designs an efficient and highly recognizable DoH traffic feature representation method based on full ciphertext information and encrypted DNS protocol features. Combined with the traffic characteristics of high-speed networks, the present invention uses a customized DoH-Sketch structure to record traffic information, providing basic support for DoH identification in high-speed real network environments.

[0046] (3) The extraction method of the present invention not only focuses on improving the accuracy of DoH traffic identification, but also attaches importance to the identification speed. By designing an efficient probabilistic data structure, the potential conflict between list matching and device performance under resource-limited conditions is resolved. A list matching algorithm based on hash function mapping is proposed to meet the real-time requirements in high-speed network environments, making the research results truly applicable to actual network security protection.

[0047] (4) In response to the problem that existing recognition methods are unable to cope with concept drift in real network environments, the present invention designs a DoH traffic recognition model optimization method for real environments. It incrementally updates samples based on the active detection module and iterates the DoH model through incremental learning to ensure the accuracy of the model in a real open network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A system framework diagram of a real-time perception method for DoH services on high-speed networks.

[0049] Figure 2 To quickly locate and count group features in the custom DoH-Sketch structure;

[0050] Figure 3 Graph for building and optimizing DoH traffic identification models for massive data. DETAILED DESCRIPTION

[0051] The technical solutions provided by the present invention will be described in detail below with reference to specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0052] Example: The present invention provides a real-time perception method for DoH services in high-speed networks, the overall system structure of which is as follows: Figure 1 As shown, the following steps are included:

[0053] Step (1) Use distributed active detection to achieve efficient and automatic collection of DoH datasets in parallel, obtaining traffic data from domestic and foreign encrypted DNS service providers. Based on the IP address of the encrypted DNS server, obtain sample labels, add labels to the traffic data, and form a training dataset;

[0054] In one embodiment of the present invention, the steps of constructing a data set for training are as follows:

[0055] (1.1) Design and develop a sample automatic collection program for global DoH services. Use automated tools to control the browser and sequentially access the URL list corresponding to the domestic and foreign DoH service supply list, and collect large-scale network traffic of global encrypted DNS service providers as a training dataset;

[0056] (1.2) Associating the IP addresses of domestic and foreign encrypted DNS servers with the traffic data of the training dataset in step (1.1) adds labels to the training dataset in step (1.1). For example, 223.6.6.6 is the IP address of Alibaba Cloud's encrypted DNS server, and the Alibaba Cloud encrypted DNS server is labeled according to 223.6.6.6;

[0057] Step (2) Based on the full ciphertext information and the traffic characteristics of the high-speed network, several feature representation methods are constructed using indicators based on proportion and distribution rules to extract several feature representations that can effectively identify DoH traffic, such as the number of TLS fragments in the data packet, the size of the TLS fragments, the proportion of the number of data packets, and the size distribution of the payload;

[0058] In one embodiment of the present invention, the steps of constructing corresponding feature groups based on traffic features are as follows:

[0059] (2.1) Analyze the principles of the encrypted DNS protocol and extract several payload-related feature data that can enhance the recognition of the encrypted DNS protocol, such as the proportion of packets containing payload in the connection and the size of the payload;

[0060] (2.2) Using indicators based on proportion and distribution rules, we extract several characteristic data related to the length of TLS fragments in data packets, which can enhance the recognition of encrypted DNS protocols, such as the length of TLS fragments in data packets and the packet length;

[0061] (2.3) In order to reduce the processing delay of packets, the data packet feature extraction is optimized. Table 1 lists some feature names and their meanings.

[0062] Table 1: Some feature names and their meanings

[0063]

[0064]

[0065] Step (3) Design a custom DoH-Sketch massive traffic processing structure and use the HASH algorithm to determine the appropriate sampling rate based on the performance of the DoH-Sketch structure. Based on the sampled traffic, store the feature data described in step (2) and achieve rapid extraction;

[0066] In one embodiment of the present invention, the steps for efficiently extracting and processing features are as follows:

[0067] (3.1) There are multiple storage units in the custom DoH-Sketch structure, each of which contains multiple counters to simultaneously record multiple traffic feature data. The DoH-Sketch structure is updated every time a data packet is obtained. Figure 2 Shows a custom DoH-Sketch structure storing DoH feature statistics;

[0068] (3.2) In order to reduce the impact of hash collisions on statistical results, when the sampling rate is set to , the DoH-Sketch structure performs optimally. The training data set collected in step (1) is Carry out systematic sampling in proportion to obtain the flow data after sampling;

[0069] (3.3) Since DoH uses TCP port 443, when the source or destination address of a packet uses TCP port 443, the corresponding IP address, port 443, and TCP protocol are used as key values ​​for hash calculation. To increase processing speed and reduce packet processing overhead, DoH-Sketch uses only one hash function to locate the bucket, divides the hash value into multiple parts, and increments the counter after locating the response location.

[0070] (3.4) When the total counter of the number of recorded messages in a bucket reaches the threshold, a saturation event occurs and the operation of extracting feature vectors is performed, i.e., packet accumulation features;

[0071] Step (4) To further improve the processing speed, a machine learning algorithm with lower complexity is selected, and the feature vector obtained in step (3) is input into the machine learning algorithm for training to obtain a recognition model for DoH traffic;

[0072] In one embodiment of the present invention, during the machine learning training of feature data, it is necessary to comprehensively consider the algorithm's processing speed, classification accuracy, and ability to resist overfitting. This method selects the random forest algorithm to train the sample features in step (3) to obtain a recognition model for DoH traffic. This algorithm is known for its excellent classification performance and robustness.

[0073] Step (5) uses online classification, active detection, and online learning technologies to design an automatic update mechanism for the model in step (4) using open world data, continuously improving the model's discrimination performance and achieving iterative updates of the DoH recognition model for high-speed networks;

[0074] In one embodiment of the present invention, the steps of performing online update iteration on the offline model are as follows:

[0075] (5.1) Capture real-time traffic from real backbone network nodes, perform traffic sampling and feature data storage statistics according to the process of step (3), and extract feature data after reaching the threshold;

[0076] (5.2) Input the features extracted in step (5.1) into the DoH traffic recognition model trained in step (4) to obtain the prediction results;

[0077] (5.3) For the group obtained in step (5.1), a distributed active detection mechanism is used to send a request to it. Combined with the prediction results obtained in step (5.2) and the active detection response traffic type, it is determined whether it is a real DoH service provider. If it is a real DoH service provider, the DoH service list information is updated to the DoH database in the format of a triple (ip, port, protocol); if it is not a DoH service provider, its group is re-labeled to generate updated sample data; Table 2 shows the detection task data structure designed for distributed active detection.

[0078] Table 2: Detection task data structure table

[0079] Field Name Data Type describe Port number P String The target port number to be detected IP address list List of strings The IP addresses of the t hosts to be detected are L[0]...L[t-1] Task ID Integer Uniquely identifies a detection task Task status S String Record task completion

[0080] (5.4) For the verification results obtained in step (5.3), the recognition model in step (4) is updated and iterated online using an incremental learning method based on machine learning to adapt to the new data continuously generated in the real network environment. Figure 3 Demonstrated the construction and optimization of DoH traffic identification model for massive data.

[0081] Step (6) uses a list matching algorithm based on hash function mapping to quickly identify DoH groups based on the triple information of the real-time data in the high-speed network according to the DoH service list in the real network environment obtained in step (5).

[0082] In one embodiment of the present invention, the steps for real-time identification and monitoring of DoH services in high-speed network data are as follows:

[0083] (6.1) Initialize a bit vector of length m with all bits set to 0. Read all DoH lists from the DoH database. For each DoH service entry to be added, use k independent hash functions h1,h2,…,hk to calculate k positions in the range of 0 to m-1, and set the bits at these positions to 1; Table 3 shows some DoH service list information in the DoH database.

[0084] Table 3: DoH service list information in the DoH database

[0085] id List DoH Service Providers 1 (223.5.5.5,443,protocol) Alibaba 2 (223.6.6.6,443,protocol) Alibaba 3 (1.12.12.12,443,protocol) Tencent 4 (120.53.53.53,443,protocol) Tencent 5 (1.0.0.1,443,protocol) Cloudflare 6 (1.1.1.1,443,protocol) Cloudflare

[0086] (6.2) When a traffic sample S is captured from a high-speed network, the k hash functions in step (6.1) are used to calculate the triplet information of S, and the bits of the bit vector corresponding to m in step (6.1) are checked to see if they are all 1. If any of the bits is 0, it can be determined that the sample is definitely not in the set; if all bits are 1, the sample exists in the DoH list, and regulatory measures are implemented for the sample.

[0087] The technical means disclosed in the solutions of the present invention are not limited to those disclosed in the above-mentioned embodiments, but also include technical solutions composed of any combination of the above-mentioned technical features. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A real-time perception method for DoH services in high-speed networks, characterized in that: The method comprises the following steps: Step (1) Using a distributed active detection method, the efficient and automatic collection of DoH datasets is realized in parallel, traffic data of domestic and foreign encrypted DNS service providers is obtained, sample labels are obtained based on the IP addresses of encrypted DNS servers, and labels are added to the traffic data to form a training dataset; Step (2) Based on the full ciphertext information and the traffic characteristics of the high-speed network, several feature representation methods are constructed using indicators based on proportion and distribution rules to extract several feature representations that can effectively identify DoH traffic, including the number of TLS fragments in the data packet, the size of the TLS fragments, the proportion of the number of data packets, and the size distribution of the payload; Step (3) designs a customized DoH-Sketch massive traffic processing structure and uses a HASH algorithm to determine the sampling rate based on the performance of the DoH-Sketch structure. Based on the sampled traffic, the feature data of step (2) is stored and quickly extracted. Step (4) To further improve the processing speed, a machine learning algorithm with lower complexity is selected, and the feature vector obtained in step (3) is input into the machine learning algorithm for training to obtain a recognition model for DoH traffic; Step (5) uses online classification, active detection, and online learning technologies to design an automatic update mechanism for the model in step (4), continuously improve the model's discrimination performance, and achieve iterative updates of the DoH recognition model for high-speed networks; Step (6) Based on the DoH service list in the real network environment obtained in step (5), the real-time data in the high-speed network is quickly identified and identified using a list matching algorithm based on hash function mapping according to its triple information; Wherein, the step (3) specifically includes the following sub-steps: (3.1) The custom DoH-Sketch structure contains multiple storage units, each of which contains multiple counters to simultaneously record multiple traffic characteristics. The DoH-Sketch structure is updated every time a data packet is obtained. (3.2) In order to reduce the impact of hash conflicts on statistical results, the training data set collected in step (1) is Systematic sampling is carried out in proportion to obtain the flow data after sampling; (3.3) Since DoH uses TCP port 443, when the source address or destination address of a packet uses TCP port 443, the corresponding IP address, port 443, and TCP protocol are used as key values ​​for hash calculation. To improve processing speed and reduce packet processing consumption, DoH-Sketch only uses one hash function to locate the bucket, divides the hash value into multiple parts, and increments the counter after locating the corresponding position; (3.4) When the total counter of the number of recorded messages in a bucket reaches the threshold, a saturation event occurs and the operation of extracting feature vectors is performed, i.e., packet accumulation features; The step (5) specifically includes the following sub-steps: (5.1) Capture real-time traffic from real backbone network nodes, perform traffic sampling and feature data storage statistics according to the process of step (3), and extract feature data after reaching the threshold; (5.2) Input the features extracted in step (5.1) into the DoH traffic recognition model trained in step (4) to obtain the prediction results; (5.3) For the group obtained in step (5.1), use the distributed active detection mechanism to send a request to it, and combine the prediction result obtained in step (5.2) and the active detection response traffic type to determine whether it is a real DoH service provider. If it is a real DoH service provider, update the DoH service list information in the format of a triplet [ip, port, protocol] to the DoH database; if it is not a DoH service provider, re-mark its group to generate updated sample data; (5.4) Based on the verification results obtained in step (5.3), the recognition model in step (4) is updated and iterated online using an incremental learning method based on machine learning to adapt to the new data continuously generated in the real network environment; The step (6) specifically includes the following sub-steps: (6.1) Initialize a bit vector of length m with all bits set to 0, read all DoH lists from the DoH database, and for each DoH service entry to be added, use k independent hash functions h1,h2,…,hk to calculate k positions in the range of 0 to m-1, and set the bits at these positions to 1; (6.2) When a traffic sample S is captured from a high-speed network, the k hash functions in step (6.1) are used to calculate the triplet information of S, and the bits of the bit vector corresponding to m in step (6.1) are checked to see if they are all 1. If any of the bits is 0, it can be determined that the sample is definitely not in the set; if all bits are 1, the sample exists in the DoH list, and regulatory measures are implemented for the sample.

2. A method for real-time perception of DoH services for high-speed networks according to claim 1, characterized in that: The step (1) specifically includes the following sub-steps: (1.1) Design and develop an automated sample collection program for DoH services. Use automated tools to control the browser and sequentially access the URL list corresponding to the DoH service provider list, collecting large-scale network traffic from encrypted DNS service providers as a training dataset. (1.2) According to the IP address of the encrypted DNS server and the traffic data of the training data set in step (1.1), a label is added to the training data set in step (1.1).

3. A method for real-time perception of DoH services for high-speed networks according to claim 1, characterized in that: The step (2) specifically includes the following sub-steps: (2.1) Analyze the principles of the encrypted DNS protocol and extract several payload-related feature data that can enhance the recognition of the encrypted DNS protocol, including the proportion of packets containing payload in the connection and the size of the payload; (2.2) Using indicators based on proportion and distribution rules, several characteristic data related to the length of TLS fragments in data packets are extracted to enhance the recognition of encrypted DNS protocols, including the length of TLS fragments in data packets and the packet length; (2.3) In order to reduce the processing delay of packets, the data packet feature extraction is optimized.

4. A method for real-time perception of DoH services for high-speed networks according to claim 1, characterized in that: In step (4), during the machine learning training of feature data, it is necessary to comprehensively consider the algorithm's processing speed, classification accuracy, and anti-overfitting ability, and select the random forest algorithm to train the sample features in step (3) to obtain a DoH traffic recognition model.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the real-time perception method of DoH services for high-speed networks as described in any one of claims 1 to 4 above.

6. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by the processor, the real-time perception method of DoH service for high-speed networks as described in any one of claims 1-4 is implemented.

Images