DoH service real-time sensing method for high-speed network

Through machine learning methods of offline training and online update, combined with DoH-Sketch structure and hash function mapping algorithm, the accuracy and efficiency of DoH service identification in high-speed networks are solved, real-time identification and supervision are realized, and adapting to changes in the network environment.

CN119011239AActive Publication Date: 2024-11-22SOUTHEAST UNIV

Patent Information

Application Number
CN202411091819.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The prior art is difficult to identify DoH services in real time in high-speed networks, resulting in identification accuracy and efficiency issues, and is unable to adapt to changes in the network environment.

Method used

Using offline training and online update machine learning methods, the DoH-Sketch structure and hash function mapping algorithm are used to extract and process traffic features, build and optimize DoH traffic recognition models, and realize real-time recognition and model iterative updates.

Benefits of technology

The rapid identification and supervision of DoH services is realized in high-speed networks, reducing identification delay and resource consumption, adapting to changes in the network environment, and improving identification accuracy and real-timeness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011239A_ABST
    Figure CN119011239A_ABST
Patent Text Reader

Abstract

The invention discloses a DoH service real-time sensing method for a high-speed network, which is divided into an offline training stage, an online updating stage and a real-time identification stage, in the offline training stage, DoH data of encrypted DNS service providers at home and abroad are collected, and a plurality of representative one-way traffic characteristics are extracted according to the characteristics of an encrypted DNS protocol; sampling the collected data, and storing the data in combination with a DoH-Sketch technology; and training to obtain a DoH service identification model. In the online updating stage, the model is iteratively optimized by using open world data, and the iterative updating of the DoH classification recognition model facing the real network environment is realized by adopting online classification, active detection and online learning technologies. In the real-time identification stage, according to the DoH service list in the real network obtained in the offline training and online updating stage, DoH grouping is rapidly identified for the traffic captured in the high-speed network. The DoH service detection and early warning method is used for DoH service detection and early warning in mass traffic scenes such as a high-speed network, and provides a basis for network security supervision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a real-time perception method of DoH service for high-speed networks, belonging to the technical field of cyberspace security. Background Art

[0002] DoH (DNS over HTTPS, DoH) is widely used as an emerging DNS resolution method. By encrypting the transmission of DNS queries, it effectively prevents third parties from eavesdropping and tampering with DNS data packets. The legal use of DoH can effectively protect user privacy. However, the widespread use of DoH may also bring new problems. Since DoH encapsulates DNS query requests in HTTPS traffic for transmission, third parties such as network operators and network security departments cannot directly view the contents of DNS data packets, nor can they analyze encrypted traffic, which brings difficulties to cyberspace security monitoring and protection. Network operators cannot obtain information such as users' network activities and access habits, which may affect network operators' advertising and user behavior analysis, thereby affecting their business models and revenues. At the same time, the encryption characteristics of DoH restrict traditional network supervision methods. Malicious users or malware can hide malicious activities in DoH traffic, thereby bypassing traditional network supervision methods, bringing new challenges to network security governance, thereby threatening the country's cyberspace security. Although DoH technology provides more protection for user privacy, it is still a necessary measure to identify DoH services under the premise of maintaining social stability and security. By identifying DoH services, regulators can better understand and monitor the dissemination of information on social media platforms, promptly detect the spread of harmful information and potential activities that threaten social security, and take necessary intervention measures to maintain social order and public safety.

[0003] At present, research on DoH traffic is mainly divided into two directions: DoH tunnel traffic identification and DoH service identification.

[0004] DoH tunnel traffic refers to malicious traffic that uses the DoH service as a tunnel for transmission. In the direction of DoH tunnel traffic identification, researchers are constantly exploring new methods to distinguish between benign DoH traffic and malicious DoH tunnel traffic. Existing methods measure the TLS fingerprint of the DoH service, establish a fingerprint library, and detect DoH tunnel traffic by comparing the TLS fingerprint of the traffic to be tested with the fingerprint in the fingerprint library. However, the DoH tunnel traffic detection method based on TLS fingerprint is not accurate and flexible. An attacker can evade TLS fingerprint-based detection by changing the parameters of the TLS fingerprint extension or information such as the cipher suite, so the detection method based on TLS fingerprint has poor fault tolerance. In recent years, some methods have used machine learning algorithms to conduct DoH tunnel traffic detection experiments based on the CIRA-CIC-DoHBrw-2020 dataset. Their experiments show that the accuracy of random forests, decision trees, support vector machines, and K-nearest neighbor algorithms is close to 99.9%, but support vector machines and K-nearest neighbor algorithms are the slowest algorithms in the training phase. The Gaussian naive Bayes algorithm identifies traffic types the fastest, but has the worst effect in the detection phase.

[0005] At present, there are invention patents for identifying DoH tunnel traffic. The existing invention patent "Method, device and electronic device for detecting malicious DoH traffic" uses an autoencoder trained by normal DoH traffic to detect the DoH traffic to be detected, because it can accurately output an output vector with extremely low mean square error with the input vector by processing normal DoH traffic, thereby effectively identifying traffic that deviates significantly from the normal mode. However, this type of method cannot be updated and adjusted to adapt to new threat patterns when facing attackers using highly concealed tunneling technology or dynamically changing attack strategies. In addition, "A DoH malicious tunnel traffic detection method based on a two-layer random forest" designs a DoH malicious tunnel traffic detection method based on a two-layer random forest. By analyzing traffic characteristics and passing through a two-layer random forest classifier, DoH traffic in normal HTTPS traffic can be extracted and DoH malicious tunnel traffic can be deeply identified. However, this type of method has a large computational overhead during training and inference, and the model cannot adapt to new threat patterns, and cannot be directly deployed and applied in real-time monitoring systems.

[0006] Although some results have been achieved in the research on DoH tunnel traffic identification, these studies mainly focus on distinguishing benign DoH traffic from malicious DoH tunnel traffic. The goal of DoH service identification is to identify all traffic packets belonging to the DoH protocol in the traffic, rather than distinguishing whether the traffic has malicious characteristics. This is inconsistent with the goal of DoH tunnel traffic identification. Therefore, the method of DoH tunnel traffic identification cannot be directly applied to the identification of DoH services. DoH service identification requires identifying DoH traffic packets in the traffic. Since DoH traffic and general encrypted HTTPS application traffic both use TCP port 443 and the packet payload is encrypted, DoH traffic cannot be directly identified by parsing the packet payload.

[0007] At present, the methods for studying DoH service identification can be mainly divided into three categories: identification methods based on fingerprint library, supervised machine learning and deep learning, but all of them have certain limitations. The identification method based on fingerprint library uses the characteristic that the response mode of DoH server is always the same when establishing TLS connection with the same client, and uses fingerprint identification to detect TLS negotiation between client and DoH server to determine the identity of DoH server. However, although this method has high accuracy and reliability in theory, it has a high dependence on the quality and integrity of fingerprint library in practical application, and the update and maintenance of fingerprint library requires a lot of manpower and material resources. Especially in high-concurrency network environment, it is impossible to quickly and accurately complete the analysis and identification of TLS fingerprint, so this method cannot be applied to high-speed network scenarios; the identification method based on supervised machine learning selects appropriate flow-based statistical features and timing features, such as packet length statistics, number of packets, flow duration and packet interval time, and then uses supervised machine learning method to train classifier to identify DoH service traffic in the network. This method generally selects statistical features based on bidirectional five-tuple flows. However, asymmetric routing is widely deployed in backbone networks. The uplink and downlink packets of the same flow may pass through different network paths, so only traffic in a certain direction can be observed on the same network node. Therefore, the statistical features based on bidirectional five-tuple flows selected by the above research are not suitable for DoH service identification in backbone networks; the recognition method based on deep learning automatically selects and extracts features by learning the inherent laws of sample data, trains detection models, and identifies DoH service traffic. However, this method requires training and prediction based on a large amount of data, and only uses sample data collected by itself for training and testing during implementation. The traffic in high-speed networks is huge. If deep learning methods are used, not only will the computational complexity be high, but the storage space requirements for the system will also be very large. Therefore, this method cannot be applied to high-speed network scenarios.

[0008] At present, there are invention patents for identifying DoH traffic. The existing invention patent "DoH service identification method and device, electronic device and storage medium" performs DoH-based domain name resolution requests for the detection domain name through a preset detection frequency, receives the resolution results, and updates the network address in the database according to these results. When responding to a domain name resolution service type query request, by comparing the network address to be queried in the query request with the network address in the database, it is determined whether the domain name resolution service is a DoH-based service. Due to the dynamic and complex nature of the network, the domain name resolution server may respond to a large number of resolution requests, resulting in delays in the identification process of the DoH-based resolution service, affecting the real-time update of the database. Therefore, this type of method cannot be applied to large-scale high-speed network scenarios. In addition, "A DoH detection method based on self-attention BiLSTM" uses the self-attention mechanism and bidirectional long short-term memory network in deep learning technology to build a complex model that can process and analyze encrypted HTTPS traffic, aiming to solve the data imbalance problem and the challenges of high detection difficulty and low accuracy faced in traditional DoH traffic detection. Although this method has made some progress in improving detection accuracy, in practical applications, the diversity and dynamics of network traffic limit the generalization ability of the model. Moreover, in a high-speed network environment that requires real-time response, the time and computing storage resources required for deep learning are very large, which will seriously affect the overall protection effect. Therefore, it cannot be applied to actual high-speed network scenarios.

[0009] In addition, the existing methods do not take into account the data drift phenomenon that exists in the real network during the model training process. Data collection and labeling are usually carried out within a limited time frame, and the obtained data set often only reflects the characteristics of the DoH service within a specific time period. However, in a real network environment, the characteristics of the DoH service may change over time, resulting in the original training data may not represent the mapping relationship in the current network environment. This will affect the performance of DoH service identification, increase the probability of false positives and false negatives, and thus reduce the accuracy of identification.

[0010] In summary, although existing DoH traffic identification research has achieved certain results, the following problems exist, which make the methods unsuitable for high-speed networks: (1) Existing methods use bidirectional flow data and do not fully consider the real network scenarios where asymmetric routing is widely present; (2) Existing methods require too much time and resource costs to extract features from traffic. In a high-speed network environment, the time and computing and storage resources required for group flow are very large, which leads to delays and efficiency problems in the identification process; (3) Although existing methods have achieved high recognition accuracy, they do not pay attention to the recognition speed, resulting in the inability of existing research results to be applied in practice; (4) Existing methods do not take into account the data drift phenomenon in the real network during the model training process, which affects the performance of DoH service identification and reduces the accuracy of identification.

[0011] The real-time perception method of DoH service provided by the present invention can solve the difficulty in real-time identification of DoH service in high-speed network in the current network security field. Summary of the invention

[0012] In order to solve the above problems, the present invention discloses a real-time perception method of DoH (DNS over HTTPS, DoH) service for high-speed networks. The method proposed by the present invention can complete the identification of DoH services. The identification method in the present invention is divided into an offline training stage, an online update stage and a real-time identification stage. In the offline training stage, DoH data of domestic and foreign encrypted DNS service providers are collected, and according to the characteristics of the full ciphertext information and the encrypted DNS protocol, several representative one-way traffic feature data are extracted and corresponding feature groups are constructed; the collected data is systematically sampled and stored in combination with the DoH-Sketch technology; finally, a specific machine learning method is used for training to obtain a DoH service identification model. In the online update stage, the model is iteratively optimized using open world data, and online classification, active detection, and online learning technologies are used to design an automatic update mechanism to continuously improve the discrimination performance of the model and realize the iterative update of the DoH classification and identification model for the real network environment. In the real-time identification stage, the traffic captured in the actual high-speed network is preprocessed according to the DoH service list in the real network environment obtained in the offline training and online update stages, and the DoH grouping is quickly identified using the list matching algorithm based on hash function mapping, so as to detect whether there is DoH traffic in the current traffic. The present invention can be used for DoH service detection and early warning in high-speed network and other high-volume traffic scenarios, providing a basis for network security supervision.

[0013] In order to achieve the purpose of the present invention, the specific technical steps of this solution are as follows: A real-time perception method of DoH service for high-speed network, the method comprising the following steps:

[0014] Step (1) Use distributed active detection to realize efficient and automatic collection of DoH data sets in parallel, and obtain traffic data of domestic and foreign encrypted DNS service providers. According to the IP address of the encrypted DNS server, obtain sample labels, add labels to the traffic data, and form a training data set;

[0015] Step (2) Based on the full ciphertext information and the traffic characteristics of the high-speed network, several feature representation methods are constructed using indicators based on proportion and distribution rules to extract several feature representations that can reflect the efficient identification of DoH traffic, such as the number of TLS fragments in the data packet, the size of the TLS fragments, the proportion of the number of data packets, and the size distribution of the payload;

[0016] Step (3) Design a custom DoH-Sketch massive traffic processing structure and use the HASH algorithm to determine the appropriate sampling rate based on the performance of the DoH-Sketch structure. Based on the sampled traffic, store the feature data described in step (2) and achieve rapid extraction;

[0017] Step (4) In order to further improve the processing speed, a machine learning algorithm with lower complexity is selected, and the feature vector obtained in step (3) is input into the machine learning algorithm for training to obtain a recognition model for DoH traffic;

[0018] Step (5) uses online classification, active detection, and online learning technologies, and uses open world data to design an automatic update mechanism for the model in step (4), continuously improve the model's discrimination performance, and achieve iterative updates of the DoH recognition model for high-speed networks;

[0019] Step (6) uses a list matching algorithm based on hash function mapping to quickly identify DoH groups based on the DoH service list in the actual network environment obtained in step (5) and the real-time data in the high-speed network according to its triple information.

[0020] Furthermore, in step (1), the steps of constructing a data set for training are as follows:

[0021] (1.1) Design and develop an automatic sample collection program for global DoH services. Use automated tools to control the browser and sequentially access the URL list corresponding to the domestic and foreign DoH service supply list, and collect large-scale network traffic of global encrypted DNS service providers as training data sets;

[0022] (1.2) According to the IP addresses of domestic and foreign encrypted DNS servers, the traffic data of the training data set in step (1.1) are associated and labels are added to the training data set in step (1.1).

[0023] Furthermore, in step (2), the steps of constructing corresponding feature groups according to traffic characteristics are as follows:

[0024] (2.1) Analyze the principle of the encrypted DNS protocol and extract several payload-related feature data that can enhance the recognition of the encrypted DNS protocol, such as the proportion of data packets containing payload in the connection and the size of the payload;

[0025] (2.2) Using indicators based on proportion and distribution rules, several characteristic data related to the length of TLS fragments in data packets are extracted to enhance the recognition of encrypted DNS protocols, such as the length of TLS fragments in data packets and packet length;

[0026] (2.3) In order to reduce the processing delay of packets, the data packet feature extraction is optimized.

[0027] Furthermore, in step (3), the steps of efficiently extracting features and processing are as follows:

[0028] (3.1) There are multiple storage units in the custom DoH-Sketch structure, each of which contains multiple counters to simultaneously record multiple traffic feature data. The DoH-Sketch structure is updated once each time a data packet is obtained. Figure 2 Shows a custom DoH-Sketch structure storing statistics of DoH features;

[0029] (3.2) In order to reduce the impact of hash conflicts on statistical results, the training data set collected in step (1) is Systematic sampling is carried out in proportion to obtain the sampled flow data;

[0030] (3.3) Since DoH uses TCP port 443, when the source address or destination address of a packet uses TCP port 443, the three values ​​of the corresponding IP address, port 443, and TCP protocol are used as key values ​​for HASH calculation. In order to increase processing speed and reduce the consumption of packet processing, DoH-Sketch only uses one HASH function to locate the bucket, divides the hash value into multiple parts, and increments the counter after locating the response position;

[0031] (3.4) When the total of the counters of the number of recorded messages in a bucket reaches the threshold, a saturation event occurs and the operation of extracting feature vectors, i.e., packet accumulation features, is performed.

[0032] Furthermore, in step (4), during the machine learning training of feature data, the algorithm's processing speed, classification accuracy, and anti-overfitting ability need to be comprehensively considered. This method selects the random forest algorithm to train the sample features in step (3) to obtain a recognition model for DoH traffic. The algorithm is known for its excellent classification performance and robustness.

[0033] Furthermore, in step (5), the steps of performing online update iteration on the offline model are as follows:

[0034] (5.1) Capture real-time traffic from real backbone network nodes, perform traffic sampling and feature data storage statistics according to the process of step (3), and extract feature data after reaching the threshold;

[0035] (5.2) Input the features extracted in step (5.1) into the DoH traffic recognition model trained in step (4) to obtain the prediction results;

[0036] (5.3) For the group obtained in step (5.1), use the distributed active detection mechanism to send a request to it, and combine the prediction result obtained in step (5.2) and the active detection response traffic type to determine whether it is a real DoH service provider. If it is a real DoH service provider, update the DoH service list information in the format of a triple (ip, port, protocol) to the DoH database; if it is not a DoH service provider, re-mark its group to generate updated sample data;

[0037] (5.4) For the verification results obtained in step (5.3), the recognition model in step (4) is updated and iterated online using an incremental learning method based on machine learning to adapt to the new data constantly generated in the real network environment. Figure 3 Demonstrated the construction and optimization of DoH traffic identification model for massive data.

[0038] Furthermore, in step (6), the steps of real-time identification and supervision of DoH services for data in the high-speed network are as follows:

[0039] (6.1) Initialize a bit vector of length m with all bits set to 0. Read all DoH lists from the DoH database, and for each DoH service entry to be added, use k independent hash functions h1, h2, …, hk to calculate k positions in the range of 0 to m-1, and set the bits at these positions to 1;

[0040] (6.2) When a traffic sample S is captured from a high-speed network, the k hash functions in step (6.1) are used to calculate the triplet information of S, and the bits of the bit vector corresponding to m in step (6.1) are checked to see if they are all 1. If any of the bits is 0, it can be determined that the sample is definitely not in the set; if all bits are 1, the sample exists in the DoH list, and regulatory countermeasures are implemented for the sample.

[0041] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for real-time perception of DoH services for high-speed networks is implemented.

[0042] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the real-time perception method of DoH services for high-speed networks.

[0043] Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0044] (1) The present invention proposes a method for real-time perception of DoH services in high-speed network scenarios. The method can extract efficient DoH traffic features, design efficient data processing structures for massive encrypted traffic data, and construct and optimize a real-time DoH traffic recognition model for massive data, thereby quickly identifying DoH groups of massive data and providing data basis for network security personnel.

[0045] (2) In response to the current problem of lack of online real-time extraction methods for grouping features, the present invention designs an efficient and highly recognizable DoH traffic feature representation method based on full ciphertext information and encrypted DNS protocol features. Combined with the traffic characteristics of high-speed networks, the present invention uses a custom DoH-Sketch structure to record traffic information, providing basic support for DoH identification in high-speed real network environments.

[0046] (3) The extraction method of the present invention not only focuses on improving the accuracy of DoH traffic identification, but also pays attention to the identification speed. By designing an efficient probabilistic data structure, the potential conflict between list matching and device performance under resource-limited conditions is resolved, and a list matching algorithm based on hash function mapping is proposed to meet the real-time requirements in high-speed network environments, so that the research results can be truly applied to actual network security protection.

[0047] (4) In view of the problem that existing recognition methods are unable to cope with concept drift in real network environments, the present invention designs a DoH traffic recognition model optimization method for real environments, incrementally updates samples based on an active detection module, and iterates the DoH model through incremental learning to ensure the accuracy of the model in a real open network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A system framework diagram of a real-time perception method for DoH services in high-speed networks;

[0049] Figure 2 To quickly locate and count group features in a custom DoH-Sketch structure;

[0050] Figure 3 Diagram for building and optimizing DoH traffic identification models for massive data. DETAILED DESCRIPTION

[0051] The technical solution provided by the present invention will be described in detail below in conjunction with specific embodiments. It should be understood that the following specific implementation methods are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0052] Embodiment: The present invention provides a real-time perception method for DoH services in high-speed networks, and its overall system structure is as follows: Figure 1 As shown, the following steps are included:

[0053] Step (1) Use distributed active detection to realize efficient and automatic collection of DoH data sets in parallel, and obtain traffic data of domestic and foreign encrypted DNS service providers. According to the IP address of the encrypted DNS server, obtain sample labels, add labels to the traffic data, and form a training data set;

[0054] In one embodiment of the present invention, the steps of constructing a data set for training are as follows:

[0055] (1.1) Design and develop an automatic sample collection program for global DoH services. Use automated tools to control the browser and sequentially access the URL list corresponding to the domestic and foreign DoH service supply list, and collect large-scale network traffic of global encrypted DNS service providers as training data sets;

[0056] (1.2) According to the IP addresses of domestic and foreign encrypted DNS servers, associate them with the traffic data of the training data set in step (1.1), and add labels to the training data set in step (1.1). For example, 223.6.6.6 is the IP address of Alibaba Cloud's encrypted DNS server, and a label is added to Alibaba Cloud's encrypted DNS server according to 223.6.6.6;

[0057] Step (2) Based on the full ciphertext information and the traffic characteristics of the high-speed network, several feature representation methods are constructed using indicators based on proportion and distribution rules to extract several feature representations that can reflect the efficient identification of DoH traffic, such as the number of TLS fragments in the data packet, the size of the TLS fragments, the proportion of the number of data packets, and the size distribution of the payload;

[0058] In one embodiment of the present invention, the steps of constructing corresponding feature groups according to traffic features are as follows:

[0059] (2.1) Analyze the principle of the encrypted DNS protocol and extract several payload-related feature data that can enhance the recognition of the encrypted DNS protocol, such as the proportion of data packets containing payload in the connection and the size of the payload;

[0060] (2.2) Using indicators based on proportion and distribution rules, several characteristic data related to the length of TLS fragments in data packets are extracted to enhance the recognition of encrypted DNS protocols, such as the length of TLS fragments in data packets and packet length;

[0061] (2.3) In order to reduce the processing delay of packets, the feature extraction of data packets is optimized. Table 1 lists some feature names and their meanings.

[0062] Table 1: Some feature names and their meanings

[0063]

[0064]

[0065] Step (3) Design a custom DoH-Sketch massive traffic processing structure and use the HASH algorithm to determine the appropriate sampling rate based on the performance of the DoH-Sketch structure. Based on the sampled traffic, store the feature data described in step (2) and achieve rapid extraction;

[0066] In one embodiment of the present invention, the steps of efficiently extracting features and processing are as follows:

[0067] (3.1) There are multiple storage units in the custom DoH-Sketch structure, each of which contains multiple counters to simultaneously record multiple traffic feature data. The DoH-Sketch structure is updated once each time a data packet is obtained. Figure 2 Shows a custom DoH-Sketch structure storing statistics of DoH features;

[0068] (3.2) In order to reduce the impact of hash conflicts on statistical results, when the sampling rate is set to , the DoH-Sketch structure performs best. The training data set collected in step (1) is Carry out systematic sampling in proportion to obtain the sampled flow data;

[0069] (3.3) Since DoH uses TCP port 443, when the source address or destination address of a packet uses TCP port 443, the three values ​​of the corresponding IP address, port 443, and TCP protocol are used as key values ​​for HASH calculation. In order to increase processing speed and reduce the consumption of packet processing, DoH-Sketch only uses one HASH function to locate the bucket, divides the hash value into multiple parts, and increments the counter after locating the response position;

[0070] (3.4) When the total of the counters of the number of recorded messages in a bucket reaches the threshold, a saturation event occurs and the operation of extracting feature vectors is performed, i.e., packet accumulation features;

[0071] Step (4) In order to further improve the processing speed, a machine learning algorithm with lower complexity is selected, and the feature vector obtained in step (3) is input into the machine learning algorithm for training to obtain a recognition model for DoH traffic;

[0072] In one embodiment of the present invention, during the machine learning training of feature data, the algorithm's processing speed, classification accuracy, and anti-overfitting ability need to be comprehensively considered. This method selects the random forest algorithm to train the sample features in step (3) to obtain a recognition model for DoH traffic. The algorithm is known for its excellent classification performance and robustness.

[0073] Step (5) uses online classification, active detection, and online learning technologies, and uses open world data to design an automatic update mechanism for the model in step (4), continuously improve the model's discrimination performance, and achieve iterative updates of the DoH recognition model for high-speed networks;

[0074] In one embodiment of the present invention, the steps of performing online update iteration on the offline model are as follows:

[0075] (5.1) Capture real-time traffic from real backbone network nodes, perform traffic sampling and feature data storage statistics according to the process of step (3), and extract feature data after reaching the threshold;

[0076] (5.2) Input the features extracted in step (5.1) into the DoH traffic recognition model trained in step (4) to obtain the prediction results;

[0077] (5.3) For the group obtained in step (5.1), use the distributed active detection mechanism to send a request to it, and combine the prediction result obtained in step (5.2) and the active detection response traffic type to determine whether it is a real DoH service provider. If it is a real DoH service provider, the DoH service list information is updated to the DoH database in the format of a triple (ip, port, protocol); if it is not a DoH service provider, re-mark its group to generate updated sample data; Table 2 shows the detection task data structure designed for distributed active detection.

[0078] Table 2: Detection task data structure table

[0079] Field Name Data Types describe Port NumberP String The target port number to be detected IP Address List List of strings IP addresses of t hosts to be detected L[0]...L[t-1] Task ID Integer Uniquely identifies a detection task Task Status String Recording the completion of tasks

[0080] (5.4) For the verification results obtained in step (5.3), the recognition model in step (4) is updated and iterated online using an incremental learning method based on machine learning to adapt to the new data constantly generated in the real network environment. Figure 3 Demonstrated the construction and optimization of DoH traffic identification model for massive data.

[0081] Step (6) uses a list matching algorithm based on hash function mapping to quickly identify DoH groups based on the DoH service list in the actual network environment obtained in step (5) and the real-time data in the high-speed network according to its triple information.

[0082] In one embodiment of the present invention, the steps of real-time identification and supervision of DoH services for data in a high-speed network are as follows:

[0083] (6.1) Initialize a bit vector of length m, with all bits set to 0. Read all DoH lists from the DoH database, and for each DoH service entry to be added, use k independent hash functions h1, h2, …, hk to calculate k positions in the range of 0 to m-1, and set the bits at these positions to 1; Table 3 shows some DoH service list information in the DoH database.

[0084] Table 3: DoH service list information in the DoH database

[0085] id List DoH Service Providers 1 (223.5.5.5,443,protocol) Ali 2 (223.6.6.6,443,protocol) Ali 3 (1.12.12.12,443,protocol) Tencent 4 (120.53.53.53,443,protocol) Tencent 5 (1.0.0.1,443,protocol) Cloudflare 6 (1.1.1.1,443,protocol) Cloudflare

[0086] (6.2) When a traffic sample S is captured from a high-speed network, the k hash functions in step (6.1) are used to calculate the triplet information of S, and the bits of the bit vector corresponding to m in step (6.1) are checked to see if they are all 1. If any of the bits is 0, it can be determined that the sample is definitely not in the set; if all bits are 1, the sample exists in the DoH list, and regulatory countermeasures are implemented for the sample.

[0087] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A real-time perception method for DoH services in high-speed networks. It is characterized in that The method comprises the following steps: Step (1) Use a distributed active detection method to realize efficient and automatic collection of DoH data sets in parallel, obtain traffic data of domestic and foreign encrypted DNS service providers, obtain sample labels based on the IP addresses of encrypted DNS servers, add labels to the traffic data, and form a training data set; Step (2) Based on the full ciphertext information and the traffic characteristics of the high-speed network, several feature representation methods are constructed using indicators based on proportion and distribution rules to extract several feature representations that can reflect the efficient identification of DoH traffic, including the number of TLS fragments in the data packet, the size of the TLS fragment, the proportion of the number of data packets, and the size distribution of the payload; Step (3) designs a customized DoH-Sketch massive traffic processing structure and uses the HASH algorithm to determine the appropriate sampling rate based on the performance of the DoH-Sketch structure. Based on the sampled traffic, the feature data of step (2) is stored and quickly extracted. Step (4) In order to further improve the processing speed, a machine learning algorithm with lower complexity is selected, and the feature vector obtained in step (3) is input into the machine learning algorithm for training to obtain a recognition model for DoH traffic; Step (5) uses online classification, active detection, and online learning technologies, and uses open world data to design an automatic update mechanism for the model in step (4), continuously improve the model's discrimination performance, and achieve iterative updates of the DoH recognition model for high-speed networks; Step (6) uses a list matching algorithm based on hash function mapping to quickly identify DoH groups based on the DoH service list in the actual network environment obtained in step (5) and the real-time data in the high-speed network according to its triple information.

2. According to claim 1, a real-time perception method of DoH service for high-speed network, It is characterized in that The step (1) specifically comprises the following sub-steps: (1.1) Design and develop an automatic sample collection program for global DoH services. Use automated tools to control the browser and sequentially access the URL list corresponding to the domestic and foreign DoH service supply list, and collect large-scale network traffic of global encrypted DNS service providers as training data sets; (1.2) According to the IP addresses of domestic and foreign encrypted DNS servers, the traffic data of the training data set in step (1.1) are associated and labels are added to the training data set in step (1.1).

3. According to claim 1, a real-time perception method of DoH service for high-speed network, It is characterized in that The step (2) specifically comprises the following sub-steps: (2.1) Analyze the principle of the encrypted DNS protocol and extract several payload-related feature data that can enhance the recognition of the encrypted DNS protocol, including the proportion of data packets containing payload in the connection and the size of the payload; (2.2) Using indicators based on proportion and distribution rules, several characteristic data related to the length of TLS fragments in data packets are extracted to enhance the recognition of encrypted DNS protocols, including the length of TLS fragments in data packets and the packet length; (2.3) In order to reduce the processing delay of packets, the data packet feature extraction is optimized.

4. According to claim 1, a real-time perception method of DoH service for high-speed network, It is characterized in that The step (3) specifically comprises the following sub-steps: (3.1) There are multiple storage units in the custom DoH-Sketch structure, each of which contains multiple counters to simultaneously record multiple traffic feature data; the DoH-Sketch structure is updated once each data packet is obtained; (3.2) In order to reduce the impact of hash conflicts on statistical results, the training data set collected in step (1) is Systematic sampling is carried out in proportion to obtain the sampled flow data; (3.3) Since DoH uses TCP port 443, when the source address or destination address of a packet uses TCP port 443, the three values ​​of the corresponding IP address, port 443, and TCP protocol are used as key values ​​for HASH calculation. In order to increase the processing speed and reduce the consumption of packet processing, DoH-Sketch only uses one HASH function to locate the bucket, divides the hash value into multiple parts, and increments the counter after locating the response position; (3.4) When the total of the counters of the number of recorded messages in a bucket reaches the threshold, a saturation event occurs and the operation of extracting feature vectors, i.e., packet accumulation features, is performed.

5. According to claim 1, a real-time perception method of DoH service for high-speed network, It is characterized in that In the step (4), during the machine learning training of feature data, it is necessary to comprehensively consider the algorithm's processing speed, classification accuracy, and anti-overfitting ability, and select the random forest algorithm to train the sample features in step (3) to obtain a DoH traffic recognition model.

6. According to claim 1, a real-time perception method of DoH service for high-speed network, It is characterized in that The step (5) specifically comprises the following sub-steps: (5.1) Capture real-time traffic from real backbone network nodes, perform traffic sampling and feature data storage statistics according to the process of step (3), and extract feature data after reaching the threshold; (5.2) Input the features extracted in step (5.1) into the DoH traffic recognition model trained in step (4) to obtain the prediction results; (5.3) For the group obtained in step (5.1), a distributed active detection mechanism is used to send a request to it, and the prediction result obtained in step (5.2) and the active detection response traffic type are combined to determine whether it is a real DoH service provider. If it is a real DoH service provider, the DoH service list information is updated to the DoH database in the format of a triple (ip, port, protocol); if it is not a DoH service provider, its group is re-marked to generate updated sample data; (5.4) For the verification results obtained in step (5.3), the recognition model in step (4) is updated and iterated online using an incremental learning method based on machine learning to adapt to the new data constantly generated in the real network environment.

7. According to claim 1, a real-time perception method of DoH service for high-speed network, It is characterized in that The step (6) specifically comprises the following sub-steps: (6.1) Initialize a bit vector of length m with all bits set to 0, read all DoH lists from the DoH database, and for each DoH service entry to be added, use k independent hash functions h1, h2, …, hk to calculate k positions in the range of 0 to m-1, and set the bits at these positions to 1; (6.2) When a traffic sample S is captured from a high-speed network, the k hash functions in step (6.1) are used to calculate the triplet information of S, and the bits of the bit vector corresponding to m in step (6.1) are checked to see if they are all 1. If any of the bits is 0, it can be determined that the sample is definitely not in the set; if all bits are 1, the sample exists in the DoH list, and regulatory countermeasures are implemented for the sample.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, Features: When the processor executes the program, it implements the real-time perception method of DoH service for high-speed networks as described in any one of claims 1 to 7.

9. A computer-readable storage medium having computer instructions stored thereon, Features: When the computer instruction is executed by the processor, the real-time perception method of DoH service for high-speed networks as described in any one of claims 1-7 is implemented.

Citation Information

Patent Citations

  • Method for defending IP address deception DDoS attack in high-speed network

    CN114172697A

  • Selective policy-driven interception of encrypted network traffic with domain name service and single sign-on service

    CN116830528A

Cited By

  • DoH server detection and identification method based on access mode

    CN116436670A