A doh malicious encrypted communication detection method and system based on tls payload similarity

Abstract

The present application relates to a kind of DoH malicious encryption communication detection method and system based on TLS payload similarity, belong to malicious encryption traffic analysis technical field, wherein the method includes: capturing network encrypted traffic, extract TLS payload sequence from it, and store the TLS payload sequence in array in stream;From array, extract uplink payload sequence array;From uplink payload sequence array, the initial cluster set is constructed by counting unique value and its occurrence frequency;Find all clusters that satisfy the distance less than dynamic threshold are merged to obtain new cluster set;From new cluster set, eliminate outlier cluster to obtain final cluster and judge whether DoH communication flow is disguised malicious traffic based on this.The present application analyzes the behavior characteristics that cannot be disguised in DoH encryption malicious communication, automatically condenses center point from TLS uplink payload sequence, and discriminates traffic type according to the distribution characteristics of cluster, gets rid of the method of traditional dependence domain name white list and heartbeat interval detection, reduces false alarm, improves the detection accuracy of malicious traffic.

Description

Technical Field

[0001] This invention relates to the field of malicious encrypted traffic analysis technology, specifically to a DoH malicious encrypted communication detection method and system based on TLS payload similarity. Background Technology

[0002] With the increasing demands for cybersecurity and privacy protection, the DoH protocol, as an important encrypted DNS technology, has been widely used due to its ability to effectively prevent DNS queries from being eavesdropped on and tampered with. However, the DoH protocol's feature of encapsulating DNS queries within a standard HTTPS encrypted tunnel, while protecting user privacy, also presents a significant challenge to network threat detection.

[0003] Traditional network security devices are rendered ineffective when facing DoH (DoH) traffic because it behaves identically to ordinary HTTPS web browsing traffic at the transport and application layers. This renders analysis methods based on payload depth testing and domain reputation databases completely useless. Existing detection technologies have largely shifted towards statistical analysis of communication flow metadata, such as query frequency, data volume per request, and communication time intervals.

[0004] Attackers exploit this blind spot, disguising malicious commands and C2 communications as legitimate DoH interactions. These trojans communicate with public DoH services or attacker-hosted DoH servers with legitimate certificates, encapsulating malicious instructions and data within encrypted HTTP requests and responses. This achieves a high degree of concealment in communication traffic, easily bypassing detection based on domain names and static characteristics. This covert communication established through the DoH protocol constitutes a new type of application-layer tunneling attack. Attackers abuse the encryption and standard port features of DoH, mixing malicious traffic with massive amounts of normal web traffic, significantly increasing the difficulty of identification.

[0005] It is evident that existing security detection technologies for DoH encrypted traffic suffer from core defects such as low detection accuracy and inability to effectively identify covert DoH tunnel attacks. These technologies are insufficient to meet the demand for accurate detection of malicious DoH covert communication in today's complex network environment, necessitating a novel detection technology solution. Summary of the Invention

[0006] To address the aforementioned issues, the purpose of this invention is to provide a DoH malicious encrypted communication detection method and system based on TLS payload similarity.

[0007] A DoH malicious encrypted communication detection method based on TLS payload similarity includes: Step 1: Capture encrypted network traffic and identify all DoH communication streams, extract the TLS payload sequence from them, and store the TLS payload sequence in the stream in an array; Step 2: Extract the uplink load sequence array from the array; Step 3: Count each unique value and its occurrence frequency from the uplink load sequence array to construct an initial cluster set; where the unique value is used as the cluster value and the occurrence frequency is used as the cluster weight in the initial cluster. Step 4: Calculate the distance between any two clusters, find all cluster pairs that satisfy the condition that the distance is less than the dynamic threshold, and select the pair with the smallest distance to merge to obtain a new cluster set; Step 5: Remove outlier clusters from the new cluster set to obtain the final cluster set; Step 6: Determine whether the DoH communication flow is a disguised malicious flow based on the final cluster value and weight.

[0008] Preferably, in step 1, the TLS payload sequence of the stream with the same source / destination IP and source / destination port is extracted from the DoH communication stream, and the TLS payload sequence in the stream is stored in an array. The uplink payload length is represented by a positive number, and the downlink payload length is represented by a negative number. The number of uplink payloads in the payload sequence array is greater than 70. If it is less than 70, all TLS payload sequences of the next stream are extracted and appended to the end of the array until the number of uplink payloads in the array is greater than 70.

[0009] Preferably, in step 4, the distance between the two clusters is calculated using the following formula:

[0010] in, The distance between the two clusters. For the p-th cluster The value, For the q-th cluster The value of .

[0011] Preferably, in step 4, all those that satisfy the condition are found. For clusters ≤ T, select the pair with the smallest distance and merge them. If no such pair exists... When there are cluster pairs ≤ T, the iteration terminates to obtain a new cluster set; where the dynamic threshold is... The calculation formula is:

[0012] in, Represents an array of uplink load sequences. This is the proportionality coefficient.

[0013] Preferably, step 5: removing outlier clusters from the new cluster set to obtain the final cluster set includes: Step 5.1: Set the threshold function for removing outliers, and calculate the outlier determination threshold based on the threshold function; Step 5.2: Remove clusters whose weights are less than the outlier threshold to obtain the final cluster set; the outlier determination process is as follows:

[0014] in, Let be the weight of the i-th cluster. For threshold function, The sum of the weights of all clusters. This is the threshold for determining outlier clusters.

[0015] Preferably, step 6: determining whether the DoH communication flow is spoofed malicious traffic based on the final cluster value and weight includes: The final cluster set is sorted in descending order of weight to obtain the sequence; The judgment criteria are set based on the total weight and weight percentage of the final cluster set, and malicious traffic is identified; the judgment criteria are as follows:

[0016] in, This represents the weight percentage of the i-th weight. Let be the weight of the i-th cluster. For the total weight, For the value of the i-th cluster, Indicates the final number of clusters. Indicates the sequence except and The smallest cluster value outside the cluster.

[0017] This invention also provides a DoH malicious encrypted communication detection system based on TLS payload similarity, comprising: The data stream acquisition module is used to capture encrypted network traffic, identify all DoH communication streams, extract TLS payload sequences from them, and store the TLS payload sequences in the streams in an array. The uplink load acquisition module is used to extract the uplink load sequence array from the array; The initial cluster construction module is used to construct an initial cluster set by counting each unique value and its occurrence frequency from the uplink payload sequence array; in the initial cluster, the unique value is used as the cluster value, and the occurrence frequency is used as the cluster weight; The new cluster merging module is used to calculate the distance between any two clusters, find all cluster pairs that satisfy the condition that the distance is less than the dynamic threshold, and select the pair with the smallest distance to merge to obtain a new cluster set. The cluster removal module is used to remove outlier clusters from the new cluster set to obtain the final cluster set; The malicious traffic detection module is used to determine whether DoH communication flows are disguised malicious traffic based on the final cluster value and weight.

[0018] The present invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. When the computer program is executed by the processor, it implements the steps in the above-described method for detecting DoH malicious encrypted communication based on TLS payload similarity.

[0019] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the above-described method for detecting DoH malicious encrypted communication based on TLS payload similarity.

[0020] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention relates to a DoH malicious encrypted communication detection method based on TLS payload similarity. Compared with the prior art, this invention analyzes the unmaskable behavioral characteristics in DoH encrypted malicious communication, and innovatively uses dynamic threshold-controlled merging clusters to automatically gather the center point from the TLS uplink payload sequence. Based on the distribution characteristics of the clusters, the traffic type is determined. This method eliminates the need for traditional methods that rely on domain name whitelists and heartbeat interval detection, reduces false alarms, and improves the detection accuracy of malicious traffic.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A schematic diagram of a DoH malicious encrypted communication detection method based on TLS payload similarity provided by the present invention; Figure 2 The above is a line graph showing the uplink load of Tencent Public DoH white traffic provided by this invention. Figure 3 The 360° common DoH white flow uplink load line chart provided by this invention; Figure 4 The microstepping common DoH white flow uplink load line diagram provided by this invention; Figure 5 The Cobalt Strike_DoH (DoH Trojan) uplink load line graph provided by this invention; Figure 6 The Cobalt Strike_DoH_Multi-Interaction (DoH Trojan) uplink load line graph provided by this invention; Figure 7 The above is a line graph of the uplink load of Vshell_DoH (DoH Trojan) provided by this invention; Figure 8 The above is a line graph of the uplink load of iodine_doh (doh tunnel) provided by this invention; Figure 9 The uplink and downlink load line graph of iodine_doh (doh tunnel) provided by this invention; Figure 10 The uplink load line graph of dnscat2_doh (doh tunnel) provided by this invention; Figure 11 The uplink and downlink load graph of dnscat2_doh (doh tunnel) provided by this invention; Figure 12 The uplink load graph of dns2tcp_doh (doh tunnel) provided by this invention; Figure 13 The uplink and downlink load graphs for dns2tcp_doh (doh tunnel) provided by this invention are shown in the figure. Detailed Implementation

[0024] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0025] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0026] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0027] Please see Figure 1 A DoH malicious encrypted communication detection method based on TLS payload similarity, comprising: Step 1: Capture encrypted network traffic and identify all DoH communication streams, extract the TLS payload sequence from them, and store the TLS payload sequence in the stream in an array; In step 1, encrypted network traffic is captured, and all DoH communication flows are identified (flow definition: flows with the same source / destination IP, source / destination port, and protocol are considered the same flow). TLS payload sequences of flows with the same four-tuple (source / destination IP, source / destination port) are extracted and stored in an array. Uplink payload length is represented by positive numbers, and downlink payload length by negative numbers. The number of uplink payloads in the payload sequence array must be greater than 70. If less than 70, all TLS payload sequences of the next immediately following flow are extracted and appended to the end of the array. If still insufficient, the next flow is appended, and so on, until the number of uplink payloads in the array exceeds 70. This array is then used as a candidate for subsequent analysis.

[0028] Step 2: Extract the uplink load sequence array from the array; In some embodiments of this application, the extracted TLS payload sequence array is assumed to be Ignoring downlink loads (negative values) and 0, we obtain an array of uplink load sequences:

[0029] Step 3: Count each unique value and its occurrence frequency from the uplink load sequence array to construct an initial cluster set; where the unique value is used as the cluster value and the occurrence frequency is used as the cluster weight in the initial cluster. Count each unique value and the number of times it appears This forms the initial set of clusters. Each cluster Value and weight .

[0030]

[0031] Step 4: Calculate the distance between any two clusters, find all cluster pairs that satisfy the condition that the distance is less than the dynamic threshold, and select the pair with the smallest distance to merge to obtain a new cluster set; For any two clusters and Its distance is defined as:

[0032] (1) Set the dynamic threshold for merging clusters

[0033] Dynamic threshold of merged clusters The definition is as follows:

[0034] in r The proportionality coefficient has the following variation rules:

[0035] (2) Merging conditions and methods (iteration)

[0036] In each iteration, find all that satisfy... For clusters of length ≤ T, select the pair with the smallest distance. Merge. New cluster The calculation is as follows:

[0037] (3) Iteration termination condition

[0038] When there is no satisfying ≤ The iteration terminates when there are clusters of T.

[0039] Step 5: Remove outlier clusters from the new cluster set to obtain the final cluster set; In step 5, a threshold function for removing outlier clusters is set.

[0040] Cluster i is considered an outlier if and only if:

[0041] The final set of clusters obtained after removing outliers can be represented as:

[0042] Step 6: Determine whether the DoH communication flow is a disguised malicious flow based on the final cluster value and weight.

[0043] In step 6, let the final cluster set be sorted in descending order of weight as follows:

[0044] in ， For the value of the cluster, The weight of the cluster.

[0045] Define the total weight:

[0046] Weighting percentage:

[0047] in To determine the number of clusters, malicious traffic is identified.

[0048] in, This represents the weight percentage of the i-th weight. Let be the weight of the i-th cluster. For the total weight, For the value of the i-th cluster, This represents the k-th cluster in the sequence. Indicates the sequence except and The cluster with the smallest external value.

[0049] Traffic that fails to meet the corresponding conditions in malicious traffic detection will be excluded from the suspect scope and regarded as ordinary DoH traffic. For candidate flows, if their TLS payload sequence distribution can meet the characteristics of malicious traffic, it can be confirmed that their interactive behavior characteristics match the DoH Trojan or DoH tunnel communication mode. At this time, it can be determined that the DoH encrypted communication flow is disguised malicious traffic, and the domain name information to which its communication points is untrusted and should not be used for whitelist filtering.

[0050] The principles of the present invention will be further explained below with reference to specific embodiments: The malicious communication principle of DoH is as follows: The attacker first takes over control of the Name Server (NS) of a target domain. The client embeds C2 data (the target data is embedded in a subdomain of a DNS request through encoding, encryption, obfuscation, etc.) as the subdomain of the domain that initiated the DNS request. As long as the subdomain of the domain is unique, the recursive server will not be able to find the cached data and will forward the request to the authoritative name server. The request will be assigned to the name server controlled by the attacker through the NS record, and the attacker will receive the target data. The attacker can also embed data in the DNS response data packet and send it to the client.

[0051] The uplink load line plot of DoH white traffic typically uses the two largest clusters (outliers ignored) as a basis, moving downwards, with the weight of the two basis clusters exceeding 40%. Generally, the largest cluster is DATA, the second largest is HEAD, and the smallest cluster is a flag (RST, ACK, etc.). However, the downlink load distribution plot has uniform characteristics and is not meaningful for reference. Figure 2-4 As shown.

[0052] The upward load line graph of a DoH Trojan typically uses the two smallest clusters as bases (ignoring outliers) and moves upwards, with the weight of the two base clusters generally exceeding 30%. Figure 5-7 As shown.

[0053] The uplink load graph of a DoH tunnel typically uses a single cluster as the base (ignoring outliers), moving upwards, with the weight of the base cluster exceeding 80%. The downlink load graph, however, resembles that of a DoH trojan, such as... Figure 8-13 As shown.

[0054] Example 1: White flow determination process The Microstep DoH white traffic TLS payload sequence is as follows (uplink payload is positive, downlink payload is negative): [87,131,51,51,51,51,51,154,154,154,154,154,154,-74,-32,-26,-379,26,-262,-151,-145,51,51,154,154,51,51,154,154,-296,-374,-327,-260,-362,-304,51,51,154,154,51 ,51,154,154,51,51,154,154,-202,-144,53,51,154,154,51,51,154,154,-302,-380,-367,-301,57,51,154,154,51,51,51,51,154,154,154,-214,-147,-222,-253,-227,51,51, 154,154,30,-258,-248,-290,-310,-358,-331,51,51,51,51,154,154,154,154,-341,51,51,154,154,-332,-341,-367,-301,-331,-341,57,51,154,154,-207,-155,51,51,154,154,- 201,-217,51,51,154,154,-176,-215,51,51,154,154,-217,-165,51,51,154,154,-293,-342,51,51,51,51,154,154,154,154,-149,-200,-280,-359,51,51,154,154,-219,-212,-19] Step 1: Extract the data stream The uplink payload sequence array is extracted as follows: [87, 131, 51, 51, 51, 51, 51, 154, 154, 154, 154, 154, 154, 26, 51, 51, 154, 154, 51, 51, 154, 154, 51, 51, 154, 154, 51, 51, 154, 154, 51, 51, 154, 51, 51, 154, 53, 51, 154, 51, 51, 154, 51, 154, 57, 51, 154, 154, 51, 51, 51, 51, 154, 154, 154, 51, 51, 154, 154, 154, 51, 51, 154, 154, 30, 51, 51, 51, 51, 154, 154, 154, 154, 51, 51, 154, 154, 57, 51, 154, 154, 51, 51, 154, 154, 51, 51, 154, 154, 51, 51, 154, 154, 51, 51, 154, 154, 51, 51, 51, 154, 154, 154, 51, 51, 154, 154, 154, 154, 51, 51, 154, 154 Forming the initial cluster set: Cluster 1: Value = 87, Weight = 1 Cluster 2: Value = 131, Weight = 1 Cluster 3: Value = 51, Weight = 46 Cluster 4: Value = 154, Weight = 50 Cluster 5: Value = 26, Weight = 1 Cluster 6: Value = 53, Weight = 1 Cluster 7: Value = 57, Weight = 2 Cluster 8: Value = 30, Weight = 1 Step 2: Merge similar clusters Set the dynamic threshold for merging clusters:

[0055] The maximum difference is 128, meaning the scaling factor is set to r = 0.2, and the dynamic threshold is T = (154-26). 0.2 = 128 0.2 = 25.60

[0056] Merge clusters (iterate): First merge: Merged value = 51, Sum = 53, Distance = 2.00 Second merge: Merged value = 26, sum = 30, distance = 4.00 Third merge: Merged value = 57, sum = 51.04, distance = 5.96 Fourth merge: Merged value = 131, sum = 154, distance = 23.00 Fifth merge: Merged value = 28.0, sum = 51.28, distance = 23.29 The remaining clusters after the iteration ends Cluster 1: Value = 87.00, Weight = 1 Cluster 2: Value = 153.55, Weight = 51 Cluster 3: Value = 50.37, Weight = 51 Step 3: Obtain the final cluster set after removing outlier clusters. Remove outlier clusters The total weight is 103, and the threshold ratio is set to 0.02 according to the function, meaning the weight is less than 103. Clusters with a weight of 0.02 = 2.06 will be removed as outliers. The outlier to be removed this time is cluster 1, whose weight is less than 2.06.

[0057] Obtain the final cluster set and sort it in descending order of weight.

[0058] Cluster 1: Value = 153.55, Weight = 51, Weight percentage = 50.00%

[0059] Cluster 2: Value = 50.37, Weight = 51, Weight percentage = 50.00%

[0060] Step 4: Malicious Traffic Detection

[0061] The final number of clusters is 2, i.e., k=2, but the weight ratio of cluster 1 is ( Only 50% meets the DoH tunnel criterion.

[0062] Step 5: Confirm Malicious Communication

[0063] Because the final cluster does not meet the criteria for malicious traffic, this traffic is classified as legitimate traffic.

[0064] Example 2: Malicious DoH Traffic Detection Process The Cobalt_Strike_doh malicious traffic TLS payload sequence is as follows (uplink payload is positive, downlink payload is negative): [-74,-74,-32,201,-74,-74,-32,201,147,62,-228,148,247,-414,-74,-74,-32,-74,-74,-32,156,79,-74,-74,-32,387,-74,-74,-32,215,-74,-74,-32,201,-74,-74,-32,201,151,69,-647,147,62,-228,-74,-74,-32,156,62,-74,-74,-32,-74,-74,-32,201,147,62,-228,147,62,-228,147,62,-228,147,62,-228,147,62,-228,-74,-74,-32,201,-74,-74,-32,201,-74,-74,-32,201,-74,-74,-32,201,147,62,-228,147,62,-228,147,62,-228,147,62,-228,147,62,-228,-74,-74,-32,201,-74,-74,-32,201,-74,-74,-32,201,147,62,-228,151,69,-647,147,62,-228,-74,-74,-32,201,147,62,-228,148,248,-415,147,62,-228,147,80,-246,148,159,-326,-74,-74,-32,225,-74,-74,-32,201,147,62,-228,-74,-74,-32,-74,-74,-32,156,62,147,62,-228,147,62,-228,147,62,-228,-74,-74,-32,201,-74,-74,-32,201,-74,-74,-32,201,-74,-74,-32,201,147,62,-228,151,69,-647,-74,-74,-32,215,-74,-74,-32,201,-74,-74,-32,201,-74,-74,-32,215,148,249,-416,148,249,-416,148,249,-416,147,62,-228,147,62,-228,147,76,-242,147,62,-228,-74,-74,-32,389,-74,-74,-32,316,-74,-74,-32,316,-74,-74,-32,201,147,62,-228,-74,-74,-32], Step 1: Extract the data stream The uplink payload sequence array is extracted as follows: Upward load: [201, 201, 147, 62, 148, 247, 156, 79, 387, 215, 201, 201,151, 69, 147, 62, 156, 62, 201, 147, 62, 147, 62, 147, 62, 147, 62, 147, 62, 201, 201, 201, 147, 62, 147, 62, 147, 62, 147, 62, 201, 201,201, 147, 62, 151, 69, 147, 62, 201, 147, 62, 148, 248, 147, 62, 147, 80,148, 159, 225, 201, 147, 62, 156, 62, 147, 62, 147, 62, 147, 62, 147, 62, 201, 201, 201, 201, 147, 62, 151, 69, 215, 201, 201, 215, 148, 249, 148, 249, 148, 249, 147, 62, 147, 62, 147, 76, 147, 62, 389, 316, 316, 201, 147, 62] Forming the initial cluster set: Cluster 1: Value = 201, Weight = 21 Cluster 2: Value = 147, Weight = 27 Cluster 3: Value = 62, Weight = 27 Cluster 4: Value = 148, Weight = 6 Cluster 5: Value = 247, Weight = 1 Cluster 6: Value = 156, Weight = 3 Cluster 7: Value = 79, Weight = 1 Cluster 8: Value = 387, Weight = 1 Cluster 9: Value = 215, Weight = 3 Cluster 10: Value = 151, Weight = 3 Cluster 11: Value = 69, Weight = 3 Cluster 12: Value = 248, Weight = 1 Cluster 13: Value = 80, Weight = 1 Cluster 14: Value = 159, Weight = 1 Cluster 15: Value = 225, Weight = 1 Cluster 16: Value = 249, Weight = 3 Cluster 17: Value = 76, Weight = 1 Cluster 18: Value = 389, Weight = 1 Cluster 19: Value = 316, Weight = 2 Step 2: Merge similar clusters Set the dynamic threshold for merging clusters:

[0065] The maximum difference is 327, meaning the scaling factor is set to r = 0.1 and the dynamic threshold is T = (389-62). 0.1 = 327 0.1 = 32.70

[0066] Merge clusters (iterate): First merge: Merged value = 147, Sum = 148, Distance = 1.00 Second merge: Merged value = 247, sum = 248, distance = 1.00 Third merge: Merged value = 79, sum = 80, distance = 1.00 Fourth merge: Merged value = 249, sum = 247.5, distance = 1.50 Fifth merge: Merged value = 387, sum = 389, distance = 2.00 6th merge: Merged value = 156, sum = 159, distance = 3.00 7th merge: Merged value = 76, Sum = 79.5, Distance = 3.50 8th merge: Merged value = 151, sum = 147.18, distance = 3.82 9th merge: Merged value = 62, Sum = 69, Distance = 7.00 10th merge: Merged value = 156.75, sum = 147.5, distance = 9.25 11th merge: Merged value = 215, sum = 225, distance = 10.00 12th merge: Merged value = 78.33, sum = 62.7, distance = 15.63 13th merge: Merged value = 201, Sum = 217.5, Distance = 16.50 The remaining clusters after the iteration ends Cluster 1: Value = 316.00, Weight = 2 Cluster 2: Value = 148.43, Weight = 40 Cluster 3: Value = 388.00, Weight = 2 Cluster 4: Value = 64.12, Weight = 33 Cluster 5: Value = 203.64, Weight = 25 Cluster 6: Value = 248.40, Weight = 5 Step 3: Obtain the final cluster set after removing outlier clusters. Remove outlier clusters The total weight is 107, and the threshold ratio is set to 0.02 according to the function, meaning the weight is less than 107. Clusters with a weight of 0.02 = 2.14 will be removed as outliers. The outliers removed this time are clusters 1 and 3, whose weights are less than 2.14.

[0067] Obtain the final cluster set and sort it in descending order of weight.

[0068] Cluster 1: Value = 148.43, Weight = 40, Weight percentage = 38.83%

[0069] Cluster 2: Value = 64.12, Weight = 33, Weight percentage = 32.04%

[0070] Cluster 3: Value = 203.64, Weight = 25, Weight percentage = 24.27%

[0071] Cluster 4: Value = 248.40, Weight = 5, Weight percentage = 4.85%

[0072] Step 4: Malicious Traffic Detection

[0073] The final number of clusters is 4, i.e., k=4. The values ​​of cluster 1 and cluster 2 are 148 and 64.12 respectively, both less than that of cluster 4. And the weight percentages are respectively for cluster 1 ( 38.83% and cluster 2 ( 32.04%, which meets the DoH Trojan determination criteria in the determination function.

[0074] Step 5: Confirm Malicious Communication

[0075] Because the final cluster meets the criteria for DoH Trojan detection, this traffic is identified as a DoH Trojan.

[0076] This invention analyzes the unmaskable behavioral characteristics of DoH encrypted malicious communications and innovatively uses dynamically threshold-controlled clustering to automatically aggregate central points from TLS uplink payload sequences. It then determines traffic types based on cluster distribution characteristics, eliminating reliance on traditional methods that depend on domain whitelists and heartbeat interval detection. This reduces false positives and improves detection accuracy. Furthermore, it establishes a systematic behavior analysis and judgment process, enhancing detection efficiency.

[0077] This invention also provides a DoH malicious encrypted communication detection system based on TLS payload similarity, comprising: The data stream acquisition module is used to capture encrypted network traffic, identify all DoH communication streams, extract TLS payload sequences from them, and store the TLS payload sequences in the streams in an array. The uplink load acquisition module is used to extract the uplink load sequence array from the array; The initial cluster construction module is used to construct an initial cluster set by counting each unique value and its occurrence frequency from the uplink payload sequence array; in the initial cluster, the unique value is used as the cluster value, and the occurrence frequency is used as the cluster weight; The new cluster merging module is used to calculate the distance between any two clusters, find all cluster pairs that satisfy the condition that the distance is less than the dynamic threshold, and select the pair with the smallest distance to merge to obtain a new cluster set. The cluster removal module is used to remove outlier clusters from the new cluster set to obtain the final cluster set; The malicious traffic detection module is used to determine whether DoH communication flows are disguised malicious traffic based on the final cluster value and weight.

[0078] Compared with the prior art, the beneficial effects of the DoH malicious encrypted communication detection system based on TLS payload similarity provided by the present invention are the same as the beneficial effects of the DoH malicious encrypted communication detection method based on TLS payload similarity described in the above technical solution, and will not be repeated here.

[0079] The present invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. When the computer program is executed by the processor, it implements the steps in the above-described method for detecting DoH malicious encrypted communication based on TLS payload similarity. Compared with the prior art, the beneficial effects of the electronic device provided by the present invention are the same as those of the above-described method for detecting DoH malicious encrypted communication based on TLS payload similarity, and will not be elaborated here.

[0080] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps in the above-described method for detecting DoH malicious encrypted communication based on TLS payload similarity. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present invention are the same as the beneficial effects of the above-described method for detecting DoH malicious encrypted communication based on TLS payload similarity, and will not be elaborated here.

[0081] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A DoH malicious encrypted communication detection method based on TLS payload similarity, characterized in that, include: Step 1: Capture encrypted network traffic and identify all DoH communication streams, extract the TLS payload sequence from them, and store the TLS payload sequence in the stream in an array; Step 2: Extract the uplink load sequence array from the array; Step 3: Count each unique value and its occurrence frequency from the uplink load sequence array to construct an initial cluster set; where the unique value is used as the cluster value and the occurrence frequency is used as the cluster weight in the initial cluster. Step 4: Calculate the distance between any two clusters, find all cluster pairs that satisfy the condition that the distance is less than the dynamic threshold, and select the pair with the smallest distance to merge to obtain a new cluster set; Step 5: Remove outlier clusters from the new cluster set to obtain the final cluster set; Step 6: Determine whether the DoH communication flow is a disguised malicious flow based on the final cluster value and weight.

2. The DoH malicious encrypted communication detection method based on TLS payload similarity according to claim 1, characterized in that, In step 1, TLS payload sequences of streams with the same source / destination IP and source / destination port are extracted from the DoH communication streams, and the TLS payload sequences in the streams are stored in an array. Uplink payload length is represented by a positive number, and downlink payload length is represented by a negative number. The number of uplink payloads in the payload sequence array is greater than 70. If it is less than 70, all TLS payload sequences of the next stream are extracted and appended to the end of the array until the number of uplink payloads in the array is greater than 70.

3. The DoH malicious encrypted communication detection method based on TLS payload similarity according to claim 1, characterized in that, In step 4, the distance between the two clusters is calculated using the following formula: in, The distance between the two clusters. For the p-th cluster The value, For the q-th cluster The value of .

4. The DoH malicious encrypted communication detection method based on TLS payload similarity according to claim 3, characterized in that, In step 4, find all that satisfy For clusters ≤ T, select the pair with the smallest distance and merge them. If no such pair exists... When there are cluster pairs ≤ T, the iteration terminates to obtain a new cluster set; where the dynamic threshold is... The calculation formula is: in, Represents an array of uplink load sequences. This is the proportionality coefficient.

5. The DoH malicious encrypted communication detection method based on TLS payload similarity according to claim 1, characterized in that, Step 5: Removing outlier clusters from the new cluster set to obtain the final cluster set includes: Step 5.1: Set the threshold function for removing outliers, and calculate the outlier determination threshold based on the threshold function; Step 5.2: Remove clusters whose weights are less than the outlier threshold to obtain the final cluster set; the outlier determination process is as follows: in, Let be the weight of the i-th cluster. For threshold function, The sum of the weights of all clusters. This is the threshold for determining outlier clusters.

6. The DoH malicious encrypted communication detection method based on TLS payload similarity according to claim 5, characterized in that, Step 6: Determine whether the DoH communication flow is spoofed malicious traffic based on the final cluster value and weight, including: The final cluster set is sorted in descending order of weight to obtain the sequence; The judgment criteria are set based on the total weight and weight percentage of the final cluster set, and malicious traffic is identified; the judgment criteria are as follows: in, This represents the weight percentage of the i-th weight. Let be the weight of the i-th cluster. For the total weight, For the value of the i-th cluster, Indicates the final number of clusters. Indicates the sequence except and The smallest cluster value outside the cluster.

7. A DoH malicious encrypted communication detection system based on TLS payload similarity, characterized in that, include: The data stream acquisition module is used to capture encrypted network traffic, identify all DoH communication streams, extract TLS payload sequences from them, and store the TLS payload sequences in the streams in an array. The uplink load acquisition module is used to extract the uplink load sequence array from the array; The initial cluster construction module is used to construct an initial cluster set by counting each unique value and its occurrence frequency from the uplink payload sequence array; in the initial cluster, the unique value is used as the cluster value, and the occurrence frequency is used as the cluster weight; The new cluster merging module is used to calculate the distance between any two clusters, find all cluster pairs that satisfy the condition that the distance is less than the dynamic threshold, and select the pair with the smallest distance to merge to obtain a new cluster set. The cluster removal module is used to remove outlier clusters from the new cluster set to obtain the final cluster set; The malicious traffic detection module is used to determine whether DoH communication flows are disguised malicious traffic based on the final cluster value and weight.

8. An electronic device comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, characterized in that, When the computer program is executed by the processor, it implements the steps in the DoH malicious encrypted communication detection method based on TLS payload similarity as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the DoH malicious encrypted communication detection method based on TLS payload similarity as described in any one of claims 1-6.

Images