Traffic analysis method, system and device and computer readable storage medium

Abstract

The invention discloses a traffic analysis method, system and device and a computer readable storage medium, and relates to the technical field of network security, and the method comprises the steps: obtaining to-be-processed multi-source data in a target network environment; performing flow analysis on the multi-source data to obtain to-be-processed DNS flow; the DNS traffic to be processed is classified to obtain a first type of DNS traffic and a second type of DNS traffic, the first type of DNS traffic comprises traffic from an intranet host to an intranet DNS transponder, and the second type of DNS traffic comprises traffic from the intranet DNS transponder to an extranet; matching the first type of DNS traffic and the second type of DNS traffic of which the time interval is within a set threshold to obtain candidate DNS traffic pairs; and carrying out flow link reduction on the candidate DNS flow pair to obtain a DNS flow link. The limitation of a single data source perspective is broken through, analysis of cross-regional DNS traffic is realized, and an accurate DNS traffic link can be restored; and the real-time performance and the applicability are relatively high.

Description

Technical Field

[0001] This application relates to the field of network security technology, and more specifically, to a traffic analysis method, system, device, and computer-readable storage medium. Background Technology

[0002] DNS (Domain Name System) is an internet service. As a distributed database that maps domain names to IP addresses, it makes it easier for users to access the internet. Traffic path restoration using DNS can be used to trace the source of network security threats, such as effectively tracking the internal network origins of malicious activities like covert communication via DNS protocols (e.g., DNS tunneling) and data infiltration.

[0003] In the process of reconstructing DNS traffic paths, reconstruction can be performed by analyzing DNS server logs, matching source / destination IPs and ports using simple five-tuple associations, or inferring from rules based on static asset mapping tables. However, the complex topologies commonly found in networks, such as multi-level proxies and NAT (Network Address Translation), fragment the complete DNS request path, making it difficult for these methods to correlate fragmented traffic across regions. Furthermore, since rule bases primarily rely on static network configurations and known traffic patterns, they struggle to detect new attacks exploiting dynamic network environments, such as those using DHCP (Dynamic Host Configuration Protocol) or mobile devices, and the analysis process suffers from significant delays. Existing methods fail to meet the requirements of security operations in terms of accuracy, adaptability, and real-time performance.

[0004] In summary, improving the accuracy, adaptability, and real-time performance of DNS traffic link reconstruction is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a traffic analysis method that can, to some extent, solve the technical problem of how to improve the accuracy, adaptability, and real-time performance of DNS traffic link reconstruction. This application also provides a traffic analysis system, an electronic device, and a computer-readable storage medium.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] A traffic analysis method, comprising:

[0008] Acquire multi-source data to be processed in the target network environment;

[0009] Traffic analysis is performed on the multi-source data to obtain the DNS traffic to be processed;

[0010] The DNS traffic to be processed is classified into a first type of DNS traffic and a second type of DNS traffic. The first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network.

[0011] The first type of DNS traffic and the second type of DNS traffic with time intervals within a set threshold are matched to obtain candidate DNS traffic pairs;

[0012] The candidate DNS traffic pairs are reconstructed to obtain the DNS traffic links.

[0013] Preferably, the step of restoring the traffic links of the candidate DNS traffic pairs to obtain the DNS traffic links includes:

[0014] The candidate DNS traffic pairs that have the same domain name and the same protocol are used as the target DNS traffic pairs;

[0015] The target DNS traffic pair is reconstructed to obtain the DNS traffic link.

[0016] Preferably, the step of performing traffic analysis on the multi-source data to obtain the DNS traffic to be processed includes:

[0017] Traffic data from the multi-source data whose target port is a set port is used as candidate DNS traffic, where the set port includes ports defined according to the DNS traffic protocol;

[0018] Traffic data carrying DNS protocol identifiers from the multi-source data will be used as candidate DNS traffic;

[0019] Traffic data from the multi-source data that matches the set DNS behavior characteristics are used as candidate DNS traffic. The set DNS behavior characteristics include query frequency characteristics and domain name distribution pattern characteristics obtained by statistically analyzing encrypted DNS traffic.

[0020] Determine the DNS traffic to be processed based on the candidate DNS traffic.

[0021] Preferably, classifying the DNS traffic to be processed into a first type of DNS traffic and a second type of DNS traffic includes:

[0022] Obtain network asset configuration information for the target network environment;

[0023] The DNS traffic to be processed is classified according to the network asset configuration information to obtain the first type of DNS traffic and the second type of DNS traffic.

[0024] Preferably, the acquisition of multi-source data to be processed in the target network environment includes:

[0025] Acquire raw data to be processed in the target network environment, including traffic data and DNS log data;

[0026] The original data is standardized to obtain multi-source data.

[0027] Preferably, the standardization process of the original data to obtain multi-source data includes:

[0028] The multi-source data is identified by protocol identifier and / or field identifier to obtain the data source type;

[0029] According to the data source type, the network layer information of the multi-source data is extracted, and the network layer information includes source IP, destination IP, port, protocol type and timestamp;

[0030] The DNS log data from the multi-source data is parsed to obtain the query domain name, query type, and response code to obtain DNS log resolution information;

[0031] The network layer information and the DNS log resolution information are mapped to a set data model to obtain candidate data;

[0032] The candidate data is verified for completeness and accuracy, and the verified candidate data is used as multi-source data to be processed.

[0033] Preferably, after performing traffic link reconstruction on the candidate DNS traffic pairs to obtain the DNS traffic links, the method further includes:

[0034] Generate a DNS traffic link map based on the DNS traffic links;

[0035] The DNS traffic link graph is security-filtered according to the established threat intelligence database to obtain a malicious DNS link graph;

[0036] The malicious DNS link spectrum was traced to obtain the malicious DNS tracing results.

[0037] A traffic analysis system, comprising:

[0038] The multi-source data acquisition module is used to acquire multi-source data to be processed in the target network environment;

[0039] The traffic analysis module is used to perform traffic analysis on the multi-source data to obtain the DNS traffic to be processed;

[0040] The traffic classification module is used to classify the DNS traffic to be processed to obtain a first type of DNS traffic and a second type of DNS traffic. The first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network.

[0041] The traffic matching module is used to match the first type of DNS traffic and the second type of DNS traffic with a time interval within a set threshold to obtain candidate DNS traffic pairs;

[0042] The traffic link restoration module is used to restore the traffic links of the candidate DNS traffic pairs to obtain the DNS traffic links.

[0043] An electronic device, comprising:

[0044] Memory, used to store computer programs;

[0045] A processor for executing the computer program to implement any of the traffic analysis methods described above.

[0046] A computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the above-described traffic analysis methods.

[0047] This application provides a traffic analysis method that acquires multi-source data to be processed in a target network environment; performs traffic analysis on the multi-source data to obtain DNS traffic to be processed; classifies the DNS traffic to be processed to obtain a first type of DNS traffic and a second type of DNS traffic, wherein the first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network; matches the first type of DNS traffic and the second type of DNS traffic with time intervals within a set threshold to obtain candidate DNS traffic pairs; and restores the traffic links of the candidate DNS traffic pairs to obtain the DNS traffic links. In this application, because multi-source data is provided by different sources in the target network environment, it can provide DNS traffic-related data from multiple perspectives, overcoming the limitations of a single data source perspective. This allows for more accurate and effective data for subsequent DNS traffic link reconstruction. Furthermore, through traffic analysis and classification, traffic from internal network hosts to internal network DNS forwarders and traffic from internal network DNS forwarders to the external network can be obtained, achieving the resolution of cross-regional DNS traffic. Then, by matching first-type DNS traffic and second-type DNS traffic with time intervals within a set threshold, since the time span of first-type DNS traffic and second-type DNS traffic on the same link before and after the internal network DNS forwarder is not large, cross-regional first-type DNS traffic and second-type DNS traffic can be accurately matched together, obtaining accurate candidate DNS traffic pairs, and thus reconstructing accurate DNS traffic links. The entire processing is not limited by rule base updates to network configuration, has low processing latency, high real-time performance, and is not limited by network environment, making it highly applicable. The traffic analysis system, electronic device, and computer-readable storage medium provided in this application also solve the corresponding technical problems. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 A flowchart of a traffic analysis method provided in an embodiment of this application;

[0050] Figure 2 This is a flowchart for traffic analysis based on DNS traffic characteristics;

[0051] Figure 3 This is a flowchart for traffic analysis based on network asset configuration;

[0052] Figure 4 This is a schematic diagram of the structure of a traffic analysis system provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of the hardware structure of the electronic device according to an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] Please see Figure 1 , Figure 1 A flowchart of a traffic analysis method provided in an embodiment of this application.

[0056] This application provides a traffic analysis method that may include the following steps:

[0057] Step S101: Obtain multi-source data to be processed in the target network environment.

[0058] In practical applications, one can first acquire multi-source data to be processed within the target network environment. Multi-source data refers to data from different data sources. For example, multi-source data could be data collected from gateways, switches, etc., by various traffic monitoring tools set up within the target network environment. Furthermore, the target network environment can be flexibly determined based on the application scenario. For instance, the target network environment could be an enterprise intranet environment, a cloud computing environment, an IoT network environment, or an industrial control system network environment.

[0059] In an exemplary embodiment, considering the differences between data provided by different data sources, in order to eliminate the impact of these differences on DNS traffic link restoration, during the process of acquiring multi-source data to be processed in the target network environment, the original data to be processed in the target network environment can be acquired. The original data may include traffic data carrying port, IP and other information, DNS log data, etc., and the original data can be obtained by collecting the data sources through real-time streaming or batch file collection according to the characteristics of different data sources; the original data is then standardized to obtain multi-source data.

[0060] In specific application scenarios, during the standardization process of raw data to obtain multi-source data, standardization can be performed according to the need for link reconstruction of DNS traffic. For example, the source of multi-source data can be identified according to protocol identifiers and / or field identifiers to obtain the data source type. For example, based on the characteristics of the data protocol, the data source type can be determined as DNS log, TCP log, or other types of logs. Network layer information of multi-source data is extracted according to the data source type. Network layer information includes source IP, destination IP, port, protocol type, and timestamp. The query domain name, query type, and response code of DNS log data in multi-source data are parsed to obtain DNS log resolution information. The network layer information and DNS log resolution information are mapped to a set data model to obtain candidate data. The set data model is used to manage DNS traffic-related data and can be flexibly determined according to the application scenario. For example, it can include IP, port, process commands, etc. The candidate data is verified for completeness and accuracy. The candidate data that passes the verification is used as the multi-source data to be processed to ensure the completeness and accuracy of the multi-source data.

[0061] Step S102: Perform traffic analysis on the multi-source data to obtain the DNS traffic to be processed.

[0062] Step S103: Classify the DNS traffic to be processed to obtain the first type of DNS traffic and the second type of DNS traffic. The first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network.

[0063] In practical applications, considering that the process involves link reconstruction of DNS traffic, and that multi-source data may contain data that does not belong to DMS traffic, traffic analysis is required to obtain the DNS traffic to be processed. Subsequently, considering that the target network environment may have DNS forwarders to forward internal DNS traffic to the external network, the DNS traffic to be processed can be classified into two categories: Category 1 DNS traffic and Category 2 DNS traffic. Category 1 DNS traffic includes traffic from internal hosts to internal DNS forwarders, and Category 2 DNS traffic includes traffic from internal DNS forwarders to the external network. Here, a DNS forwarder can be a DNS server, etc. It should be noted that if the DNS traffic to be processed does not belong to Category 1 or Category 2 DNS traffic, it can be classified as other traffic, such as direct external traffic or communication traffic between DNS servers. In this application, other traffic may not be used for DNS traffic link reconstruction.

[0064] Step S104: Match the first type of DNS traffic and the second type of DNS traffic with time intervals within a set threshold to obtain candidate DNS traffic pairs.

[0065] In practical applications, DNS traffic forwarding can be understood as traffic from A to B. Assuming a domain name request is for aac.com, and B forwards it to C (aac.com), due to the characteristics and efficient processing capabilities of routing and other devices in the target network environment, M-level DNS traffic forwarding can be achieved in seconds or minutes. Therefore, the data collection software uploads collected data in a similar forwarding manner, resulting in the time interval between DNS traffic in multi-source data and the actual DNS traffic being basically consistent. Although data storage and processing take time, causing a certain time delay between DNS traffic in multi-source data and the original DNS traffic (e.g., network transmission causing later-generated data to be collected first), the high efficiency of computer design allows DNS traffic forwarding to be completed within minutes. This means that the time interval between DNS traffic belonging to the same link in multi-source data is not large. Therefore, first-type DNS traffic and second-type DNS traffic with time intervals within a set threshold can be matched to obtain candidate DNS traffic pairs. The set threshold can be flexibly determined according to the application scenario; for example, a minute-level threshold can be dynamically designed based on the traffic volume. Alternatively, the set threshold can be a fixed threshold. For ease of understanding, assume there is a Type I DNS traffic from A to B and a Type II DNS traffic from B to C. Set a threshold of 3 minutes. If the traffic from A to B and B to C is obtained within 3 minutes, it can be used as a candidate DNS traffic pair for subsequent analysis of the entire A to C link. If the time interval between obtaining the traffic from A to B and obtaining the traffic from B to C exceeds 3 minutes, then the traffic from A to B and B to C can be considered as DNS traffic from two different sources and cannot be used as a candidate DNS traffic pair.

[0066] In an exemplary embodiment, during the process of matching first-type DNS traffic and second-type DNS traffic with time intervals within a set threshold to obtain candidate DNS traffic pairs, a time window with a duration consistent with the set threshold can be set. The time window can be a fixed time window, a sliding time window, an adaptive time window, etc. First-type DNS traffic and second-type DNS traffic are collected according to the time window, and the collected first-type DNS traffic and second-type DNS traffic are matched as candidate DNS traffic pairs, so as to quickly determine candidate DNS traffic pairs with the help of the time window.

[0067] Step S105: Perform traffic link restoration on the candidate DNS traffic pairs to obtain the DNS traffic links.

[0068] In practical applications, the DNS traffic in candidate DNS traffic pairs may belong to the same DNS traffic link. Therefore, the traffic link of the candidate DNS traffic pairs can be restored to obtain the DNS traffic link. In this process, considering that traffic belonging to the same DNS traffic link satisfies both domain name consistency and protocol consistency, the candidate DNS traffic pairs with consistent domain names and protocols can be used as target DNS traffic pairs. Domain name consistency can mean that the request domain name is the same, and protocol consistency can mean that both are DNS protocols. The traffic link of the target DNS traffic pairs is restored to obtain the DNS traffic link. The form of the DNS link can be flexibly determined according to the application scenario. For example, the DNS link can include the real original initiator (Source) and the final recipient (Destination) of the DNS traffic, and the content can be "internal network host IP → internal network DNS server IP → external network DNS server IP", etc.

[0069] In an exemplary embodiment, traffic links are reconstructed for candidate DNS traffic pairs to obtain DNS traffic links. The role of DNS traffic links can be flexibly determined according to the application scenario. For example, DNS traffic links can be used for network behavior analysis, security threat tracing, etc. Taking security threat tracing as an example, a DNS traffic link map can be generated based on the DNS traffic links to visually describe the topology of the proxy server. The DNS traffic link map is then security-filtered according to the set threat intelligence database to obtain a malicious DNS link map. The malicious DNS link map is then traced to obtain malicious DNS tracing results. Malicious DNS tracing results may include external malicious internal network hosts, etc., without specific limitations here.

[0070] This application provides a traffic analysis method that acquires multi-source data to be processed in a target network environment; performs traffic analysis on the multi-source data to obtain DNS traffic to be processed; classifies the DNS traffic to be processed to obtain a first type of DNS traffic and a second type of DNS traffic, wherein the first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network; matches the first type of DNS traffic and the second type of DNS traffic with time intervals within a set threshold to obtain candidate DNS traffic pairs; and restores the traffic links of the candidate DNS traffic pairs to obtain the DNS traffic links. In this application, since multi-source data is provided by different sources in the target network environment, it can provide DNS traffic-related data from multiple perspectives, breaking through the limitations of a single data source perspective. This can provide more accurate and effective data for subsequent DNS traffic link reconstruction. Furthermore, through traffic analysis and traffic classification, traffic from internal network hosts to internal network DNS forwarders and traffic from internal network DNS forwarders to the external network can be obtained, realizing the resolution of cross-regional DNS traffic. Then, if the first type of DNS traffic and second type of DNS traffic with time intervals within a set threshold are matched, since the time span of the first type of DNS traffic and second type of DNS traffic on the same link before and after the internal network DNS forwarder is not large, the cross-regional first type of DNS traffic and second type of DNS traffic can be accurately matched together to obtain accurate candidate DNS traffic pairs, thereby reconstructing accurate DNS traffic links. The entire processing process is not limited by the rule base updating network configuration, has low processing latency, high real-time performance, and is not limited by the network environment, making it highly applicable.

[0071] Based on the above embodiments, since link reconstruction is performed on DNS traffic, traffic analysis needs to be based on the characteristics of DNS traffic. Please refer to [link to relevant documentation]. Figure 2 The traffic analysis method provided in this application embodiment may include the following steps:

[0072] Step S201: Obtain multi-source data to be processed in the target network environment.

[0073] Step S202: Select traffic data from the multi-source data whose destination port is a set port as candidate DNS traffic. The set port includes ports defined according to the DNS traffic protocol.

[0074] In practical applications, considering that the DNS protocol defines the port for DNS traffic transmission, traffic data from multiple sources with a target port set as the port can be used as candidate DNS traffic. The target port includes ports defined according to the DNS traffic protocol, such as port 53. The traffic data on the target port can be TCP (Transmission Control Protocol) / UDP (User Datagram Protocol) traffic data transmitted through port 53, etc.

[0075] Step S203: Select traffic data carrying DNS protocol identifiers from multi-source data as candidate DNS traffic.

[0076] In practical applications, DNS traffic data carries DNS protocol identifiers, so traffic data carrying DNS protocol identifiers from multiple sources can be used as candidate DNS traffic.

[0077] Step S204: Select traffic data from the multi-source data that matches the set DNS behavior characteristics as candidate DNS traffic. The set DNS behavior characteristics include query frequency characteristics and domain name distribution pattern characteristics obtained after statistical analysis of encrypted DNS traffic.

[0078] In practical applications, considering that DNS traffic may be encrypted in certain scenarios, the characteristics of encrypted DNS traffic will be inconsistent with those of unencrypted DNS traffic, making it impossible to identify it using the same characteristics as unencrypted DNS traffic. To accurately identify encrypted DNS traffic, analysis is conducted. Encrypted DNS traffic exhibits specific statistical patterns in terms of time series and packet size. For example, unlike ordinary file downloads or web browsing traffic, encrypted DNS traffic queries typically manifest as dense, short bursts of traffic with specific intervals. Therefore, DNS query behavior can be distinguished by analyzing the number of requests and packet arrival intervals per unit time, thereby identifying encrypted DNS traffic. Furthermore, although the content of encrypted DNS traffic is encrypted, the target address (destination IP) for establishing the connection or the SNI (Server Name Indication) during the handshake phase often points to a specific public DNS service provider (such as 8.8.8.8 for service provider A, 1.1.1.1 for service provider B, or a specific DoH domain name). In other words, encrypted DNS traffic has specific domain name distribution patterns, which can be used to identify encrypted DNS traffic. In short, query frequency characteristics are used to characterize the appearance of encrypted DNS traffic, and domain distribution pattern characteristics are used to characterize the destination of encrypted DNS. Therefore, traffic data from multiple sources that are consistent with the set DNS behavior characteristics can be used as candidate DNS traffic. The set DNS behavior characteristics include query frequency and domain distribution pattern characteristics obtained after statistical analysis of encrypted DNS traffic.

[0079] Step S205: Determine the DNS traffic to be processed based on the candidate DNS traffic.

[0080] In practical applications, after obtaining candidate DNS traffic, the DNS traffic to be processed can be determined based on the candidate DNS traffic. For example, candidate DNS traffic obtained by setting the port and DNS protocol identifier can be used as target DNS traffic, and candidate DNS traffic obtained by setting DNS behavior characteristics can also be used as target DNS traffic.

[0081] Step S206: Classify the DNS traffic to be processed to obtain the first type of DNS traffic and the second type of DNS traffic. The first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network.

[0082] Step S207: Match the first type of DNS traffic and the second type of DNS traffic with time intervals within a set threshold to obtain candidate DNS traffic pairs.

[0083] Step S208: Perform traffic link restoration on the candidate DNS traffic pairs to obtain the DNS traffic links.

[0084] As can be seen from the implementation process, in the process of analyzing traffic from multi-source data to obtain the DNS traffic to be processed, this application uses traffic data from multi-source data with a target port as a set port as candidate DNS traffic. The set port includes ports defined according to the DNS traffic protocol. Traffic data from multi-source data carrying DNS protocol identifiers is also used as candidate DNS traffic. This achieves accurate identification of unencrypted DNS traffic by leveraging the port characteristics and protocol identifiers reflected in it. Furthermore, considering the DNS encryption scenario, the query frequency characteristics and domain name distribution pattern characteristics obtained by statistically analyzing encrypted DNS traffic are used as set DNS behavior characteristics. Traffic data from multi-source data that matches the set DNS behavior characteristics is used as candidate DNS traffic, achieving accurate filtering of encrypted DNS traffic. Finally, if the DNS traffic to be processed is determined based on the candidate DNS traffic, both encrypted and unencrypted DNS traffic can be accurately filtered, ensuring the comprehensiveness of DNS traffic filtering and facilitating accurate subsequent DNS traffic link reconstruction.

[0085] Based on the above embodiments, if a malicious device exists in the target network environment and participates in DNS traffic forwarding, DNS traffic related to the malicious device will be obtained. However, since the malicious device is not a user's network asset, it is not conducive to the user's management of network assets in the target network environment. To avoid this situation, please refer to [link to relevant documentation]. Figure 3 The traffic analysis method provided in this application embodiment may include the following steps:

[0086] Step S301: Obtain multi-source data to be processed in the target network environment.

[0087] Step S302: Perform traffic analysis on the multi-source data to obtain the DNS traffic to be processed.

[0088] Step S303: Obtain network asset configuration information for the target network environment.

[0089] Step S304: Classify the DNS traffic to be processed according to the network asset configuration information to obtain the first type of DNS traffic and the second type of DNS traffic. The first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network.

[0090] In practical applications, considering that network asset configuration information reflects the logical relationships between various assets (such as servers, personal computers, network devices, etc.) within the target network, including but not limited to contextual information such as IP address range attribution, department to which the device belongs, and business applications carried by the device, network asset configuration information can be used to identify assets within the target network. Therefore, network asset configuration information of the target network environment can be obtained. Taking an enterprise network environment as an example, it is possible to connect to the enterprise's CMDB-type system to obtain the latest configuration periodically, or to combine it with the enterprise's ASM-type system to automatically identify DNS assets. Then, based on the verification results of dynamic asset discovery and the verification based on network topology relationships, the DNS assets are processed to obtain network asset configuration information. Then, based on the network asset configuration information, the DNS traffic to be processed is classified to obtain the first type of DNS traffic and the second type of DNS traffic.

[0091] Step S305: Match the first type of DNS traffic and the second type of DNS traffic with time intervals within a set threshold to obtain candidate DNS traffic pairs.

[0092] Step S306: Perform traffic link restoration on the candidate DNS traffic pairs to obtain the DNS traffic links.

[0093] As can be seen from this implementation process, in classifying the DNS traffic to be processed into Category I and Category II DNS traffic, the network asset configuration information of the target network environment is first obtained. Then, the DNS traffic to be processed is classified according to the network asset configuration information to obtain Category I and Category II DNS traffic. Since the network asset configuration information reflects the various assets under user control within the target network, classifying the DNS traffic to be processed based on the network asset configuration information can verify the legitimacy of the DNS traffic participants, filter out Category I and Category II DNS traffic related to user-controlled assets, and ensure the accuracy and reliability of the reconstruction results when performing DNS traffic link restoration subsequently. Furthermore, it can obtain the relevant DNS traffic links of user-controlled assets, facilitating user management of controlled assets based on the DNS traffic links.

[0094] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a traffic analysis system provided in an embodiment of this application.

[0095] This application provides a traffic analysis system that may include:

[0096] Multi-source data acquisition module 101 is used to acquire multi-source data to be processed in the target network environment;

[0097] Traffic analysis module 102 is used to perform traffic analysis on multi-source data to obtain the DNS traffic to be processed;

[0098] The traffic classification module 103 is used to classify the DNS traffic to be processed to obtain the first type of DNS traffic and the second type of DNS traffic. The first type of DNS traffic includes the traffic from the internal network host to the internal network DNS forwarder, and the second type of DNS traffic includes the traffic from the internal network DNS forwarder to the external network.

[0099] The traffic matching module 104 is used to match first-type DNS traffic and second-type DNS traffic with time intervals within a set threshold to obtain candidate DNS traffic pairs;

[0100] The traffic link restoration module 105 is used to restore the traffic links of candidate DNS traffic pairs to obtain the DNS traffic links.

[0101] This application provides a traffic analysis system, in which a traffic link reconstruction module may include:

[0102] The traffic pair filtering unit is used to select candidate DNS traffic pairs with the same domain name and protocol as target DNS traffic pairs;

[0103] The traffic link restoration unit is used to restore the traffic links of the target DNS traffic pair to obtain the DNS traffic links.

[0104] This application provides a traffic analysis system, wherein the traffic analysis module may include:

[0105] The port analysis unit is used to identify traffic data from multiple sources with a target port of a specified port as candidate DNS traffic. The specified port includes ports defined according to the DNS traffic protocol.

[0106] The protocol identifier analysis unit is used to identify traffic data carrying DNS protocol identifiers from multi-source data as candidate DNS traffic.

[0107] The behavioral feature analysis unit is used to select traffic data from multi-source data that is consistent with the set DNS behavioral features as candidate DNS traffic. The set DNS behavioral features include query frequency features and domain name distribution pattern features obtained after statistical analysis of encrypted DNS traffic.

[0108] The traffic analysis unit is used to determine the DNS traffic to be processed based on the candidate DNS traffic.

[0109] This application provides a traffic analysis system, in which a traffic classification module may include:

[0110] The asset configuration information acquisition unit is used to acquire network asset configuration information of the target network environment;

[0111] The traffic classification unit is used to classify the DNS traffic to be processed according to the network asset configuration information to obtain the first type of DNS traffic and the second type of DNS traffic.

[0112] This application provides a traffic analysis system, in which a multi-source data acquisition module may include:

[0113] The raw data acquisition unit is used to acquire raw data to be processed in the target network environment, including traffic data and DNS log data.

[0114] Standardized units are used to standardize raw data to obtain multi-source data.

[0115] In specific application scenarios, for ease of understanding, the original data may include: Data1: " <857> {{thirdLogTime}} host WAG: SerialNum=0123211512249999 GenTime=\"{{ datetime}}\" Module=Uploaded file type and extension are the same as common webshell files SrcIP={{ outer_ip_0}} DstIP={{ inner_ip_0}} SrcPort=49910 DstPort=8001 In=ge0 / 1 Out=ge0 / 2 Action=drop URL=\"\" EvtCount=1 Evt_level=40 Evt_type=Uploaded file type and extension are the same as common webshell files Evt_log_level=1 Host={{ inner_ip_0}} Evt_response=0BeforeNat="。Data2:{"originProductType": "STA","manage": "65ACE679","srcPort":12345,"logType": "dns","srcType": 0,"srcIpTag": 0,"srcIp": "{{ src_ip}}","dstIp": "{{ dst_ip}}","dstIpTag": 1,"moduleType": "","dstCity": "","dstProvince": "","srcMac": "","recordTime": "2022-08-04 10:05:43","dstMac":"","originProductVer": "STA8.0.70.180 Build20220719","hostIpValid": "","dstCountry": "","vendor": "sangfor","uuId": "dns_log-c1659578743-9634cdd9-f6dc-4e93-8ea2-835a913bdb6","v": 1,"srcCity": "","srcProvince": "","deviceId": "65ACE679","hostIp": "","uploadTimestamp": 1659578745000,"dstPort": 80,"tenant": "88602882","customer": "88602882","productVer": "STA8.0.70.180 Build20220719","uploadTime": "2022-08-04 10:05:45","srcCountry": "","dstType": 0,"recordTimestamp": 1659578745000,"sessionId":"","srcSubType": 0,"productType":"","insertTimestamp": 1659578745000,"dstSubType": 0,"id": 0,"qr": 0,"opCode": 0,"aa": 0,"tc": 0,"rd": 0,"ra": 0,"z": 0,"ad": 0,"cd": 0,"rCode": 0,"qdCnt": 0,"anCnt": 0,"nsCnt": 0,"arCnt":0,"queries": "www.baidu.com","qClasses": "1","qTypes": "1","answers": "2.2.2.2","ttls": "660","aTypes": "","aClasses": "","length": 0}。DATA3:{"originProductType": "STA","responseHead": "rsp_head","manage": "65ACE679","srcPort": 12345,"logType": "tcp","srcType": 0,"srcIpTag": 0,"srcIp": "{{ src_ip}}","dstIp": "{{ dst_ip}}","dstIpTag": 1,"srcSubType": 1,"dration":1000,"serAgent": "Mozilla","modleType": "","dstCity": "shenzhen","dstProvince": "guandom","srcMac": "6C-4B-90-EE-CE-A5","srcAssetId": "sde122","respContentType": "text / html","recordTime": "2022-08-04 10:05:43","dstMac": "6C-4B-90-EE-CE-A5","originProductVer": "STA8.0.70.180Bild20220719","hostIpValid": "","dstCountry": "china","vendor": "sangfor","uuId": "tcp_log-{{ increase}}-{{ rand_num_str_10000_99999}}","respStats":200,"v": 1,"srcCity": "shenhen","productType": "STA","productVer": "STA3.0.39","manageIp": "6.6.6.6","deviceIp": "6.6.6.6","srcProvince": "guangd","deviceId": "65ACE679","hostIp": "","uploadTimestamp":1659578745000,"dstPort": 80,"tenant": "88602882","customer": "88602882","prodctVer": "AF8.0.70.180 Bild20220719","uploadTime": "2022-08-04 10:05:45","srcCountry": "china","dstAssetId": "sce123","dstType": 0,"recordTimestamp": "{{ nowMS}}","sessionId": "sss","srcSbType": 0,"prodctType": "","insertTimestamp": 1659578745000,"dstSubType": 0,"proto": "http","startState":1,"sessionState": 1,"elapsedTime": 0,"upLinkLength": 1,"upAppLength": 1,"downLinkLength": 1,"downAppLength": 1,"downPkts": 1,"upPkts": 1,"appTypeCrc": 1,"appNameCrc": 1,"upAppLengthFlowRate": "","downAppLengthFlowRate": "","interval": 0}. .

[0116] This application provides a traffic analysis system in which a standardized unit can be used to: identify the source of multi-source data according to protocol identifiers and / or field identifiers to obtain the data source type; extract network layer information of the multi-source data according to the data source type, the network layer information including source IP, destination IP, port, protocol type and timestamp; parse the query domain name, query type and response code of DNS log data in the multi-source data to obtain DNS log resolution information; map the network layer information and DNS log resolution information to a set data model to obtain candidate data; verify the integrity and accuracy of the candidate data, and use the candidate data that passes the verification as multi-source data to be processed.

[0117] In specific application scenarios, according to the requirements of DNS traffic link restoration, the following information can be included in the set data model: {"name": "originProductType", "type": "string"},{"name": "originProductVer", "type": "string"},{"name": "deviceId", "type": "string"},{"name": "uuId", "type": "string"},{"name": "hostIp", "type": "string"},{"name": "hostName", "type": "string"},{"name": "hostMac", "type": "string"},{"name": "threatClass", "type": "int"},{"name": "threatType", "type": "int"},{"name": "threatSubType", "type": "int"},{"name": "moduleTypeOrigin", "type": "int"},{"name": "moduleType", "type": "string"},{"name": "threatTypeCode", "type": "string"},{"name": "threatSubTypeCode", "type": "string"},{"name": "severity", "type": "int"},{"name": "confidence", "type": "int"},{"name": "riskLevel", "type": "int"},{"name": "attackState", "type": "int"},{"name": "white", "type": "int"},{"name": "action", "type": "int"},{"name": "engine", "type": "string"},{"name": "engineVersion", "type": "string"},{"name": "engineRuleVersion", "type": "string"},{"name": "assetId", "type": ["null","string"], "default": null},{"name": "cloudTs", "type": ["null", "long"], "default": null},{"name": "processId", "type": ["null","int"], "default":null},{"name": "processGuid", "type": ["null","string"], "default": null},{"name": "processName", "type": ["null","string"], "default": null},{"name": "processCmd", "type": ["null","string"], "default": null},{"name": "processMd5", "type": ["null","string"], "default": null},{"name": "processSha2", "type": ["null","string"], "default": null},{"name": "processSigner", "type": ["null","string"], "default": null},{"name": "processCopyright", "type": ["null","string"], "default": null}, {"name": "processUser", "type": ["null","string"], "default": null},{"name": "processGroup", "type": ["null","string"], "default": null},{"name": "processCurrentDir", "type": ["null","string"], "default": null},{"name": "processStartTime", "type": ["null","long"], "default": null},{"name": "processExitCode", "type": ["null","int"], "default": null},{"name": "processLoginGuid", "type": ["null","string"], "default": null}, {"name": "networkDstPort", "type": ["null", "int"], "default": null},{"name": "networkProtocol", "type": ["null", "string"], "default": null},{"name": "networkHostName", "type": ["null", "string"], "default": null},{"name": "dnsQueryName", "type": ["null", "string"], "default": null},{"name": "dnsStatus", "type": ["null", "int"], "default": null},{"name": "dnsAnswers","type": ["null", "string"], "default": null}. ,

[0118] The traffic analysis system provided in this application embodiment may further include:

[0119] The link image generation module is used by the traffic link restoration module to restore the traffic links of candidate DNS traffic pairs. After obtaining the DNS traffic links, a DNS traffic link map is generated based on the DNS traffic links.

[0120] The security filtering module is used to perform security filtering on the DNS traffic link graph according to the set threat intelligence database to obtain a malicious DNS link graph;

[0121] The tracing module is used to trace the malicious DNS link graph and obtain the malicious DNS tracing results.

[0122] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 5 This is a schematic diagram of the hardware structure of the electronic device according to an embodiment of this application, as shown below. Figure 5 As shown, the electronic device includes:

[0123] Communication interface 1 enables information exchange with other devices, such as network devices;

[0124] Processor 2 is connected to communication interface 1 to enable information exchange with other devices and, when running a computer program, executes the traffic analysis methods provided by one or more of the above-mentioned technical solutions. The computer program is stored in memory 3.

[0125] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 4 The general will label all buses as Bus System 4.

[0126] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0127] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0128] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0129] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0130] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by a processor 2 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0131] In an exemplary embodiment, this application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any of the preceding embodiments.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, terminal, and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0133] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0134] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0135] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0136] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0137] For descriptions of relevant parts of the traffic analysis system, electronic device, and computer-readable storage medium provided in the embodiments of this application, please refer to the detailed descriptions of the corresponding parts in the traffic analysis method provided in the embodiments of this application, and they will not be repeated here. Furthermore, parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0138] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0139] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A flow analysis method, characterized in that, include: Acquire multi-source data to be processed in the target network environment; Traffic analysis is performed on the multi-source data to obtain the DNS traffic to be processed; The DNS traffic to be processed is classified into a first type of DNS traffic and a second type of DNS traffic. The first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network. The first type of DNS traffic and the second type of DNS traffic with time intervals within a set threshold are matched to obtain candidate DNS traffic pairs; The candidate DNS traffic pairs are reconstructed to obtain the DNS traffic links.

2. The method according to claim 1, characterized in that, The step of restoring the traffic links of the candidate DNS traffic pairs to obtain the DNS traffic links includes: The candidate DNS traffic pairs that have the same domain name and the same protocol are used as the target DNS traffic pairs; The target DNS traffic pair is reconstructed to obtain the DNS traffic link.

3. The method according to claim 1, characterized in that, The step of performing traffic analysis on the multi-source data to obtain the DNS traffic to be processed includes: Traffic data from the multi-source data with a target port of a set port is used as candidate DNS traffic, where the set port includes ports defined according to the DNS traffic protocol; Traffic data carrying DNS protocol identifiers from the multi-source data will be used as candidate DNS traffic; Traffic data from the multi-source data that matches the set DNS behavior characteristics are used as candidate DNS traffic. The set DNS behavior characteristics include query frequency characteristics and domain name distribution pattern characteristics obtained by statistically analyzing encrypted DNS traffic. Determine the DNS traffic to be processed based on the candidate DNS traffic.

4. The method according to claim 1, characterized in that, The process of classifying the DNS traffic to be processed into a first category of DNS traffic and a second category of DNS traffic includes: Obtain network asset configuration information for the target network environment; The DNS traffic to be processed is classified according to the network asset configuration information to obtain the first type of DNS traffic and the second type of DNS traffic.

5. The method according to claim 1, characterized in that, The acquisition of multi-source data to be processed in the target network environment includes: Acquire raw data to be processed in the target network environment, including traffic data and DNS log data; The original data is standardized to obtain multi-source data.

6. The method according to claim 5, characterized in that, The standardization process of the original data to obtain multi-source data includes: The multi-source data is identified by protocol identifier and / or field identifier to obtain the data source type; According to the data source type, the network layer information of the multi-source data is extracted, and the network layer information includes source IP, destination IP, port, protocol type and timestamp; The DNS log data from the multi-source data is parsed to obtain the query domain name, query type, and response code to obtain DNS log resolution information; The network layer information and the DNS log resolution information are mapped to a set data model to obtain candidate data; The candidate data is verified for completeness and accuracy, and the verified candidate data is used as multi-source data to be processed.

7. The method according to claim 1, characterized in that, After restoring the traffic links of the candidate DNS traffic pairs to obtain the DNS traffic links, the process further includes: Generate a DNS traffic link map based on the DNS traffic links; The DNS traffic link graph is security-filtered according to the established threat intelligence database to obtain a malicious DNS link graph; The malicious DNS link spectrum was traced to obtain the malicious DNS tracing results.

8. A flow analysis system, characterized in that, include: The multi-source data acquisition module is used to acquire multi-source data to be processed in the target network environment; The traffic analysis module is used to perform traffic analysis on the multi-source data to obtain the DNS traffic to be processed; The traffic classification module is used to classify the DNS traffic to be processed to obtain a first type of DNS traffic and a second type of DNS traffic. The first type of DNS traffic includes traffic from internal network hosts to internal network DNS forwarders, and the second type of DNS traffic includes traffic from internal network DNS forwarders to the external network. The traffic matching module is used to match the first type of DNS traffic and the second type of DNS traffic with a time interval within a set threshold to obtain candidate DNS traffic pairs; The traffic link restoration module is used to restore the traffic links of the candidate DNS traffic pairs to obtain the DNS traffic links.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the traffic analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the traffic analysis method as described in any one of claims 1 to 7.

Images