A flow-based deep-level intranet topology inference method and system

Abstract

The application discloses a deep-level internal network topology inference method and system based on traffic, and belongs to the technical field of network topology construction. The method comprises the following steps: acquiring NTP traffic features and TCP traffic features of traffic; inferring the Windows device scale and the Linux device scale of each level of internal network according to the NTP traffic features; performing hierarchical feature clustering on TCP traffic based on the TCP traffic features and the Windows device scale and the Linux device scale of each level of internal network, so as to obtain a TCP data packet array classified according to host devices; and performing host device relationship inference based on the TCP data packet array classified according to host devices, so as to construct a deep-level internal network topology. The application can accurately infer the network level of a deep-level local area network.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of network topology construction, and particularly relates to a deep-level intranet topology structure inference method and system based on traffic. BACKGROUND

[0002] Nowadays, with the rapid development of new information technologies such as computers, big data, cloud computing, the Internet of Things, 5G, etc., network equipment is being upgraded, scaled up, and structured, and the requirements for network management are becoming higher, especially the information security assurance, network management real-time performance, and effectiveness of defense measures of enterprises.

[0003] With the rapid development of the network, the Internet of Things has not only become a concept, but also a reality. Various Internet of Things devices penetrate every aspect of life, such as smart cameras, smart phones, smart homes, etc. While the network brings convenience to people, it also brings some hidden problems. First, the security problem, the network security problem caused by the comprehensive popularization and application of the network gradually highlights, which brings a huge threat to the information security of enterprises. Especially the concealment and upgrading of attack means, today's attackers can use various high-tech means to hide their positions, making many security measures ineffective. Second, the management problem, the large-scale and complex structure of the intranet has become an irreversible trend, as the number of devices in the intranet continues to increase and the structure of the subnetwork continues to expand, the management of the entire network becomes more difficult. When there is a fault in the network, if the fault cannot be located in time, the loss will be incalculable in many cases. In addition, the defense problem, only by understanding the topology structure of the entire network, when unknown sources of hosts and devices appear, it can be quickly discovered and the possible network attack path and attack target can be predicted in advance, and the network security line can be advanced. If the defense mechanism and strategy of the network are not perfect, it may lead to unpredictable disasters.

[0004] In terms of the Internet, in network operation and data management, most enterprises are facing various problems. Some problems are caused by internal mistakes, such as lack of network security awareness or lack of reliable security precautions; other problems may be caused by external threats. Network attacks often do not occur suddenly, in the "network kill chain" (KillChain) model, Martin divides it into 7 stages: target reconnaissance, initial attack, privilege escalation, internal reconnaissance, lateral movement, persistent residence, and target achievement. Attackers will first conduct reconnaissance on the selected target, first externally using various means to understand the network situation and resource situation of the target attack object, and then break through the defense line by means such as water pit attack, spear phishing attack, vulnerability penetration, etc. to enter the target system, and then further penetrate and attack by means such as probing the intranet environment, escalating privileges, and implanting persistent backdoors, and finally launching attack actions to achieve their goals.

[0005] In summary, a comprehensive and accurate network topology can effectively help complete tasks such as network fault location, asset inventory, security design, etc. However, important enterprises represented by schools, scientific research and military industries have many characteristics such as numerous types of network equipment, complex partition and domain management, and complex security policy configuration, which make internal network topology mapping an important foundation to solve the above problems.

[0006] The acquisition of internal network topology is of great significance, but the commonly used detection technology cannot realize the detection of the entire internal network, mainly because the internal network contains a large number of routers that open NAT services. NAT, or Network Address Translation, is a common network technology commonly used in home or enterprise networks. It allows the use of private IP addresses within a local area network and public IP addresses on the Internet, thus enabling local area network hosts to access the Internet. NAT services are usually provided by network devices such as routers, which map the IP addresses and port numbers of local area network hosts to public IP addresses and port numbers, and forward data packets between the Internet and the local area network. This technology can effectively reduce the use of IP addresses, improve network security, and simplify network management. However, it is also due to NAT that will affect the topology inference function based on traffic.

[0007] Currently, due to the existence of NAT, there are many difficulties in the process of network topology inference. For example, the existence of gateway devices such as network address translation will affect the accuracy of capturing traffic from outside the target network to infer the target network topology. The existence of these devices makes it difficult to determine the number of internal network host devices, the relationship is difficult to judge, and the topology of the deep internal network is difficult to obtain. In recent years, methods based on TCP / IP protocol field changes or traffic clustering have been gradually applied to topology inference, but more focus on the detection of NAT devices and the determination of host scale, or it is difficult to effectively integrate related features, which will lead to a large deviation in the inference of network topology, such as the following two methods:

[0008] Identification of Hosts behind a NAT Device Utilizing Multiple Fields of IP and TCP, published in 2016, is a method that utilizes multiple IP and TCP field characteristics to identify hosts behind a NAT network. The method first performs a TCP SYN scan on the target network, and then analyzes the returned SYN-ACK packets to determine the NAT device IP address in the target network. Next, the method utilizes the changes in the values of different fields to identify the hosts behind the NAT, including the ID and length fields of the IP header and the sequence number and window size fields of the TCP header. However, this technical solution relies on passively observing network traffic, so it cannot directly control the test conditions or actively trigger traffic, which is not suitable for certain scenarios. On the other hand, it requires analysis of multiple IP and TCP fields, which has high algorithm complexity and may cause performance problems. In addition, in some cases, the NAT device may modify the fields in the traffic, causing the algorithm to fail.

[0009] "Exploring NAT Detection and Host Identification Using Machine Learning" published in 2019

Khatouni, Ali Safari et al. "Exploring NAT Detection and Host Identification Using Machine Learning." 2019 15th International Conference on Network and Service Management (CNSM) (2019): 1-8.

[0010] 1. Requires a large amount of data set and computing resources: Since machine learning algorithms are used, a large amount of data set is needed for training and testing, and corresponding computing resources are needed to process these data. This may limit the application of the method, especially in the case of limited computing resources.

[0011] 2. Limited scalability: Since the method is based on machine learning, it may need to be retrained and tested when dealing with new network data. This means that when the network size increases or the network topology changes, the method may not be well extended.

[0012] 3. May not be suitable for some NAT devices: This method uses some general features in network packets for classification and identification, which may not be suitable for some specific types of NAT devices, resulting in inaccurate identification and classification of hosts.

[0013] Therefore, there is currently a need for a technical solution to accurately infer the network hierarchy of deep local area networks. SUMMARY

[0014] In order to solve the difficulty of deep local area network detection caused by NAT technology, etc., the present application proposes a traffic-based deep internal network topology inference method and system, which can accurately infer the network hierarchy of deep local area networks.

[0015] To achieve the above object, the technical scheme of the present application includes the following contents.

[0016] A flow-based deep internal network topology inference method, the method comprising:

[0017] Obtaining NTP flow features and TCP flow features of the flow;

[0018] Inferencing Windows device scale and Linux device scale of each level of internal network according to the NTP flow features;

[0019] Based on the TCP flow features and the Windows device scale and the Linux device scale of each level of internal network, performing hierarchical feature clustering on the TCP flow to obtain a TCP packet array classified by host device;

[0020] Based on the TCP packet array classified by host device, performing host device relationship inference to construct a deep internal network topology.

[0021] Further, the TCP flow features of the flow include:

[0022] Parsing the TCP flow packet and extracting fields in the TCP handshake packet, the fields in the TCP handshake packet including: capture_time field, seq_num field, ack_num field, SYN field, ACK field and ttl field;

[0023] Based on the fields in the TCP handshake packet, integrating each handshake process to obtain the TCP flow features.

[0024] Further, the NTP flow features include: time when the NTP packet is captured by the NAT router, mode of the NTP packet, time to live value of the NTP packet and destination IP of the NTP packet;

[0025] The Windows device scale and the Linux device scale of each level of internal network are inferred according to the NTP flow features, including:

[0026] Classifying the NTP packets by layer using the time to live value of the NTP packet to obtain the total device scale of each level of internal network;

[0027] Based on the destination IP of the NTP packet, distinguishing the devices in each level of internal network, and combining the total device scale of the level of internal network to obtain the Windows device scale and the Linux device scale of the level of internal network.

[0028] Furthermore, the TCP traffic characteristics include: the time it takes for the NAT router to capture TCP packets, the latency of the TCP packets, and the time to life (TTL) of the TCP packets;

[0029] The method of hierarchical feature clustering of TCP traffic based on TCP traffic characteristics and the scale of Windows and Linux devices at each level of the intranet yields a TCP packet array categorized by host device, including:

[0030] TCP packets are classified by layer based on their time-to-live (TTL) values.

[0031] Using the latency of the TCP packets as the feature value and the total number of Windows and Linux devices in each layer of the intranet as the cluster size, the TCP packets of each layer are clustered respectively, and an array of TCP packets classified by host device is generated based on the clustering results.

[0032] Furthermore, host device relationship inference is performed based on the TCP packet array categorized by host device to construct a deep intranet topology, including:

[0033] Based on the latency array of the host devices, an associated latency array is constructed between each pair of host devices; wherein, the latency array of the host devices is a TCP packet array categorized by host device;

[0034] Calculate the correlation coefficient between host devices based on the associated latency array;

[0035] The similarity of the latency arrays is calculated, and combined with the correlation coefficients between the host devices, a deep intranet topology is constructed.

[0036] Furthermore, based on the correlation between the host devices, a deep intranet topology is constructed by calculating the similarity of the associated latency arrays, including:

[0037] Calculate the similarity of the latency arrays of two host devices;

[0038] If the similarity of the latency arrays is greater than a set threshold and the TCP packet lifetime values ​​are the same, the two host devices with the highest correlation coefficients are selected and merged. The average latency array of the two host devices is used as the latency array of the merged node. After a global update of the associated latency array, the similarity of the latency arrays of the two host devices is recalculated.

[0039] In a case where the similarity of the delay arrays is greater than a set threshold and the time-to-live values of the TCP packets are different, a hierarchy of the two host device nodes corresponding to the TCP packets is constructed based on the time-to-live values of the TCP packets, and the similarity of the delay arrays of the two host devices is recalculated.

[0040] In a case where the similarity of the delay arrays of a host device or a merged node is less than the set threshold, the host device is connected to a NAT router or replaced by a NAT router, and the time-to-live value of the TCP packets of the NAT router is set to the time-to-live value of the TCP packets of the previous level.

[0041] When only one node is left after merging, the node is connected to a NAT router representing the source node.

[0042] A deep-level internal network topology inference system based on traffic, the system comprising:

[0043] a feature extraction module configured to obtain NTP traffic features and TCP traffic features of the traffic;

[0044] a scale analysis module configured to infer a Windows device scale and a Linux device scale of each level of the internal network according to the NTP traffic features;

[0045] a feature clustering module configured to perform hierarchical feature clustering of the TCP traffic based on the TCP traffic features and the Windows device scale and the Linux device scale of each level of the internal network, to obtain a TCP packet array classified by host devices;

[0046] a structure inference module configured to perform host device relationship inference based on the TCP packet array classified by host devices, to construct a deep-level internal network topology.

[0047] An electronic device, comprising a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the deep-level internal network topology inference method based on traffic according to any one of the preceding embodiments.

[0048] A computer readable storage medium, characterized in that the computer readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the deep-level internal network topology inference method based on traffic according to any one of the preceding embodiments.

[0049] Compared with the prior art, the application can infer the device scale in the network by analyzing the traffic data and using NTP traffic packets and infer the relationship between devices by using TCP traffic packets, thereby inferring the topology structure of the deep internal network without prior knowledge. In view of the fact that the topology structure under the NAT device in the internal network is less researched, the topology structure inference method provided by the application is based on the method of inferring the traffic in the public network, and at least has the following technical advantages:

[0050] (1) Device scale inference algorithm: all NTP traffic packets are classified by host based on the destination IP address and period of the NTP request data packet sent by the same device for time synchronization request, and the scale of the host device is inferred.

[0051] (2) Device traffic clustering algorithm: the half of the capture time difference value of the second and third handshake packets in the TCP three-way handshake data packet is taken as the one-way delay between the NAT device and the device in the internal network, and the TCP handshake flow is clustered by host based on the one-way delay.

[0052] (3) Device relationship inference algorithm: first, the half of the round-trip delay of the NAT device to the internal network device is taken as the one-way delay from the probe device to the detected device calculated by the "adjacent" grouping measurement method. Secondly, according to the fact that the network state at similar time is approximately the same, the correlation delay array between two devices is constructed based on the delay of all similar time between two devices. Finally, the correlation coefficient and cosine similarity of the correlation delay array are taken as the correlation index, and the correlation matrix is constructed. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a deep internal network topology structure inference framework based on traffic.

[0054] Figure 2 is a deep internal network topology structure inference method schematic diagram.

[0055] Figure 3 is the period characteristic of the NTP request of the Linux device.

[0056] Figure 4 is the period characteristic of the NTP request of the Windows device.

[0057] Figure 5 is a construction schematic diagram of the correlation delay array between devices.

[0058] Figure 6 is a TCP three-way handshake process.

[0059] Figure 7 is a measurement model of the "adjacent" grouping measurement method.

[0060] Figure 8Ais the original network topology.

[0061] Figure 8B is the inferred network topology. DETAILED DESCRIPTION

[0062] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with specific embodiments and in conjunction with the drawings.

[0063] Figure 1 is the main flow chart of the technical solution of the present application. As shown in Figure 1 , the topology inference method proposed by the present application includes four steps: data acquisition and preprocessing, device scale inference, device flow clustering, and device relationship inference. The specific method is shown in Figure 2 .

[0064] Step one, capture the flow at the NAT router exit, parse the NTP flow and TCP flow therein, extract the key feature values and save them as structured data.

[0065] In step one, first, the flow at the NAT router exit needs to be monitored and captured, and then the flow at the router exit is captured and parsed according to the protocol (NTP, TCP). Next, the corresponding structure is extracted and relevant calculations are performed to obtain its key feature values. These feature values can be used to describe the structured data of the corresponding network device state, so as to facilitate subsequent device scale inference and device flow clustering.

[0066] First, the flow is very large, and second, not all flows can be used for topology inference, so only the flow packets containing key features are parsed. The parsing of the flow packet and the extraction of the feature values will be described in detail below:

[0067] (1) NTP flow packet

[0068] Since almost all devices have the need for time synchronization, they will generate NTP flow packets. After further investigation, the time interval of sending NTP request packets of devices with Linux and Winows operating systems has certain regular characteristics, and the NTP flow packet can be parsed for device scale inference.

[0069] The application uses the destination IP address and periodic characteristics of the NTP request packet to infer the number of devices contained in the deep internal network, and therefore selects the following characteristic data in the NTP flow packet to form a four-tuple structure (capture_time, mode, ttl, dest_ip). Among them, capture_time is the time when the NAT router captures the packet. Since the actual time when the target device sends a request cannot be obtained, but the time delay of the target device to the NAT router is stable under the condition that the network state is stable, it can be approximately considered that the time interval captured by the NAT router is equal to the time interval when the target device sends a request. Mode is the mode of the NTP request packet, which indicates the client mode when mode = 3. The application only analyzes the time interval when the target device sends a request to the NTP server, so it only needs to analyze the NTP flow packet when mode = 3. TTL is the time to live value, and the default initial TTL value of Linux devices and Windows devices is constant. Linux devices are generally 64, while Windows devices are generally 64 or 128 due to operating systems and other reasons. dest_ip is used to record the IP address of the NTP server, and the NTP server requested by the Windows device is constant. Since the capture_time, mode, ttl and dest_ip do not change after NAT conversion, they can be used as device characteristics and further used to infer the device scale.

[0070] (2) TCP flow packet

[0071] Because almost all devices in the network have the need to establish a connection request with other devices in the network or servers that can provide Internet services, TCP three-way handshake packets will be generated in this process to prepare for data transmission. Through analysis, the time difference between the second and third TCP handshake packets can be used as the round-trip time delay from the host device to the NAT router, which can be used as an important indicator of device relevance in the subsequent device relationship inference stage.

[0072] The application infers the relationship between nodes based on TCP flow packets. When analyzing the TCP flow packet, the following fields in the TCP handshake packet (capture_time, seq_num, ack_num, SYN, ACK and ttl) are extracted. After integrating each handshake process, a three-tuple structure (capture_time, delay_time, ttl) composed of three characteristic data is constructed.

[0073] Step two, according to the TTL, the NTP flow is layered, and then based on the NTP server and time period characteristics when the device requests time synchronization, the NTP flow packet of each layer is classified by host device, so as to infer the host device scale.

[0074] In step two, since the key component of the deep internal network is the host device, the core of the inference is the host device, the scale of the inference is the scale of the host device, the relationship of the inference is the relationship between the host devices, and in the process of clustering the traffic, the host device is clustered. This can make the number of clusters consistent with the number of hosts, and the attribution of traffic clustering consistent with the host device in the topology.

[0075] For Linux devices, the traffic cycle characteristics of NTP request packets are as shown in Figure 3 The default time synchronization tool of modern Linux distributions (such as Ubuntu, Debian, Arch Linux, OpenSUSE, etc.) is systemd-timesyncd. Using the "timectl" command and according to "timesync-status", it can be seen that the minimum time interval for the tool to send NTP requests from the same device twice is 32 seconds, and the maximum is 2048 seconds. Therefore, if the time interval of two request data meets the above rules, it can be inferred that they come from the same Linux device.

[0076] For Windows devices, the time points of sending different NTP request packets usually have periodic characteristics, and the NTP server requested by Windows devices is also fixed by default. If the computer runs Windows XP HomeEdition or is not joined to the domain, it will be configured by default to obtain time from time.windows.com. As shown in Figure 4 , where time.windows.com represents the server domain name of the Windows device NTP request. The time interval of the Windows device NTP request can be obtained from the current synchronization time and the next synchronization time 2023 / 5 / 1 8:02-2023 / 4 / 30 22:56 ≈ 9 hours (next synchronization time minus current synchronization time). Using this feature, the NTP packet of the Windows device can be quickly judged and classified by host. Therefore, by collecting and analyzing the NTP request data packet of all Windows devices in the internal network, the scale of the Windows device can also be inferred.

[0077] The present application will implement the inference of the scale of the deep internal network device based on the data structure, and the steps are as follows:

[0078] 1. Classify NTP traffic by layer using TTL: ntp traffic packets with the same TTL value are clustered into a class, indicating that the traffic belongs to the same layer of device;

[0079] 2. Continue to classify the traffic in each class by operating system type (Windows and Linux) using the destination IP: NTP traffic packets with destination IP 52.231.114.183 (time.windows) or 20.189.79.72 (time.nist.gov) are grouped into a class, indicating host device traffic of Windows system; others are grouped into a class, indicating host device traffic of Linux system;

[0080] 3. Group and cluster the traffic of each operating system in each layer respectively: first, the scale of Windows device, the number of NTP request packets captured at the time, is inferred as the scale of Windows device; second, the scale of Linux device, the traffic packets with the difference of the capture time of two data packets being the power of 2 are grouped into a class, and the number of clusters is the scale of Linux device.

[0081] Step three, calculate the one-way delay between each group of TCP flows and the NAT router through the TCP three-way handshake packets, and cluster the TCP flows by host according to the one-way delay, the clustering algorithm being k-means and the number of clusters being the number of device scales inferred in the previous stage. After clustering, the time delay array composed of the one-way delays of the host devices at different times is obtained.

[0082] The TCP three-way handshake process is shown in Figure 6 The TCP three-way handshake process usually includes the following three steps, each of which uses SYN, SEQ, ACK, etc. flags and sequence numbers:

[0083] (1) Client sends a SYN packet:

[0084] SYN: synchronization flag, used to initialize the connection.

[0085] SEQ=x: the client randomly generates an initial sequence number x, indicating that this is the client's sending sequence number.

[0086] The client sends a SYN packet to the server, indicating that it wants to establish a connection and informs the server of the client's initial sequence number.

[0087] (2) Server replies with a SYN-ACK packet:

[0088] SYN: the server also sends a SYN flag, indicating that it agrees to establish a connection.

[0089] SEQ=y: the server generates an initial sequence number y, indicating that this is the server's sending sequence number.

[0090] ACK=x+1: The server acknowledges the receipt of the client's SYN packet and adds 1 to the client's sequence number x in the ACK field, indicating that the next packet sequence number expected from the client is x+1.

[0091] The server agrees to establish the connection by sending a SYN-ACK packet and acknowledges to the client that it has received the client's SYN request.

[0092] (3) The client sends an ACK packet:

[0093] ACK: Acknowledgement flag.

[0094] SEQ=x+1: The client's sequence number continues to increase, indicating the next sending sequence number of the client.

[0095] ACK=y+1: The client acknowledges the receipt of the server's SYN packet and adds 1 to the server's sequence number y in the ACK field, indicating that the next packet sequence number expected from the server is y+1.

[0096] The client acknowledges the server's SYN packet by sending an ACK packet, and the three-way handshake is completed, and the TCP connection is established.

[0097] In this process, SYN indicates a synchronization request for initializing a connection, SEQ indicates the current sequence number of the sender, ACK indicates the received sequence number, and ACK=x+1 means that the next packet sequence number expected from the opposite side is x+1.

[0098] In step three, due to the great difference in link latency of different links, the link latency of the same device remains within a certain range without the influence of network state, so the application clusters the TCP handshake traffic packets by keeping the link latency of the same device within a certain range. Through the foregoing analysis, it can be considered that half of the capture time difference of the last two handshake processes in the TCP three-way handshake packet is approximately equal to the one-way latency from the host device to the NAT router in an ideal state.

[0099] Firstly, all TCP traffic lists are layered by TTL, and then the traffic of each layer is clustered by host device. The invention selects the k-means clustering algorithm to cluster the delay_time as the characteristic value, and the number of clusters is the device scale inferred in the previous step. After clustering, the inferred result of the TCP handshake packet attribution can be obtained and saved as the label field. At the same time, the feature structured data is saved as a four-tuple structure (capture_time, delay_time, ttl and label), and the label field is used as the device identifier in the next stage. After this stage is completed, the packet array of each TCP handshake of different devices can be obtained, and the delay array classified by devices is further obtained, which will be used for device relationship inference in the next stage.

[0100] Step four, according to the traffic packets of similar time being approximately the same network state experienced, the correlation delay array between two devices is constructed, and the correlation coefficient and cosine similarity of the correlation array are calculated, which are important indicators of the correlation between two host devices, thereby constructing the correlation matrix. The two devices with the largest correlation are selected for merging each time, and the correlation matrix is updated until only one device is left at the end of the merging.

[0101] In step four, almost all devices need to send data packets related to establishing communication requests, i.e. TCP handshake traffic packets. In the past, the delay covariance was often used as an important indicator of the correlation between two devices for public network device scale inference. The topology of the invention is similar to that of the public network, so this idea is used for node relationship inference, thereby completing the inference of the entire deep internal network topology. In the previous stage, the four-tuple (capture_time, delay_time, ttl and label) of the TCP packet clustered by host can be obtained, and the TCP packets are divided into arrays classified by host according to the lable.

[0102] Next, the devices nodes will be merged according to the correlation between two devices. The specific process is as follows:

[0103] (1) In the initialization stage, the NAT router and the target node are added to the source node set and the target node set respectively, and the empty edge set is initialized.

[0104] (2) In the correlation matrix construction stage, the characteristic array between two devices, i.e. the delay array captured at similar time, needs to be determined. The construction of the correlation delay array borrows the basic idea of the "adjacent" grouping measurement method, such as Figure 5As shown, based on the latency pairs of all capture moments of the two devices within a certain time range, the correlation latency array between the two devices is constructed. Wherein the horizontal axis represents the time axis of capturing the data packet at the NAT router, and the lines at both ends of the horizontal axis respectively represent the one-way latency of device A and device B arriving at the NAT device at the corresponding moment, and the length represents the size of the latency, marked as TX. From Figure 5 As can be seen from the above, the two devices have 6 groups of related one-way latencies, and then the correlation latency array between the two devices can be obtained as: A(T2 T3 T5T6T8 T9) and B(T11 T12 T14 T15 T17 T19).

[0105] Secondly, the correlation coefficient and cosine similarity between all devices are calculated according to the correlation latency array of each pair of devices, so as to construct the correlation matrix. Figure 7 The shared link and non-shared link model of the "adjacent" grouping measurement method is shown, wherein represents the shared link between node i and node j, and respectively represent the link from the probe node S to node i and the link from the probe node S to node j. Since different links are in a state of mutual independence, through data formula reasoning and actual verification of past experiments, the relationship as shown in formula (1) can be obtained:

[0106]

[0107] Therefore, in the "adjacent" grouping measurement method, the more shared links between two nodes, the greater the latency covariance between them, which means that the higher the correlation between them. This is mainly due to the fact that under the condition of medium network load, the background traffic will cause sufficient changes to the queuing delay of the probe packet, and the latency covariance of the shared link is large. In the device relationship inference algorithm proposed in this paper, based on this core idea, the correlation between any two devices in the deep internal network is calculated, and the topology structure between them is further inferred.

[0108] (3) In the most similar device selection stage, first, when the cosine similarity exceeds the correlation threshold, select the group of devices with the maximum correlation coefficient for merging.

[0109] (4) In the correlation matrix updating stage, after the devices are merged, the correlation matrix also needs to be updated.

[0110] (5) Finally, if there is only one node left in the target node set, connect the node with the source node, and then terminate the inference algorithm, otherwise return to the most similar device selection stage, and then repeat the process. Figure 8A is the original network topology structure, Figure 8B is the network topology structure inferred by the present application.

[0111] Through building simulation topologies on EVE-NG and performing multiple experiments, it can be obtained that the network hierarchy can be accurately inferred by the application, the average accuracy of inferring the device scale is 80%, and the network hierarchy can be correctly inferred each time. In addition, the application also performs excellently in terms of link integrity, link accuracy and link recall rate, which are 0.87, 0.84 and 0.73 respectively. Compared with other methods, the application focuses on using NTP and TCP traffic packets to infer, first obtains reliable device scale, and then infers the device relationship, so better performance and higher accuracy can be shown.

[0112] In summary, the application creatively proposes a deep-level internal network topology inference scheme based on traffic, which innovatively "migrates and applies" the topology inference scheme in the public network to a new research object, so that the influence of the NAT service can be ignored; and the complex problem is also divided, the device scale is first inferred, the traffic is clustered based on the obtained node scale, and then the correlation between two nodes is calculated to infer the node relationship, so that the purpose of effectively inferring the topology structure is achieved.

[0113] The inference method proposed by the application is based on the traffic at the outlet of the NAT router, and the NAT will map the IP address and port number of the host in the local area network to the public IP address and port number. For this case, the application selects the attribute unchanged before and after the NAT conversion as the feature to infer the topology structure, and also combines the characteristics of sending NTP request packets during time synchronization to infer the device scale, combines the difference in the three-way handshake to calculate the correlation coefficient to infer the node relationship, so that the influence of the NAT can be avoided as much as possible, thereby improving the accuracy of the inference result.

[0114] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including modifications and equivalents of the present disclosure that are obvious to those skilled in the art. The specification and examples are to be regarded as exemplary only, and the present disclosure is not to be limited by the exact construction as set forth above and in the following claims, and all changes and modifications that come within the spirit of the present disclosure are intended to be embraced therein.

Claims

1. A flow-based deep-level intranet topology inference method, characterized in that, The method comprises: Obtaining NTP traffic features and TCP traffic features of the traffic; Infer the Windows device scale and Linux device scale of each level of intranet according to the NTP traffic features; Based on the TCP traffic features and the Windows device scale and Linux device scale of each level of intranet, perform hierarchical feature clustering on the TCP traffic to obtain a TCP packet array classified by host devices; Based on the TCP packet array classified by host devices, infer the relationship of host devices to construct a deep-level intranet topology structure; The NTP traffic features include: the time when the NTP packet is captured by the NAT router, the mode of the NTP packet, the time to live value of the NTP packet, and the destination IP of the NTP packet; The Windows device scale and Linux device scale of each level of intranet are inferred according to the NTP traffic features, which comprises: Classify the NTP packets by layers using the time to live value of the NTP packets to obtain the total device scale of each level of intranet; Based on the destination IP of the NTP packets, distinguish the devices in each level of intranet, and combine the total device scale of the level of intranet to obtain the Windows device scale and Linux device scale of the level of intranet; The TCP traffic features include: the time when the TCP packet is captured by the NAT router, the delay time of the TCP packet, and the time to live value of the TCP packet, wherein the delay time of the TCP packet is half of the capture time difference between the second and third handshake packets in the TCP three-way handshake packet; Based on the TCP traffic features and the Windows device scale and Linux device scale of each level of intranet, perform hierarchical feature clustering on the TCP traffic to obtain a TCP packet array classified by host devices, which comprises: Classify the TCP packets by layers according to the time to live value of the TCP packets; Take the delay time of the TCP packet as a feature value, and take the total scale of the Windows devices and Linux devices of each level of intranet as the clustering number, respectively cluster the TCP packets of each level, and generate a TCP packet array classified by host devices according to the clustering results; Based on the TCP packet array classified by host devices, infer the relationship of host devices to construct a deep-level intranet topology structure, which comprises: Based on the host device delay array, construct an associated delay array between two host devices; wherein the host device delay array is the TCP packet array classified by host devices; Calculate the correlation coefficient between host devices according to the associated delay array; Calculate the similarity of the delay array, and combine the correlation coefficient between host devices to construct a deep-level intranet topology structure.

2. The method of claim 1, wherein, Obtaining TCP traffic features of the traffic comprises: Parsing the TCP traffic packet to extract the fields in the TCP handshake packet, wherein the fields in the TCP handshake packet include: capture_time field, seq_num field, ack_num field, SYN field, ACK field, and ttl field; The TCP traffic features are obtained based on the fields in the TCP handshake packets.

3. The method of claim 1, wherein, The deep-level internal network topology is constructed by calculating the similarity of the correlation delay arrays based on the correlation between the host devices, including: calculating the similarity of the delay arrays of two host devices; in the case that the similarity of the delay arrays is greater than a set threshold and the time-to-live values of the TCP packets are the same, selecting two host devices with the largest correlation coefficient between them to be merged, taking the mean of the delay arrays of the two host devices as the delay array of the merged node, and after global updating of the correlation delay arrays, re-executing the calculation of the similarity of the delay arrays of two host devices; in the case that the similarity of the correlation delay arrays is greater than a set threshold and the time-to-live values of the TCP packets are different, constructing the hierarchical structure of the corresponding two host device nodes based on the time-to-live values of the TCP packets, and re-executing the calculation of the similarity of the delay arrays of two host devices; in the case that the similarity of the delay array of a host device or a merged node is less than the set threshold, connecting the host device to a NAT router or replacing the merged node with a NAT router, and taking the mean of the delay array of the host device or the delay array involved in the merged node as the delay array of the NAT router, and setting the time-to-live value of the TCP packet of the NAT router to the time-to-live value of the TCP packet of the previous level; when only one node is left after merging, connecting the node to a NAT router representing the source node.

4. A flow-based deep intra-network topology inference system, characterized in that, The system comprises: a feature extraction module for obtaining NTP traffic features and TCP traffic features of the traffic; a scale analysis module for inferring the Windows device scale and the Linux device scale of each level of internal network according to the NTP traffic features; a feature clustering module for performing hierarchical feature clustering of TCP traffic based on the TCP traffic features and the Windows device scale and the Linux device scale of each level of internal network to obtain TCP packet arrays classified by host devices; a structure inference module for performing host device relationship inference based on the TCP packet arrays classified by host devices to construct a deep-level internal network topology; wherein the NTP traffic features comprise the time when the NTP packet is captured by the NAT router, the mode of the NTP packet, the time-to-live value of the NTP packet, and the destination IP of the NTP packet; the inference of the Windows device scale and the Linux device scale of each level of internal network according to the NTP traffic features comprises: classifying NTP packets by layers using the time-to-live value of the NTP packets to obtain the total device scale of each level of internal network; distinguishing devices in each level of internal network based on the destination IP of the NTP packet, and combining the total device scale of the level of internal network to obtain the Windows device scale and the Linux device scale of the level of internal network; The TCP flow features include: time when the NAT router captures the TCP packet, delay time of the TCP packet, and time-to-live value of the TCP packet, the delay time of the TCP packet being half of the difference between the capture times of the second and third handshake packets in the TCP three-way handshake packet; The TCP flow is hierarchically clustered based on the TCP flow features and the Windows device scale and the Linux device scale in each hierarchy, to obtain a TCP packet array classified by host devices, including: The TCP packets are classified by layers according to the time-to-live value of the TCP packets; The TCP packets in each hierarchy are clustered respectively by taking the delay time of the TCP packet as a feature value and the total scale of the Windows device and the Linux device in each hierarchy as a clustering number, and a TCP packet array classified by host devices is generated according to the clustering results; Host device relationship inference is performed based on the TCP packet array classified by host devices, to construct a deep hierarchy intranet topology, including: An associated time delay array between two host devices is constructed based on a time delay array of the host devices, wherein the time delay array of the host devices is the TCP packet array classified by host devices; A correlation coefficient between the host devices is calculated according to the associated time delay array; The similarity of the time delay array is calculated, and the deep hierarchy intranet topology is constructed in combination with the correlation coefficient between the host devices.

5. An electronic device, comprising: The electronic device includes a processor and a memory storing computer program instructions; and the processor implements the deep hierarchy intranet topology inference method based on flow as claimed in any one of claims 1-3 when executing the computer program instructions.

6. A computer readable storage medium characterized by, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the deep hierarchy intranet topology inference method based on flow as claimed in any one of claims 1-3.

Images