Method and system for constructing time-space association diagram of network flow

Abstract

The invention discloses a network flow time-space association diagram construction method and system, and the method comprises the steps: obtaining network flow record data from network equipment, and each record comprises quintuple information and an active period; defining each network flow as a node in the graph; establishing an edge connection between the two nodes when the two network flows satisfy a spatial correlation condition (sharing an IP address) and a temporal correlation condition (overlapping during an active period); calculating the weight of the edge, wherein the weight value is determined based on the overlapping degree of the two streams in the active period; an adjacency matrix representation of the spatio-temporal association graph is generated. According to the method, the space and time correlation of the network flow is considered at the same time, a more accurate flow correlation model is constructed, group behavior characteristics of malicious flow can be effectively captured, and a more accurate graph structure basis is provided for malicious flow detection.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, and in particular relates to network traffic analysis, graph data processing and spatiotemporal data mining technologies. Background Technology

[0002] With the continuous expansion of network scale and the increasing complexity of network security threats, traditional network traffic analysis methods face significant challenges. Existing network traffic analysis technologies mainly suffer from the following problems: 1. Limitations of Single Traffic Analysis: Most existing methods focus primarily on the characteristic analysis of single network flows, neglecting the crucial fact that malicious traffic often exhibits group behavior characteristics. For example, in botnets, multiple infected hosts simultaneously establish connections with the controller; in DDoS attacks, multiple attack sources coordinate their attacks on the same target. 2. Insufficient consideration of spatial correlation: While some graph-based methods consider the correlation between traffic, they often only focus on the connection relationships between hosts, ignoring fine-grained correlations at the traffic level. This coarse-grained analysis cannot effectively distinguish between normal and malicious traffic; 3. The time dimension is ignored: Most existing methods ignore the temporal correlation between traffic, which may lead to unrelated traffic being incorrectly associated. For example, two traffic streams sharing the same IP address but with a large time interval are actually weakly correlated, but traditional methods may still associate them together; 4. Lack of quantitative analysis of correlation strength: Existing methods typically use binary correlation (related or unrelated), lacking a quantitative representation of correlation strength. This simplified approach fails to reflect the true strength of correlations between flows, affecting the accuracy of subsequent analysis. 5. Real-time requirements and challenges: In large-scale network environments, constructing and updating traffic correlation graphs in real time is a significant challenge. Existing methods often have high computational complexity, making it difficult to meet the needs of real-time analysis.

[0003] In recent years, graph neural network technology has shown great potential in the field of cybersecurity, but its performance largely depends on the quality of the graph structure. A high-quality traffic correlation graph should be able to accurately reflect the real correlation between traffic, distinguish between correlations of different strengths, consider the correlation in both spatial and temporal dimensions, and support real-time construction and updates.

[0004] Therefore, there is an urgent need for a spatiotemporal correlation graph construction method that can simultaneously consider the spatial and temporal correlation of network flows and quantify the correlation strength, so as to provide a more accurate graph structure foundation for malicious traffic detection. Summary of the Invention

[0005] The purpose of this invention is to provide a network flow spatiotemporal correlation graph construction scheme, which constructs a more accurate traffic correlation model by simultaneously considering the spatial correlation (shared IP address) and temporal correlation (overlapping active periods) of network flows.

[0006] First, network flow record data is obtained from network devices, with each record including a 5-tuple and an active period. Each network flow is defined as a node in the graph. When two network flows satisfy both spatial correlation (shared IP address) and temporal correlation (overlapping active periods), an edge connection is established between the two nodes. The weight of the edge is calculated, with the weight value determined based on the degree of overlap between the active periods of the two flows. An adjacency matrix representation of the spatiotemporal correlation graph is generated. This invention can construct the spatiotemporal correlation graph of network flows in real time, providing a more accurate graph structure foundation for malicious traffic detection.

[0007] This invention specifically comprises the following components: The first step is to collect network stream data. Obtain network flow record data from network devices (such as routers and switches). This data can be NetFlow records, sFlow records, or other forms of network flow statistics. Each flow record contains at least a five-tuple of information (source IP address, source port, destination IP address, destination port, and protocol type) and activity period information (start time and end time). The second step is node definition. Each network flow is defined as a node in the graph. (Node set) It consists of network flows in traffic data. Each node For a given network flow, it contains all the feature information of that flow. The third step is to detect spatial and temporal correlation. For any two network flows and We then check whether they meet the spatial correlation condition. Spatial correlation is defined as: two flows sharing the same IP address (source IP or destination IP). That is, they must meet one of the following conditions: 1. Destination IP address equal to Source IP address 2. The source IP address equals destination IP address For two flows that satisfy the spatial correlation condition, we further examine whether they satisfy the temporal correlation condition. Temporal correlation is defined as: the active periods of the two flows overlap. Let... The active period is , The active period is The time correlation condition is: or Let's assume... The quintuple information is Active time is ,at the same time The quintuple information is Active time is The time requirement is met. or At this point, we establish a... point to directed edges . Step 4: Calculate edge weights side weight Calculated based on the degree of overlap during the active periods of the two flows: This weighting formula reflects the importance of the source node in the target node representation optimization; the higher the degree of overlap, the greater the impact of traffic on each other. Step 5: Adjacency matrix generation Generate an adjacency matrix based on the constructed graph structure. If there exists from arrive edge but ,otherwise . Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is an overall flowchart of the network flow spatiotemporal correlation graph construction method and system described in the embodiments of the present invention. Figure 2 , 3 Figures 4 and 5 are schematic diagrams based on this construction method and system in DDoS attack scenarios, botnet attack scenarios, SSH brute-force attack scenarios, and port scanning attack scenarios, respectively. Specific implementation

[0010] In a DDoS attack scenario, NetFlow attack flow records (including UDP flooding and DNS reflection) are collected from the border router and categorized into two key flows: one is the flow from the attack source to intermediate nodes (the 5-tuple contains the attack source IP, random source port, etc., along with the active time interval); the other is the flow from intermediate nodes to the target (the 5-tuple contains the intermediate node IP, etc., along with the active time interval). In the generated adjacency matrix, non-attack flow column values ​​are 0.

[0011] In botnet attack scenarios, binetflow format data (including Storm botnet heartbeat packets) is collected from the core switch. This data contains two key communication flows: one is the flow from the infected host to the C&C server (TCP, destination port 6667), and the other is the flow from the C&C server to the infected host (TCP, source port 6667). The bidirectional flow nodes associated with the C&C server form a dense submatrix, which can be used to locate the botnet communication cluster.

[0012] In SSH brute-force attack scenarios, sFlow records are collected from the IoT gateway, containing two types of flows: one is the SSH request flow (TCP, target port 22) from the attack source to the target IoT device, and the other is the authentication failure response flow from the target device to the attack source. The high weight of the diagonal region of the generated adjacency matrix corresponds to the attack flow pairs of "request-response" IP reverse matching, which can reflect continuous brute-force attempts.

[0013] In port scanning attack scenarios, NetFlow records are collected from the firewall, containing two types of flows: one is the scan request flow from the scanning source to the target device (TCP / SYN, target ports 1-1024), and the other is the response flow from the target device to the scanning source (TCP / RST). In the generated adjacency matrix, the "request-response" flow nodes of the scanning source and the target device form a high-weight submatrix, which can distinguish the scanning patterns.

Claims

1. A method and system for constructing a spatiotemporal correlation graph of network flows, characterized in that, The method includes: Step 1, Network Flow Data Acquisition: Obtain network flow record data containing 5-tuple information and active periods from network devices; Step 2, Node Definition: Define each network flow as a node in the graph. The node contains the characteristic information of the network flow. Step 3, Spatial and Temporal Correlation Detection: Determine whether any two network flows satisfy the spatial correlation condition and the temporal correlation condition. If both are satisfied, establish an edge connection between the corresponding nodes. Step 4, Edge Weight Calculation: Calculate the weight values ​​of the edges between nodes based on the degree of overlap during the active periods of the two network flows; Step 5: Adjacency Matrix Generation: Based on the constructed graph structure, generate an adjacency matrix containing edge weight information to represent the spatiotemporal correlation graph of network flows.

2. The method and system for constructing a network flow spatiotemporal correlation graph according to claim 1, characterized in that, The process of step 1 is as follows: The process involves acquiring network flow record data from network devices such as routers and switches. The data format may include NetFlow records and sFlow records. Network flow statistics records or other types of network flow statistics; each network flow record must include at least a 5-tuple and an active period information, where the 5-tuple information... The information includes the source IP address, source port, destination IP address, destination port, and protocol type. During active periods, the information includes the start time of the network flow. The end time and this information provide the foundational data for constructing the spatiotemporal relationship graph of the network flow.

3. The method and system for constructing a network flow spatiotemporal correlation graph according to claim 1, characterized in that, The process of step 2 is as follows: The structure is as follows: Each independent network flow is defined as a node in the graph, forming a set of nodes. The set It consists of all network flows in the acquired network flow data; for each node in the set It is not only a concrete representation of network flow in the graph. The representation also fully contains all the feature information of the corresponding network flow, which covers the five-tuple information obtained in step 1, activity... Information such as jump duration is used to ensure that each node can fully reflect the attributes of the corresponding network flow.

4. The method and system for constructing a network flow spatiotemporal correlation graph according to claim 1, characterized in that, The process of step 3 is as follows: The body is: for any two nodes corresponding to network flows and First, spatial correlation detection is performed; the criteria for determining spatial correlation are... For two network flows to share the same IP address (source IP address or destination IP address), it can be determined if either of the following two conditions is met. For spatial correlation: Firstly destination IP address and The source IP addresses are the same; secondly, source IP address and The destination IP addresses are the same; after confirming that the two network flows meet the spatial correlation condition, further temporal correlation detection is carried out; assuming The active period is , The active period is If satisfied or If two network flows are temporally correlated, then they are determined to be spatially correlated; if two network flows simultaneously satisfy both spatial and temporally correlated conditions, then at the corresponding nodes... and Establish a directed edge between them The direction of the edge is from and .

5. The method and system for constructing a network flow spatiotemporal correlation graph according to claim 1, characterized in that, The process of step 4 is as follows: The body is: for the directed edges established in step 3 Its weight The calculation is based on the corresponding two network flows and Overlap during active periods Degree; calculation formula is ,in yes During the active period, yes The formula determines the edge weight by quantifying the overlap ratio of the active times of two network flows; the higher the overlap, the higher the weight. The larger the value, the stronger the connection between different network flows.

6. The method and system for constructing a network flow spatiotemporal correlation graph according to claim 1, characterized in that, The process of step 5 is as follows: The process involves generating an adjacency matrix based on the node set defined in step 2 and the edges and edge weights constructed in steps 3 and 4. (n is) (Number of nodes); in the adjacency matrix, if there are slave nodes... To the node directed edges Then matrix elements If not In such a directed edge, the matrix elements By using an adjacency matrix, the spatiotemporal relationships of the entire network flow can be clearly and intuitively presented. The structure of the network graph and the strength of the connections between nodes provide structured data support for subsequent network traffic analysis, malicious traffic detection, and other tasks. hold.

Images