A network space mapping host group discovery and behavior analysis method and device based on graph analysis
By constructing a network graph model based on graph analysis, identifying and analyzing the graph evolution events of the network space surveying and mapping host group, the problem of difficult to identify the behavior patterns of the network space surveying and mapping host group in the existing technology is solved, and the network space surveying and mapping organization is realized Efficient identification and behavioral analysis have improved network security.
Patent Information
- Application Number
- CN202411068388.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-06
AI Technical Summary
The prior art is difficult to effectively identify and analyze the behavior patterns of network space surveying and mapping host groups, making it difficult for network administrators to identify potential organizations and behaviors, increasing network security risks.
Using a graph analysis method, by constructing a network graph model, the graph evolution events of the network space surveying and mapping host group are identified, and statistical attribute analysis is carried out, and host group is divided and merged to form a network space surveying and mapping organization and conduct multi-dimensional behavior analysis.
Improve the recognition rate of cyberspace surveying and mapping organizations, and can efficiently identify events at the evolution level of cyberspace traffic host groups and graphs, explore potential behavior patterns, accurately identify traffic of cyberspace surveying and mapping behaviors, and analyze the detection modes and common tools of different surveying and mapping organizations. , target scope and scale changes, etc.
Smart Images

Figure CN118590324B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of network space mapping, and in particular relates to a network space mapping host group discovery and behavior analysis method and device based on graph analysis. Background Art
[0002] With the continuous development of the information age, the popularity of mobile devices and IoT devices has shown a rapid growth trend year after year. In-depth analysis of the behavior of existing surveying and mapping organizations has also become the basis for formulating effective defense measures.
[0003] (1) Cyberspace resource detection
[0004] Cyberspace resource detection technology can be divided into active detection and passive detection according to its detection method. Active detection sends a detection data packet to the target, and then analyzes the content and pattern of the target's response data packet to identify the target. This type of detection packet is often targeted at a specific protocol or application on the target host; while passive detection usually does not actively send any data packets to the target system, but uses various network traffic acquisition tools or packet capture tools to collect data packets flowing through the network and obtain information about the target device from them. At present, network resource detection technology can be divided into traditional network detection technology and stateless detection technology according to whether the connection status is recorded. Compared with traditional network detection technology, stateless detection technology does not need to record the connection status, does not resend detection data packets, occupies less resources, and is faster.
[0005] (2) Cyberspace Mapping
[0006] Cyberspace mapping is an interdisciplinary research based on computer science, network science and mapping science. It aims to comprehensively depict the structure and properties of global cyberspace through the detection, data collection, analysis and processing, and visualization of cyberspace. Research in this field focuses on understanding and describing the node distribution, network topology, distribution of physical and virtual resources, and the geographic information and logical connections between them.
[0007] In practice, the cyberspace mapping platform uses active detection technology to perform port scanning and protocol identification on network targets around the world, integrates and displays the acquired data, and provides users with tools to search and retrieve specific types of network assets. However, this active detection behavior also brings certain security risks, which may expose the target to the Internet, increasing its security risks. Due to the use of active detection technology, cyberspace mapping nodes usually actively send data packets to the target, leaving traces in the network connection log of the target host. However, the scans from the cyberspace mapping platform usually have a short connection time and only occupy a very small amount of resources on the target host. Compared with the traffic of large-scale denial of service and other network attack behaviors, it is difficult to attract the attention of network administrators. Therefore, identifying the host group from the cyberspace mapping organization in the network data traffic and analyzing its behavior pattern is of great significance for defending against the scanning of the mapping organization and avoiding the widespread exposure of assets to the Internet.
[0008] (3) Graph analysis
[0009] Graph analysis is a method of analyzing graph data structures, which involves studying the nodes, edges, and relationships between them in the graph to reveal patterns, trends, and structures between data. Graph analysis has a wide range of applications in various fields, including social network analysis, bioinformatics, transportation network optimization, financial risk management, etc. In graph analysis, nodes represent entities of data, while edges represent the relationships or connections between nodes. By analyzing the properties, structure, and connection methods of nodes and edges, we can discover the correlation, clustering, and propagation paths between data, thereby gaining insight into the inherent laws and characteristics of the data.
[0010] The application of graph analysis is not limited to static graph data, but can also be applied to the analysis of dynamic graph data, such as network topology that changes over time, information dissemination process in social networks, etc. In the context of cyberspace mapping, graph analysis can be used to analyze network topology, identify the relationship between network nodes, discover community structure in the network, etc., providing important support for the visualization, understanding and management of cyberspace.
[0011] The process of network mapping organization discovery and behavior analysis involves two important areas, namely community discovery and behavior analysis. Community discovery algorithms provide important tools and technical support for mapping organization discovery, and behavior analysis technology analyzes and studies the traffic generated during network mapping.
[0012] Community discovery refers to the process of identifying a collection of nodes with close connections and intrinsic relationships in a complex network. This method is widely used to identify groups and organizations in network structures and discover potential social circles, interest groups or functional modules.
[0013] Behavioral analysis refers to the process of analyzing and identifying the behavioral patterns and characteristics of nodes or entities in the network. Behavioral analysis technology is generally used to discover abnormal behaviors, malicious activities or potential threats in the network, involving a variety of technical means such as data mining, machine learning, and statistical analysis, including rule-based, anomaly detection-based, and machine learning model-based methods.
[0014] In summary, the current network background community discovery and behavior analysis technologies usually only target social traffic, Web traffic, network attack traffic, etc., and have not yet paid attention to cyberspace mapping traffic. In cyberspace mapping, community discovery can help identify important node sets in the network and discover potential attack targets or network threats. Behavior analysis can identify the characteristics of mapping organizations in different aspects of the network and understand the behavior patterns of the other party. Therefore, by following the graph structure of host group level community discovery and using graph analysis technology to reflect the relationship between the communication behaviors of hosts, we can mine the communication information in network traffic and discover influential host nodes and closely connected mapping host groups. Network security personnel have also made preliminary attempts to identify cyberspace mapping organizations, but they only stay at a relatively single level such as domain names, and the behavior of cyberspace mapping organizations has not been further analyzed. Overall, there is still a lack of effective means for the identification of cyberspace mapping host groups and the analysis of potential behavior patterns. Summary of the invention
[0015] The present invention proposes a network space mapping host group discovery and behavior analysis solution based on graph analysis. The solution aims to solve the problem that network administrators find it difficult to identify potential organizations and behaviors from a large amount of data from network space mapping. By fully utilizing the synergy of mapping traffic through graph analysis methods, network behaviors are linked to potential social relationships, thereby improving the recognition rate of network space mapping organizations and conducting multi-dimensional behavior analysis.
[0016] The first aspect of the present invention provides a network space mapping host group discovery and behavior analysis method based on graph analysis, the method comprising:
[0017] Step S1, constructing a network graph model based on the collected mapping traffic, and dividing the network space mapping host group by performing network analysis on the network graph model;
[0018] Step S2, extracting the graph evolution events of the network space mapping host group within a preset time window, and performing mapping behavior recognition on the graph evolution events according to the network space mapping behavior characteristics to adjust the division of the network space mapping host group;
[0019] Step S3: disassemble or merge the adjusted cyberspace mapping host group through statistical attribute analysis to form a cyberspace mapping organization, and perform cyberspace mapping behavior analysis on the cyberspace mapping organization.
[0020] According to the method of the first aspect of the present invention, in step S1:
[0021] Arrange sensing nodes, collect the surveying and mapping traffic through the sensing nodes, and convert the surveying and mapping traffic into structured data to form a surveying and mapping traffic PCAP data packet;
[0022] Filter out data packets with the same five-tuple information from the surveying and mapping traffic PCAP data packets, and sort and merge the filtered data packets in chronological order to form a network flow;
[0023] The five-tuple information includes: source IP address, destination IP address, source port number, destination port number, and protocol;
[0024] Extract the source IP and destination IP from each network flow as physical host nodes; use the port number as the node attribute; connect each node through a directed edge, wherein the directed edge represents the communication behavior from the source IP to the destination IP; thereby constructing the network graph model;
[0025] Wherein, the network graph model is stored in the Neo4j graph database.
[0026] According to the method of the first aspect of the present invention, in step S1:
[0027] Determine the network structure and node importance by calculating the degree, betweenness centrality, and closeness centrality of each node;
[0028] Among them, degree represents the number of node connections, betweenness centrality represents the degree of intermediary of a node in the network, and closeness centrality represents the degree of close connection between a node and other nodes;
[0029] The community discovery algorithm is used to divide the nodes in the network into community structures with maximum internal connections and minimum external links, which serve as host groups for network space mapping.
[0030] According to the method of the first aspect of the present invention, in step S2, based on the relationship mapping of the network graph model in adjacent time windows, the graph evolution events of the network space mapping host group are extracted; wherein:
[0031] The graph evolution events include the increase, stability, reduction, merger, split, addition, and disappearance of the network space mapping host group;
[0032] When the graph evolution event is increase, the number of host group members increases, the number of group identifiers remains unchanged, and the event change type is scale change;
[0033] When the graph evolution event is stable, the number of host group members remains unchanged, the number of group identifiers remains unchanged, and the event change type is scale change;
[0034] When the graph evolution event is shrinking, the number of host group members decreases, the number of group identifiers remains unchanged, and the event change type is scale change;
[0035] When the graph evolution event is a merge, the number of host group members increases / remains unchanged / decreases, the number of group identifiers decreases, and the event change type is a structural change;
[0036] When the graph evolution event is split, the number of host group members increases / remains unchanged / decreases, the number of group identifiers increases, and the event change type is structural change;
[0037] When the graph evolution event is a new addition, the number of host group members increases, the number of group identifiers increases, and the event change type is a structural change;
[0038] When the graph evolution event is disappearance, the number of host group members decreases, the number of group identifiers decreases, and the event change type is structural change.
[0039] According to the method of the first aspect of the present invention, in step S2, the cyberspace mapping behavior characteristics include short-term characteristics, high-frequency characteristics, specific port scanning, remote sources, and irregularity; wherein:
[0040] The transient characteristic means that the surveying and mapping traffic has the transient characteristic, and the communication connection time between different nodes is lower than the first judgment value;
[0041] High-frequency characteristics refer to: the mapping traffic has high-frequency characteristics, the number of targets scanned in a fixed time interval is higher than the second discrimination value, and the node degree distribution is skewed;
[0042] Specific port scanning means: the mapping traffic is scanned for a specific port or a specific protocol, and the port attribute of the node is a specific port;
[0043] Off-site sources refer to: surveying and mapping organizations use distributed surveying and mapping nodes to perform scanning tasks, surveying and mapping traffic comes from different IP addresses, and there is a connection relationship between nodes that spans geographical locations;
[0044] Irregularity means that the surveying and mapping traffic does not have periodicity, and the network space surveying and mapping host group has structural changes.
[0045] According to the method of the first aspect of the present invention, in the statistical attribute analysis of step S3, the statistical attributes analyzed specifically include:
[0046] Similarity between host groups within the current time window, similarity of host IP addresses between host groups, similarity of IP addresses of member hosts within a host group;
[0047] Comparison of the number of member hosts within the host group in the current time window, and comparison of the number of member hosts within the host group in the current time window and in the historical time window;
[0048] The frequency of occurrence of member hosts within the host group in the historical time window within the current time window.
[0049] According to the method of the first aspect of the present invention, in step S3, the cyberspace mapping behavior analysis includes:
[0050] Calculate the statistical characteristics of the number of organizational members of the cyberspace mapping organization, including quartiles, mean, variance, and maximum value; wherein: the quartiles represent the central position of the number of organizational members and the degree of dispersion of the distribution; the mean represents the central trend of the organizational data; the variance represents the degree of change and uncertainty of the number of organizations, and when the variance is higher than the third discriminant value, the cyberspace mapping organization acts alone, otherwise the cyberspace mapping organization acts in a unified manner; the maximum value represents the abnormal behavior of the cyberspace mapping organization;
[0051] Utilizing the attributes of nodes and edges in the graph model, statistical display of the scanning ports, protocols and tools of the cyberspace mapping organization is performed, including visual display in different scanning time windows, different scanning frequencies and different scanning cycles; the mapping behavior of the cyberspace mapping organization is visualized based on geographic location distribution.
[0052] A second aspect of the present invention provides a network space mapping host group discovery and behavior analysis device based on graph analysis, the device comprising a processing unit, and the processing unit is configured to perform the following steps:
[0053] Step S1, constructing a network graph model based on the collected mapping traffic, and dividing the network space mapping host group by performing network analysis on the network graph model;
[0054] Step S2, extracting the graph evolution events of the network space mapping host group within a preset time window, and performing mapping behavior recognition on the graph evolution events according to the network space mapping behavior characteristics to adjust the division of the network space mapping host group;
[0055] Step S3: disassemble or merge the adjusted cyberspace mapping host group through statistical attribute analysis to form a cyberspace mapping organization, and perform cyberspace mapping behavior analysis on the cyberspace mapping organization.
[0056] The third aspect of the present invention provides an electronic device. The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the network space mapping host group discovery and behavior analysis method based on graph analysis in the first aspect of the present invention are implemented.
[0057] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a network space mapping host group discovery and behavior analysis method based on graph analysis according to the first aspect of the present invention.
[0058] In summary, the present invention first configures and deploys different traffic sensing nodes in multiple regions around the world, collects various types of scanning traffic, discovers host groups and selects group identifiers by establishing a network flow graph model, identifies host group graph evolution events to detect cyberspace mapping patterns, and finally redivides different mapping organizations and conducts behavioral analysis. The following effects are specifically achieved: (1) The method can efficiently identify cyberspace traffic host groups and events at the graph evolution level, analyze the interactive relationships of hosts from a graph theory perspective, and mine potential behavioral patterns; (2) The method can be used to obtain the scanning behavior graph structure of the mapping organization, and accurately identify traffic with cyberspace mapping behavior from the collected network traffic; (3) The method can be used to analyze the detection patterns, common tools, target ranges, scale changes and other mapping behavior information of different mapping organizations. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0060] Figure 1 A flowchart of network space mapping host group discovery and behavior analysis based on graph analysis according to an embodiment of the present invention;
[0061] Figure 2 A schematic diagram of constructing a network graph model according to an embodiment of the present invention;
[0062] Figure 3 The figure is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0064] The goals of the cyberspace mapping host cluster discovery and behavior analysis method include: (1) identifying host clusters in the connection graph composed of network communication behaviors; (2) identifying the dynamic evolution events of network communication behavior host clusters; (3) extracting the characteristics of cyberspace mapping traffic that are different from network background traffic and filtering the corresponding traffic; (4) analyzing the attribution of different mapping organizations and mapping behavior information such as detection modes, common tools, target ranges, and scale changes.
[0065] In order to achieve the above objectives, the method of the present invention mainly includes the following key technical points:
[0066] (1) Cyberspace mapping host cluster discovery
[0067] Appropriate sensing nodes are deployed globally to collect the required traffic data. Prior knowledge, such as one's own IP and third-party information, is used to preliminarily filter out some basic non-network mapping traffic, such as traffic generated by management nodes. The collected PCAP data packets are formed into network flows. A graph structure model is established based on the constructed network flows. The community discovery algorithm is used on the constructed graph structure to realize the division of host groups. Group identifiers are selected in each host group based on the properties of the complex network model.
[0068] (2) Cyberspace mapping and host group graph evolution event identification
[0069] Define a time window that can adapt to network mapping traffic, calculate and count the host group information in each time window interval, change the information according to the host group identification in adjacent time windows, identify the dynamic evolution events of the host group in terms of scale and structure, and build a host group portrait.
[0070] (3) Cyberspace mapping and traffic behavior pattern recognition
[0071] According to the characteristics of cyberspace mapping behavior in the graph structure, we extract the behavioral patterns of cyberspace mapping traffic that are different from other background traffic, such as transience, high frequency, specific ports, remote sources, irregularity, etc. Based on the patterns and combined with a small amount of deep packet inspection and analysis, we filter out more accurate cyberspace mapping traffic again, and finally extract the corresponding organization and other basic information from the traffic.
[0072] (4) Behavioral analysis of cyberspace mapping organizations
[0073] From the previous step, filter out a certain number of cyberspace mapping host groups, merge or disassemble some host groups according to the similarity of the host groups to form cyberspace mapping organizations, and conduct multi-dimensional analysis on each cyberspace mapping organization, including frequency, scanning protocol and port, scale, structure, geography, dynamic events, constructed request messages, etc., so as to analyze the scanning time and cycle, ports and protocols, scanning node regional deployment strategies and possible scanning tools of different mapping organizations.
[0074] The present invention proposes a network space mapping host group discovery and behavior analysis solution based on graph analysis. Through efficient and accurate discovery of network space mapping host groups and detailed dynamic behavior analysis, it provides reference and judgment basis for network administrators to protect their important target network assets from scanning from network space mapping platforms.
[0075] The first aspect of the present invention provides a network space mapping host group discovery and behavior analysis method based on graph analysis, the method comprising:
[0076] Step S1, constructing a network graph model based on the collected mapping traffic, and dividing the network space mapping host group by performing network analysis on the network graph model;
[0077] Step S2, extracting the graph evolution events of the network space mapping host group within a preset time window, and performing mapping behavior recognition on the graph evolution events according to the network space mapping behavior characteristics to adjust the division of the network space mapping host group;
[0078] Step S3: disassemble or merge the adjusted cyberspace mapping host group through statistical attribute analysis to form a cyberspace mapping organization, and perform cyberspace mapping behavior analysis on the cyberspace mapping organization.
[0079] In some embodiments, the method first deploys perception nodes in multiple regions around the world and reasonably configures open port information, monitors the inflow traffic of the perception nodes, obtains relatively pure cyberspace mapping traffic, extracts cyberspace mapping host nodes and edges, and constructs a graph model for complex network analysis and host group division. Then, based on a certain time window and host group identifier mapping relationship change, the host group behavior is identified through graph evolution events. After obtaining the host group behavior graph evolution events, they are compared with the cyberspace mapping behavior characteristics to obtain a more accurate cyberspace mapping host group. Finally, according to the similarity of the host group members and related statistical characteristics, the mapping behavior characteristics of different mapping organizations are analyzed and visualized. The process is as follows: Figure 1 shown.
[0080] According to the method of the first aspect of the present invention, in step S1:
[0081] Arrange sensing nodes, collect the surveying and mapping traffic through the sensing nodes, and convert the surveying and mapping traffic into structured data to form a surveying and mapping traffic PCAP data packet;
[0082] Filter out data packets with the same five-tuple information from the surveying and mapping traffic PCAP data packets, and sort and merge the filtered data packets in chronological order to form a network flow;
[0083] The five-tuple information includes: source IP address, destination IP address, source port number, destination port number, and protocol;
[0084] Extract the source IP and destination IP from each network flow as physical host nodes; use the port number as the node attribute; connect each node through a directed edge, wherein the directed edge represents the communication behavior from the source IP to the destination IP; thereby constructing the network graph model;
[0085] Wherein, the network graph model is stored in the Neo4j graph database.
[0086] In some embodiments, cyberspace mapping host group discovery aims to achieve group division of hosts so as to manage and analyze traffic initiators at the group level, mainly including mapping traffic collection, graph model establishment, and complex network analysis.
[0087] In some embodiments, according to relevant intelligence, sensing nodes are deployed in various places, mainly in places with more surveying and mapping organizations. Some commonly used protocol services are counted and deployed in various nodes to ensure that the configurations between nodes are different and have a certain degree of diversity. They are disguised as ordinary host nodes and actively trapping is used to collect surveying and mapping traffic.
[0088] The traffic data is preprocessed to filter out normal access traffic and traffic used for node management. The information defined as the following fields is extracted from the traffic data and converted into structured data, as shown in Table 1.
[0089] Table 1: Field Description
[0090]
[0091] Action indicates the action of the network, such as data, connect, close, etc. Src_IP and Src_port refer to the recorded IP address of the host initiating the mapping behavior and the port information used for the connection. Dst_IP and Dst_port refer to the IP address of the sensing node and the open port information. Services indicates the transport layer protocol or application layer protocol used by the host initiating the mapping behavior to connect. Time indicates the timestamp information of the connection made by the host initiating the mapping behavior. Payload records the payload of the message. The Payload content can be used to preliminarily filter network attack traffic, such as password cracking.
[0092] From the above preprocessed mapping traffic PCAP packets, filter out packets with the same five-tuple information (source IP address, destination IP address, source port number, destination port number, and protocol). Then, sort the filtered packets in chronological order to ensure that the constructed network flow is arranged in the order in which the communication occurs. Merge the packets arranged in chronological order into traffic flows to form network flows. This step usually divides the flow according to a certain time window, such as seconds, minutes, or other time intervals. In each time window, the packets with the same five-tuple information are merged into the same network flow.
[0093] After constructing the network flow, the source IP and target IP are extracted from each network flow as nodes. The nodes represent the host entities, and the node attributes are the port numbers. They are connected into directed edges. The edges point from the source IP nodes to the target IP nodes. The actual meaning of the edges is the communication behavior between hosts. The edge attributes are the protocol, network flow characteristics and payload collection, so as to establish a network graph model and store it in the Neo4j graph database. The establishment process is as follows: Figure 2 As shown in the figure. In theory, the sensing node will not actively send data packets to the mapping node, but there are also cases where the sensing node replies to the mapping node. Therefore, the edge from the sensing node to the mapping node can also be added to the overall model, which can represent the degree of connection of the mapping method used by the mapping node. For example, in the semi-connected detection, the sensing node may retransmit the SYN-ACK data packet, resulting in more transmission times than in the fully connected detection.
[0094] According to the method of the first aspect of the present invention, in step S1:
[0095] Determine the network structure and node importance by calculating the degree, betweenness centrality, and closeness centrality of each node;
[0096] Among them, degree represents the number of node connections, betweenness centrality represents the degree of intermediary of a node in the network, and closeness centrality represents the degree of close connection between a node and other nodes;
[0097] The community discovery algorithm is used to divide the nodes in the network into community structures with maximum internal connections and minimum external links, which serve as host groups for network space mapping.
[0098] In some embodiments, when performing complex network analysis on the constructed network traffic graph model, the basic complex network model attributes must first be studied, which may include the degree, betweenness centrality, closeness centrality, etc. of the node. The degree indicates the number of connections of the node, the betweenness centrality indicates the degree of mediation of the node in the network, and the closeness centrality indicates the degree of close connection between the node and other nodes. By analyzing these attributes, the overall structure of the network and the importance and influence of each node in the network can be understood.
[0099] In some embodiments, network space mapping traffic is often group-based, and a community discovery algorithm is used to cluster hosts, grouping hosts with similar connection patterns or functions in the network together, so as to better understand the structure and function of the network and identify host groups. The specific method is a modularity-based community discovery algorithm, which aims to divide the network into a community structure with maximum internal connections and minimum external connections, and to achieve community discovery by optimizing the density of internal connections in the community and the sparseness of connections between communities in the network. The algorithm flow is:
[0100] (1) Initialize the network and treat each host as a host group.
[0101] (2) Calculate the modularity of the initial network.
[0102] (3) Iteratively move nodes from one host cluster to another until the modularity reaches its maximum value or optimization cannot be continued.
[0103] (4) Output the optimal host group division result.
[0104] In some embodiments, when selecting appropriate attributes as the identification of the host group for mapping traffic, the characteristics and objectives of the mapping traffic need to be considered. The node IP with the largest degree of the host group is selected as the identification number of the host group, or the node IP with the largest betweenness centrality is selected as the identification number of the host group. These nodes have greater influence or more information transmission capabilities in the network, can better represent a host group and have a certain stability.
[0105] According to the method of the first aspect of the present invention, in step S2, based on the relationship mapping of the network graph model in adjacent time windows, the graph evolution events of the network space mapping host group are extracted; wherein:
[0106] The graph evolution events include the increase, stability, reduction, merger, split, addition, and disappearance of the network space mapping host group;
[0107] When the graph evolution event is increase, the number of host group members increases, the number of group identifiers remains unchanged, and the event change type is scale change;
[0108] When the graph evolution event is stable, the number of host group members remains unchanged, the number of group identifiers remains unchanged, and the event change type is scale change;
[0109] When the graph evolution event is shrinking, the number of host group members decreases, the number of group identifiers remains unchanged, and the event change type is scale change;
[0110] When the graph evolution event is a merge, the number of host group members increases / remains unchanged / decreases, the number of group identifiers decreases, and the event change type is a structural change;
[0111] When the graph evolution event is split, the number of host group members increases / remains unchanged / decreases, the number of group identifiers increases, and the event change type is structural change;
[0112] When the graph evolution event is a new addition, the number of host group members increases, the number of group identifiers increases, and the event change type is a structural change;
[0113] When the graph evolution event is disappearance, the number of host group members decreases, the number of group identifiers decreases, and the event change type is structural change.
[0114] In some embodiments, cyberspace mapping host group evolution event identification aims to describe the behavior of the host group and extract the cyberspace mapping host group according to the changes of the graph model on the timeline, mainly including identification relationship mapping and mapping behavior identification.
[0115] In some embodiments, a suitable time window is defined to represent the dynamic evolution of the network space mapping host group. This time window is different from the time window when building the network flow. It is necessary to describe the frequency of changing the strategy adopted by the mapping organization in detail, which may be a span of hours, days, weeks, or months. Each host group has a unique group ID, and there is a relationship mapping of adjacent time window graph structures. The specific relationship mapping includes whether the host group ID has changed and the change in the number of host group members. The event example is shown in Table 2.
[0116] Table 2: Event description
[0117]
[0118] According to the method of the first aspect of the present invention, in step S2, the cyberspace mapping behavior characteristics include short-term characteristics, high-frequency characteristics, specific port scanning, remote sources, and irregularity; wherein:
[0119] The transient characteristic means that the surveying and mapping traffic has the transient characteristic, and the communication connection time between different nodes is lower than the first discrimination value (the first discrimination value is the discrimination value of the communication connection time between different nodes);
[0120] High-frequency characteristics refer to: the surveyed traffic has high-frequency characteristics, the number of targets scanned in a fixed time interval is higher than the second discrimination value (the second discrimination value is the discrimination value of the number of targets scanned in a fixed time interval), and the node degree distribution is skewed;
[0121] Specific port scanning means: the mapping traffic is scanned for a specific port or a specific protocol, and the port attribute of the node is a specific port;
[0122] Off-site sources refer to: surveying and mapping organizations use distributed surveying and mapping nodes to perform scanning tasks, surveying and mapping traffic comes from different IP addresses, and there is a connection relationship between nodes that spans geographical locations;
[0123] Irregularity means that the surveying and mapping traffic does not have periodicity, and the network space surveying and mapping host group has structural changes.
[0124] In some embodiments, the method of constructing a mapping host group profile based on the characteristics of cyberspace mapping behavior is to extract the differences between the behavior patterns of cyberspace mapping traffic and other background traffic. These behavior patterns include transience, high frequency, specific port scanning, remote sources, and irregularity.
[0125] (1) Transience: Mapping traffic is often transient, that is, mapping organizations usually quickly scan the target and collect necessary information before immediately disconnecting. This transience makes mapping traffic difficult to detect in network traffic. Transience is manifested in the short duration of node connections, that is, the edges between nodes exist in the graph for a short time, causing the structure of the host group to change continuously, corresponding to evolutionary events such as mergers, splits, additions, and disappearances.
[0126] (2) High frequency: Mapping traffic is performed at a high frequency, scanning a large number of targets in a short period of time to obtain as much information as possible. This causes the node degree distribution in the graph to show a significant skewed distribution, that is, some nodes have very high degrees. In terms of evolutionary events, high-frequency traffic leads to a large number of short-term connections between nodes in the graph, forming host groups with similar sizes in the same time window or adjacent time windows.
[0127] (3) Specific port scanning: Mapping traffic usually scans specific ports or protocols. For example, mapping organizations may focus on scanning commonly used service ports to understand the services and applications running on the target system. In the graph model, in a host group with a large number of host group members, the port attribute of the host node is a specific port.
[0128] (4) Off-site sources: Surveying and mapping organizations usually use distributed surveying and mapping nodes to perform scanning tasks, resulting in surveying and mapping traffic coming from different IP addresses around the world. The characteristics of off-site sources may result in connections between nodes in the graph that span geographical locations, which is manifested in the wide geographical distribution of nodes in the graph, specifically the presence of a particularly large host cluster.
[0129] (5) Irregularity: The occurrence of mapping traffic may not have obvious periodicity or regularity, but may change according to the activities and goals of the mapping organization. This irregularity makes it difficult to detect and identify it through traditional traffic analysis methods. Irregularity may cause the host group structure to change uncertainly over time, with frequent structural changes and reorganizations, such as splits, merges, additions, and disappearances.
[0130] In some embodiments, by comprehensively considering the performance of these features in the static properties and evolutionary events of the graph, combined with a small amount of deep packet inspection analysis, more accurate cyberspace mapping traffic and other basic information can be filtered out again.
[0131] According to the method of the first aspect of the present invention, in the statistical attribute analysis of step S3, the statistical attributes analyzed specifically include:
[0132] Similarity between host groups within the current time window, similarity of host IP addresses between host groups, similarity of IP addresses of member hosts within a host group;
[0133] Comparison of the number of member hosts within the host group in the current time window, and comparison of the number of member hosts within the host group in the current time window and in the historical time window;
[0134] The frequency of occurrence of member hosts within the host group in the historical time window within the current time window.
[0135] In some embodiments, the cyberspace mapping organizational behavior analysis aims to span the research unit from host groups to organizations, achieving a leap from abstract entities to real entities, mainly including the similarity of host group members, statistical characteristics of organizational numbers, and visual display of organizational behavior.
[0136] In some embodiments, the similarity of host group members, the number of groups, and the historical statistical attributes of the number of group members are calculated, specifically whether there are host groups with high similarity in the current time window, such as the host IP address similarity between host groups, and the IP address similarity of member hosts within the host group; whether there are host groups with the same number of group members in the current time window, and whether there are host groups with the same number of group members in the current time window and the historical time window; whether the host group members in the current time window have appeared in the historical time window. According to the similarity of the host groups, some host groups are merged or disassembled to form a cyberspace mapping organization.
[0137] According to the method of the first aspect of the present invention, in step S3, the cyberspace mapping behavior analysis includes:
[0138] Calculate the statistical characteristics of the number of organizational members of the cyberspace mapping organization, including quartiles, mean, variance, and maximum value; wherein: the quartiles represent the central position of the number of organizational members and the degree of dispersion of the distribution; the mean represents the central trend of the organizational data; the variance represents the degree of change and uncertainty of the number of organizations; when the variance is higher than the third discriminant value (the third discriminant value is the variance discriminant value), the cyberspace mapping organization acts alone, otherwise the cyberspace mapping organization acts in a unified manner; the maximum value represents the abnormal behavior of the cyberspace mapping organization;
[0139] Utilizing the attributes of nodes and edges in the graph model, statistical display of the scanning ports, protocols and tools of the cyberspace mapping organization is performed, including visual display in different scanning time windows, different scanning frequencies and different scanning cycles; the mapping behavior of the cyberspace mapping organization is visualized based on geographic location distribution.
[0140] In some embodiments, for the newly divided organizational level, the statistical characteristics of the number of organizational members are calculated according to the definitions of statistical characteristics such as quartiles, mean, variance and extreme value. The quartiles are used to understand the central position of the number of organizational members and the degree of dispersion of the distribution. The mean provides the average level of the mapping organization, which helps to understand the central trend of the organizational data; the variance is an indicator of the volatility and degree of change of the data set. For cyberspace mapping organizations, the variance is used to understand the degree of change and uncertainty of the data in the number of organizations. If the variance is large, it means that there is a mapping organization acting alone, otherwise it means that there is a unified mapping organization; the extreme value provides the maximum observed value of the data in the host group. In cyberspace mapping, understanding the extreme values in the host group can identify abnormal behavior of the mapping organization, and then conduct security analysis and defense. At the same time, the total number of organizations can also be reflected in when and where the mapping organization conducts more mapping behaviors.
[0141] In some embodiments, representative surveying and mapping organization traffic can be screened out for cyberspace surveying and mapping behavior analysis, such as more active surveying and mapping organizations, that is, organizations with a large number of members and a more stable structure. Using the attributes of nodes and edges in the group, the scan ports, protocols, and tools of the cyberspace surveying and mapping organizations are statistically displayed, and the ports, protocols, and tools they frequently scan are analyzed; the different scanning time windows of the surveying and mapping organizations for different protocols and tools are clustered and statistically displayed, and the scanning frequency and period of different protocols are analyzed; the global surveying and mapping behavior of the organization is geographically displayed, and its geographical distribution strategy is analyzed; the overall activities of the surveying and mapping organization are displayed, and its uniform regularity in terms of time and other aspects is analyzed.
[0142] A second aspect of the present invention provides a network space mapping host group discovery and behavior analysis device based on graph analysis, the device comprising a processing unit, and the processing unit is configured to perform the following steps:
[0143] Step S1, constructing a network graph model based on the collected mapping traffic, and dividing the network space mapping host group by performing network analysis on the network graph model;
[0144] Step S2, extracting the graph evolution events of the network space mapping host group within a preset time window, and performing mapping behavior recognition on the graph evolution events according to the network space mapping behavior characteristics to adjust the division of the network space mapping host group;
[0145] Step S3: disassemble or merge the adjusted cyberspace mapping host group through statistical attribute analysis to form a cyberspace mapping organization, and perform cyberspace mapping behavior analysis on the cyberspace mapping organization.
[0146] The third aspect of the present invention provides an electronic device. The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the network space mapping host group discovery and behavior analysis method based on graph analysis in the first aspect of the present invention are implemented.
[0147] Figure 3 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 3 As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.
[0148] Those skilled in the art will understand that Figure 3The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0149] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a network space mapping host group discovery and behavior analysis method based on graph analysis according to the first aspect of the present invention.
[0150] In summary, the present invention first configures and deploys different traffic sensing nodes in multiple regions around the world, collects various types of scanning traffic, discovers host groups and selects group identifiers by establishing a network flow graph model, identifies host group graph evolution events to detect cyberspace mapping patterns, and finally redivides different mapping organizations and conducts behavioral analysis. The following effects are specifically achieved: (1) The method can efficiently identify cyberspace traffic host groups and events at the graph evolution level, analyze the interactive relationships of hosts from a graph theory perspective, and mine potential behavioral patterns; (2) The method can be used to obtain the scanning behavior graph structure of the mapping organization, and accurately identify traffic with cyberspace mapping behavior from the collected network traffic; (3) The method can be used to analyze the detection patterns, common tools, target ranges, scale changes and other mapping behavior information of different mapping organizations.
[0151] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.
Claims
1. A network space mapping host group discovery and behavior analysis method based on graph analysis, characterized in that: The method comprises: Step S1, constructing a network graph model based on the collected mapping traffic, and dividing the network space mapping host group by performing network analysis on the network graph model; Step S2, extracting the graph evolution events of the network space mapping host group within a preset time window, and performing mapping behavior recognition on the graph evolution events according to the network space mapping behavior characteristics to adjust the division of the network space mapping host group; Step S3, disassembling or merging the adjusted cyberspace mapping host group through statistical attribute analysis to form a cyberspace mapping organization, and performing cyberspace mapping behavior analysis on the cyberspace mapping organization; Wherein, in step S1: Arrange sensing nodes, collect the surveying and mapping traffic through the sensing nodes, and convert the surveying and mapping traffic into structured data to form a surveying and mapping traffic PCAP data packet; Filter out data packets with the same five-tuple information from the surveying and mapping traffic PCAP data packets, and sort and merge the filtered data packets in chronological order to form a network flow; The five-tuple information includes: source IP address, destination IP address, source port number, destination port number, and protocol; Extract the source IP and destination IP from each network flow as physical host nodes; use the port number as the node attribute; connect each node through a directed edge, wherein the directed edge represents the communication behavior from the source IP to the destination IP; thereby constructing the network graph model; Wherein, the network graph model is stored in a Neo4j graph database; Wherein, in step S1: Determine the network structure and node importance by calculating the degree, betweenness centrality, and closeness centrality of each node; Among them, degree represents the number of node connections, betweenness centrality represents the degree of intermediary of a node in the network, and closeness centrality represents the degree of close connection between a node and other nodes; The community discovery algorithm is used to divide the nodes in the network into community structures with maximum internal connections and minimum external links, which serve as the network space mapping host group; In the statistical attribute analysis of step S3, the statistical attributes analyzed specifically include: Similarity between host groups within the current time window, similarity of host IP addresses between host groups, similarity of IP addresses of member hosts within a host group; Comparison of the number of member hosts within the host group in the current time window, and comparison of the number of member hosts within the host group in the current time window and in the historical time window; The frequency of occurrence of member hosts within the host group in the historical time window in the current time window; Wherein, in the step S3, the cyberspace mapping behavior analysis includes: Calculate the statistical characteristics of the number of organizational members of the cyberspace mapping organization, including quartiles, mean, variance, and maximum value; wherein: the quartiles represent the central position of the number of organizational members and the degree of dispersion of the distribution; the mean represents the central trend of the organizational data; the variance represents the degree of change and uncertainty of the number of organizations, and when the variance is higher than the third discriminant value, the cyberspace mapping organization acts alone, otherwise the cyberspace mapping organization acts in a unified manner; the maximum value represents the abnormal behavior of the cyberspace mapping organization; Utilizing the attributes of nodes and edges in the graph model, statistical display of the scanning ports, protocols and tools of the cyberspace mapping organization is performed, including visual display in different scanning time windows, different scanning frequencies and different scanning cycles; the mapping behavior of the cyberspace mapping organization is visualized based on geographic location distribution.
2. According to the method of network space mapping host group discovery and behavior analysis based on graph analysis in claim 1, it is characterized in that: In the step S2, based on the relationship mapping of the network graph model in adjacent time windows, the graph evolution events of the network space mapping host group are extracted; wherein: The graph evolution events include the increase, stability, reduction, merger, split, addition, and disappearance of the network space mapping host group; When the graph evolution event is increase, the number of host group members increases, the number of group identifiers remains unchanged, and the event change type is scale change; When the graph evolution event is stable, the number of host group members remains unchanged, the number of group identifiers remains unchanged, and the event change type is scale change; When the graph evolution event is shrinking, the number of host group members decreases, the number of group identifiers remains unchanged, and the event change type is scale change; When the graph evolution event is a merge, the number of host group members increases / remains unchanged / decreases, the number of group identifiers decreases, and the event change type is a structural change; When the graph evolution event is split, the number of host group members increases / remains unchanged / decreases, the number of group identifiers increases, and the event change type is structural change; When the graph evolution event is a new addition, the number of host group members increases, the number of group identifiers increases, and the event change type is a structural change; When the graph evolution event is disappearance, the number of host group members decreases, the number of group identifiers decreases, and the event change type is structural change.
3. According to the method of network space mapping host group discovery and behavior analysis based on graph analysis in claim 2, it is characterized in that: In step S2, the cyberspace mapping behavior characteristics include short-term characteristics, high-frequency characteristics, specific port scanning, remote sources, and irregularity; wherein: The transient characteristic means that the surveying and mapping traffic has the transient characteristic, and the communication connection time between different nodes is lower than the first judgment value; High-frequency characteristics refer to: the mapping traffic has high-frequency characteristics, the number of targets scanned in a fixed time interval is higher than the second discrimination value, and the node degree distribution is skewed; Specific port scanning means: the mapping traffic is scanned for a specific port or a specific protocol, and the port attribute of the node is a specific port; Off-site sources refer to: surveying and mapping organizations use distributed surveying and mapping nodes to perform scanning tasks, surveying and mapping traffic comes from different IP addresses, and there is a connection relationship between nodes that spans geographical locations; Irregularity means that the surveying and mapping traffic does not have periodicity, and the network space surveying and mapping host group has structural changes.
4. A network space mapping host group discovery and behavior analysis device based on graph analysis, characterized in that: The apparatus comprises a processing unit configured to perform the following steps: Step S1, constructing a network graph model based on the collected mapping traffic, and dividing the network space mapping host group by performing network analysis on the network graph model; Step S2, extracting the graph evolution events of the network space mapping host group within a preset time window, and performing mapping behavior recognition on the graph evolution events according to the network space mapping behavior characteristics to adjust the division of the network space mapping host group; Step S3, disassembling or merging the adjusted cyberspace mapping host group through statistical attribute analysis to form a cyberspace mapping organization, and performing cyberspace mapping behavior analysis on the cyberspace mapping organization; Wherein, in step S1: Arrange sensing nodes, collect the surveying and mapping traffic through the sensing nodes, and convert the surveying and mapping traffic into structured data to form a surveying and mapping traffic PCAP data packet; Filter out data packets with the same five-tuple information from the surveying and mapping traffic PCAP data packets, and sort and merge the filtered data packets in chronological order to form a network flow; The five-tuple information includes: source IP address, destination IP address, source port number, destination port number, and protocol; Extract the source IP and destination IP from each network flow as physical host nodes; use the port number as the node attribute; connect each node through a directed edge, wherein the directed edge represents the communication behavior from the source IP to the destination IP; thereby constructing the network graph model; Wherein, the network graph model is stored in a Neo4j graph database; Wherein, in step S1: Determine the network structure and node importance by calculating the degree, betweenness centrality, and closeness centrality of each node; Among them, degree represents the number of node connections, betweenness centrality represents the degree of intermediary of a node in the network, and closeness centrality represents the degree of close connection between a node and other nodes; The community discovery algorithm is used to divide the nodes in the network into community structures with maximum internal connections and minimum external links, which serve as the network space mapping host group; In the statistical attribute analysis of step S3, the statistical attributes analyzed specifically include: Similarity between host groups within the current time window, similarity of host IP addresses between host groups, similarity of IP addresses of member hosts within a host group; Comparison of the number of member hosts within the host group in the current time window, and comparison of the number of member hosts within the host group in the current time window and in the historical time window; The frequency of occurrence of member hosts within the host group in the historical time window in the current time window; Wherein, in the step S3, the cyberspace mapping behavior analysis includes: Calculate the statistical characteristics of the number of organizational members of the cyberspace mapping organization, including quartiles, mean, variance, and maximum value; wherein: the quartiles represent the central position of the number of organizational members and the degree of dispersion of the distribution; the mean represents the central trend of the organizational data; the variance represents the degree of change and uncertainty of the number of organizations, and when the variance is higher than the third discriminant value, the cyberspace mapping organization acts alone, otherwise the cyberspace mapping organization acts in a unified manner; the maximum value represents the abnormal behavior of the cyberspace mapping organization; Utilizing the attributes of nodes and edges in the graph model, statistical display of the scanning ports, protocols and tools of the cyberspace mapping organization is performed, including visual display in different scanning time windows, different scanning frequencies and different scanning cycles; the mapping behavior of the cyberspace mapping organization is visualized based on geographic location distribution.
5. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the network space mapping host group discovery and behavior analysis method based on graph analysis described in any one of claims 1 to 3 are implemented.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps in the network space mapping host group discovery and behavior analysis method based on graph analysis described in any one of claims 1 to 3 are implemented.
Citation Information
Patent Citations
Abnormal mainframe group detection method based on dynamic graph
CN107528734A
Network asset surveying and mapping discovery method and device based on big data
CN111555988A
Abnormal traffic detection method for host community behaviors
CN112437091A
Network space surveying and mapping node discovery and affiliation discrimination method and device
CN117041070A
Network scanning behavior detection method and device
CN117675273A
