A spatial domain alarm community division method based on diffusion nodes

By adopting a community division method based on diffusion nodes in the industrial Internet, combining business attributes and topological characteristics, the problems of high cost and poor accuracy in alarm log correlation analysis are solved, and stronger business feature interpretability and real structure are achieved. Retention improves analysis efficiency.

CN115695147BActive Publication Date: 2024-11-26XI AN JIAOTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211337758.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-28
Publication Date
2024-11-26
Estimated Expiration
2042-10-28

AI Technical Summary

Technical Problem

There are problems of high cost and poor accuracy in the correlation analysis of industrial Internet alarm logs. The existing community division method cannot meet the business scenario needs of industrial Internet, and it is easy to miss high-risk events.

Method used

The spatial domain alarm community division method based on diffusion nodes is adopted. By constructing an alarm log association diagram, diffusion nodes are selected based on the business attribute characteristics and topological structure characteristics of the node, community division is carried out, and optimized communities are obtained through tag propagation and merger. structure.

Benefits of technology

It improves the accuracy and interpretability of community division, can correctly handle overlapping nodes, maximize the real structural characteristics of the community, and reduce the complexity of the algorithm, and achieve real-time analysis speed and efficiency improvement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115695147B_ABST
    Figure CN115695147B_ABST
Patent Text Reader

Abstract

The present invention provides a method for dividing a spatial domain alarm community based on diffusion nodes, comprising the following steps: Step 1: extract information from the obtained alarm log data to be processed to obtain an alarm data set; Step 2: construct an alarm log association graph according to the alarm data set; Step 3: select diffusion nodes in combination with the business attribute characteristics and topological structure characteristics of the nodes in the alarm log association graph; Step 4: divide the spatial domain alarm community using the selected diffusion nodes to obtain multiple community structures. The present invention effectively solves the difficulties faced by existing community division methods in processing the association relationship between communities, allows the same node to belong to multiple alarm communities, improves the certainty and accuracy of the overlapping community division algorithm results, and is more in line with the actual business scenarios of the actual industrial Internet.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of industrial Internet alarm log correlation analysis, and in particular relates to a method for dividing spatial domain alarm communities based on diffusion nodes. Background Art

[0002] With the rapid development of Internet technology, the scale of the industrial Internet continues to develop and expand, and its level of intelligence, informatization, and automation is rapidly improving. More and more users and terminal smart devices are frequently interacting with the industrial Internet and remotely controlling it, which greatly improves the production efficiency and quality of the industrial Internet. However, due to the extension and expansion of the protection boundary, new network attacks and security risks emerge in an endless stream. Even though most industrial Internet systems have deployed massive security monitoring and protection equipment, the total number of security alarms still shows an exponential upward trend, and has the characteristics of complex types and wide sources, which brings severe challenges to the safe and stable operation of the industrial Internet. At present, the three major difficult problems faced in the field of correlation analysis and processing of industrial Internet alarm logs are:

[0003] (1) The alarm data volume is large, the analysis cost is high, and it is time-consuming and labor-intensive;

[0004] (2) Alarm data is mainly false alarms and low-risk alarms, which seriously distracts the attention of security analysts;

[0005] (3) The equipment alarm monitoring systems of various security vendors are independent of each other, which makes it easy to miss high-risk events and difficult to conduct correlation analysis and trace back to the source to take countermeasures.

[0006] In response to the above problems, a large number of technologies based on community partitioning methods have been proposed. However, the existing community partitioning methods mainly consider the topological characteristics and compact structure of the network, lack the scalability of business scenarios, and are suitable for application scenarios with characteristics such as compact clusters within clusters and sparse clusters between clusters, such as social media. There is a large amount of randomness and uncertainty in the industrial Internet alarm log association network, and the interpretability of its partitioning results is poor, which cannot meet the needs of industrial Internet alarm log association analysis. Summary of the invention

[0007] The purpose of the present invention is to provide a spatial domain alarm community division method based on diffusion nodes, which solves the defects of high cost and poor accuracy in the existing industrial Internet alarm log correlation analysis.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is:

[0009] The present invention provides a method for dividing a spatial domain alarm community based on diffusion nodes, comprising the following steps:

[0010] Step 1: extract information from the acquired alarm log data to be processed to obtain an alarm data set;

[0011] Step 2: construct an alarm log association graph based on the obtained alarm data set;

[0012] Step 3: Select diffusion nodes based on the business attribute characteristics and topological structure characteristics of the nodes in the alarm log association graph;

[0013] Step 4: Use the selected diffusion nodes to divide the spatial domain alarm community to obtain multiple community structures.

[0014] Preferably, in step 1, the obtained alarm log data to be processed is parsed to obtain an alarm data set, and the specific method is:

[0015] Extract the alarm time, alarm source IP address, source port number, alarm destination IP address, destination port number and alarm type from the alarm log data to be processed;

[0016] The extracted alarm time, alarm source IP address, source port number, alarm destination IP address, destination port number and alarm type are processed according to the format to obtain an alarm event;

[0017] All alarm events are combined to form an alarm data set.

[0018] Preferably, in step 2, an alarm log association graph is constructed based on the obtained alarm data set, and the specific method is:

[0019] According to the association between the source IP and the destination IP of each alarm event in the alarm data set, each alarm event is converted into a directed edge;

[0020] The alarm log association graph is constructed based on the obtained directed edges.

[0021] Preferably, in step 3, the diffusion nodes are selected by combining the business attribute characteristics and topological structure characteristics of the nodes in the alarm log association graph, and the specific method is:

[0022] S301, randomly select a node from the alarm log association diagram, and determine whether the node is a known business node based on the asset ledger information. If yes, execute S302, otherwise execute S304;

[0023] S302, determining whether the known service node is a leaf node in the graph, if not a leaf node, marking it as a service diffusion node and executing S301, if it is a leaf node, executing S303;

[0024] S303, determining whether its neighbor node is a service diffusion node, if yes, executing S301, if no, marking it as a service diffusion node and executing S301;

[0025] S304, determining whether the degree of the node is greater than 3, if yes, marking it as a topological diffusion node and executing S301, if no, executing S305;

[0026] S305 , determining whether the neighbor node of the node is a diffusion node, if yes, executing S301 , if no, marking it as a service diffusion node, and executing S301 .

[0027] Preferably, in step 4, the space domain alarm community is divided using the selected diffusion nodes to obtain multiple community structures, and the specific method is:

[0028] The obtained diffusion node is used as the center of the community for label propagation, and its neighboring nodes are classified into the community where it is located, thus obtaining multiple single-center communities;

[0029] Merge the single-center communities with highly similar topological features to obtain the merged community structure;

[0030] Redundant communities in the merged community structure are eliminated to obtain optimized community division results, thereby obtaining multiple community structures.

[0031] Preferably, the obtained diffusion node is used as the center of the community for label propagation, and its neighboring nodes are classified into the community where it is located, so as to obtain multiple single-center communities. The specific method is:

[0032] Each diffusion node is regarded as a community center, and the nodes directly connected to the diffusion node are called neighbor nodes of the diffusion node;

[0033] All neighbor nodes constitute the neighbor node set corresponding to the diffusion node;

[0034] The neighbor node set corresponding to the diffusion node is included in the community where the node belongs to, and the single-center community corresponding to the diffusion node is obtained.

[0035] Preferably, community structures with highly similar topological features are merged to obtain a merged community structure, and the specific method is:

[0036] Traverse all single-center communities and calculate the Jaccard similarity coefficient between the neighbor node sets corresponding to two single-center communities;

[0037] The obtained Jaccard similarity coefficient is compared with the preset threshold. If the Jaccard similarity coefficient is greater than the preset threshold, the two single-center communities are merged to obtain the merged community structure.

[0038] Preferably, redundant communities in the merged community structure are eliminated to obtain an optimized community division result. The specific method is:

[0039] Traverse the community partition result set. If there is a community whose node set is a subset of another community, eliminate the community as a redundant community.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) The present invention can correctly handle overlapping nodes. It effectively solves the problem faced by existing community division methods in handling the relationship between communities, allowing the same node to belong to multiple alarm communities, which is more in line with the actual industrial Internet business scenario. It also solves the problem of low certainty and poor accuracy of existing overlapping community division algorithms.

[0042] (2) The community division method proposed in this invention has stronger interpretability of business features. Combining actual business scenarios and adopting a business-centric approach, this method solves the limitation of most community division algorithms that only consider the topological features of the graph structure, and divides the log association graph into communities with highly similar attribute features, which is more interpretable.

[0043] (3) The present invention can retain the true structural characteristics of the community to the greatest extent possible. Existing community division algorithms often divide multiple independent alarm communities into the same community, while the present invention can not only effectively split the communities that are incorrectly associated, but also correctly merge the multi-center alarms that are incorrectly split, thus retaining the true structural characteristics of the community to the greatest extent possible.

[0044] (4) The algorithm complexity of the method of the present invention is relatively low, and real-time analysis speed can be achieved in a larger-scale industrial Internet. The efficiency of alarm log correlation analysis can be significantly improved through smaller computing overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is an overall flow chart of the spatial domain alarm community division method based on diffusion nodes of the present invention;

[0046] Figure 2 The original alarm log association graph is constructed after parsing and preprocessing the test data;

[0047] Figure 3 Find the algorithm flow chart for diffusion nodes;

[0048] Figure 4 For Figure 2 The effect diagram of applying the diffusion node search algorithm based on the foundation;

[0049] Figure 5This is a diagram showing the effect of community label propagation centered on diffusion nodes;

[0050] Figure 6 For Figure 5 The effect diagram after merging similar communities based on the above data;

[0051] Figure 7 For Figure 6 The effect diagram after eliminating redundant communities. DETAILED DESCRIPTION

[0052] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and test data.

[0053] Figure 1 This is the overall flow chart of the community division method based on diffusion nodes proposed in the present invention, which shows the basic framework of the method for correlation analysis of security alarm logs in the industrial Internet scenario.

[0054] The data used in the implementation process of the present invention are all from a power company's network security alarm log database. Since the alarm log data is large in scale and the present method is more suitable for short-term correlation analysis, the alarm log data of four consecutive hours on a normal working day is selected as an example to illustrate the spatial domain community division method based on diffusion nodes proposed in the present invention.

[0055] Specifically, the present invention provides a method for dividing a spatial domain alarm community based on diffusion nodes, comprising the following steps:

[0056] Step 1: Log data analysis and preprocessing

[0057] The network alarm logs collected by various security devices are parsed and field preprocessed according to the corresponding collection format, and six field information including alarm time, alarm source IP address, source port number, alarm destination IP address, destination port number and alarm type are extracted. The alarm logs are converted into alarm events. The format of the alarm event is shown in Table 1.

[0058] Table 1 Alarm event format example

[0059] Alarm time Source IP Source Port Destination IP Destination Port Alarm Type 2011-09-01 09:51:00 A 45210 B 80 Critical Event Audit 2011-09-01 09:57:14 C 61203 D 8080 scanner

[0060] Step 2: Build the alarm log correlation graph

[0061] In step 1, the original device collection log is parsed and preprocessed to obtain the alarm data set C. According to the association between the source IP and the destination IP, each alarm event is converted into a directed edge to construct a log association graph G = (V, E). V is a node set consisting of IP addresses, and E is a set of directed edges from the source IP to the destination IP in each alarm. If an alarm is c = (time, sip, sport, dip, dport, caiegory), then the set V will contain two nodes, sip and dip, and the set E will contain a directed edge.<sip,dip> . Figure 2 The result of constructing a log correlation graph for all alarm logs within four hours is shown.

[0062] Step 3: Diffusion node selection

[0063] Based on the alarm log association graph constructed in step 2, two types of diffusion nodes with significant characteristics are defined: business diffusion nodes and topology diffusion nodes.

[0064] 1) Business diffusion nodes: Depends on the business characteristics of the specific application scenario. In the power system scenario implemented and tested by the present invention, it mainly includes web application servers, cloud data centers, security audit equipment, and terminal Internet access equipment, etc. Business diffusion nodes are obtained by importing asset ledger information in a specific scenario.

[0065] 2) Topological diffusion node: a non-business diffusion node, but with a high degree distribution and a type of node with complex connection relationships in the log association graph. In the present invention, if the out-degree of the external IP address of a non-business diffusion node is greater than 3, it is classified as a topological diffusion node.

[0066] Figure 3 The flowchart of a diffusion node search algorithm proposed by the present invention is shown. The algorithm traverses all nodes in the log topology graph and integrates the business characteristics and topological characteristics of each node and its neighboring nodes to quickly find two types of diffusion nodes. The specific steps of the algorithm are as follows:

[0067] S301: Select a node from the alarm log association diagram, and determine whether the node is a known business node based on the imported asset ledger. If yes, execute S302; otherwise, execute S304;

[0068] S302: Determine whether the known service node is a leaf node in the graph. If it is not a leaf node, mark it as a service diffusion node and execute S301. If it is a leaf node, execute S303.

[0069] S303: Determine whether its neighbor node is a service diffusion node, if yes, execute S301, if no, mark it as a service diffusion node and execute S301;

[0070] S304: Determine whether the degree of the node is greater than 3. If so, mark it as a topological diffusion node and execute S301. If not, execute S305.

[0071] S305: Determine whether the neighbor node of the node is a diffusion node. If so, execute S301. If not, mark it as a service diffusion node and execute S301.

[0072] The algorithm can ensure that each node in the alarm log association graph meets one of the following two conditions: it is a diffusion node or its neighbor node is a diffusion node, and all nodes that may become community centers are marked as diffusion nodes. Since the algorithm only needs to traverse all nodes once, the algorithm complexity is O(N) for an alarm log topology graph with N nodes.

[0073] Figure 4 Shown in Figure 2 The effect diagram of applying the diffusion node search algorithm based on the above is shown. Different types of nodes are characterized by the size of the nodes. The largest node is the topological diffusion node, the second largest node is the business diffusion node, and the smallest node is the non-diffusion node.

[0074] Step 4: Social media tag dissemination

[0075] A community refers to a set of nodes in a network structure that exhibits cluster characteristics and has strong internal associations. Community label propagation refers to the process of propagating community labels along the connection relationship between nodes and edges in the network structure, so that each node belongs to the set of nodes with the strongest association, that is, the community.

[0076] After obtaining all the diffusion nodes in the previous step, each diffusion node is regarded as a community center, and the nodes directly connected to each diffusion node are called the neighbor nodes of the diffusion node, and all the nodes connected to it constitute its neighbor node set. Then, a round of label propagation is performed with each diffusion node as the center, that is, all its neighbor nodes are included in the community to which it belongs. Figure 5 The figure shows the division results after the initial community label propagation. It can be seen from the figure that after the initial label propagation, Figure 2 The coupled communities in the graph are all split into single-center communities with diffusion nodes as the center.

[0077] Step 5: Merge similar communities

[0078] Based on the single-center community obtained after a round of label propagation, the neighbor node sets of all single-center communities are traversed, and the Jaccard similarity coefficient is used to measure the similarity between two neighbor node sets. Specifically:

[0079] After comparing the Jaccard similarity coefficients between multiple groups of similar communities, a higher similarity threshold was set at 0.7;

[0080] Traverse all single-center communities and calculate the Jaccard similarity coefficient between the neighbor node sets of any two communities A and B. If the Jaccard similarity coefficient is greater than 0.7, it is considered that the business characteristics represented by the two communities are highly similar and need to be merged into a new community to obtain the merged community structure.

[0081] The expression of Jaccard similarity coefficient is as follows:

[0082]

[0083] The six communities in Table 2 are used as examples to illustrate the process of merging similar communities. Each community is traversed, and the Jaccard similarity coefficient between the current community and the neighbor node sets of other communities is calculated to obtain the similarity matrix.

[0084] Table 2 Examples of six community center nodes combined with neighbor nodes

[0085] Community ID Central Node Neighbor node set Community 1 A {n1, n2, n3, n4, n5, n6, n7} Community 2 B {n1, n2, n3, n4, n5, n6, n7, n8} Community 3 C {n1, n2, n3, n4, n5, n6, n8, n9} Community 4 D {n10, n11, n12} Community 5 E {n10, n11, n12} Community 6 F {n13, n14}

[0086] Table 3 Jaccard similarity coefficient matrix

[0087]

[0088]

[0089] It can be concluded from Table 3 that the similarity coefficients among communities 1, 2, and 3 all exceed the similarity threshold of 0.7, and the similarity coefficients between communities 4 and 5 exceed the threshold of 0.7. Therefore, the above six communities are merged into three new communities, namely {1, 2, 3}, {4, 5}, and {6}.

[0090] Figure 6 The figure shows the results of similarity merging the initial community division results. The three communities framed in black in the figure are the new community structures obtained after similarity merging.

[0091] Step 6: Eliminate redundant communities

[0092] exist Figure 4 At the center of the network, there are two topological diffusion nodes connecting a large number of business diffusion nodes. After label propagation, several repeated community structures are formed. Therefore, it is necessary to use the subset inclusion relationship to further optimize the community division results. The diffusion node communities are traversed two by two. If all the node sets of community A are included in community B, community A is considered to be a redundant community and is removed from the community division results.

[0093] Figure 7The final result of community division after redundant community elimination is shown. Figure 2 Compared with the original log association graph shown in , by selecting two types of diffusion nodes as community centers, we can not only split the complex coupled connection relationship into a highly interpretable community model, but also restore the real structural characteristics of the community to the greatest extent, effectively solving the problem of complex and difficult to understand association of alarm log data.

Claims

1. A spatial domain alarm community division method based on diffusion nodes, characterized in that: The following steps are involved: Step 1: extract information from the acquired alarm log data to be processed to obtain an alarm data set; Step 2: Construct an alarm log association graph based on the alarm data set; Step 3: Select diffusion nodes based on the business attribute characteristics and topological structure characteristics of the nodes in the alarm log association graph; Step 4: Use the selected diffusion nodes to divide the spatial domain alarm community to obtain multiple community structures; In step 3, the diffusion nodes are selected based on the business attribute characteristics and topological structure characteristics of the nodes in the alarm log association graph. The specific method is: S301, randomly select a node from the alarm log association diagram, and determine whether the node is a known business node based on the asset ledger information. If yes, execute S302, otherwise execute S304; S302, determining whether the known service node is a leaf node in the graph, if not a leaf node, marking it as a service diffusion node and executing S301, if it is a leaf node, executing S303; S303, determining whether its neighbor node is a service diffusion node, if yes, executing S301, if no, marking it as a service diffusion node and executing S301; S304, determining whether the degree of the node is greater than 3, if so, marking it as a topological diffusion node and executing S301, if not, executing S305; S305, determining whether the neighbor node of the node is a diffusion node, if yes, executing S301, if no, marking it as a service diffusion node, and executing S301; In step 4, the selected diffusion nodes are used to divide the spatial domain alarm community to obtain multiple community structures. The specific method is: The obtained diffusion node is used as the center of the community for label propagation, and its neighboring nodes are classified into the community where it is located, thus obtaining multiple single-center communities; Merge the single-center communities with highly similar topological features to obtain the merged community structure; Redundant communities in the merged community structure are eliminated to obtain optimized community division results, thereby obtaining multiple community structures.

2. According to the method for dividing spatial domain alarm communities based on diffusion nodes in claim 1, it is characterized in that: In step 1, the obtained alarm log data to be processed is parsed to obtain an alarm data set. The specific method is: Extract the alarm time, alarm source IP address, source port number, alarm destination IP address, destination port number and alarm type from the alarm log data to be processed; The extracted alarm time, alarm source IP address, source port number, alarm destination IP address, destination port number and alarm type are processed according to the format to obtain an alarm event; All alarm events are combined to form an alarm data set.

3. According to the method for dividing spatial domain alarm communities based on diffusion nodes in claim 1, it is characterized in that: In step 2, an alarm log association graph is constructed based on the obtained alarm data set. The specific method is: According to the association between the source IP and the destination IP of each alarm event in the alarm data set, each alarm event is converted into a directed edge; The alarm log association graph is constructed based on the obtained directed edges.

4. According to the method for dividing spatial domain alarm communities based on diffusion nodes in claim 1, it is characterized in that: The obtained diffusion node is used as the center of the community for label propagation, and its neighboring nodes are classified into the community where it is located, and multiple single-center communities are obtained. The specific method is: Each diffusion node is regarded as a community center, and the nodes directly connected to the diffusion node are called neighbor nodes of the diffusion node; All neighbor nodes constitute the neighbor node set corresponding to the diffusion node; The neighbor node set corresponding to the diffusion node is included in the community where the node belongs to, and the single-center community corresponding to the diffusion node is obtained.

5. The method for dividing spatial domain alarm communities based on diffusion nodes according to claim 1 is characterized in that: The community structures with highly similar topological features are merged to obtain the merged community structure. The specific method is: Traverse all single-center communities and calculate the Jaccard similarity coefficient between the neighbor node sets corresponding to two single-center communities; The obtained Jaccard similarity coefficient is compared with the preset threshold. If the Jaccard similarity coefficient is greater than the preset threshold, the two single-center communities are merged to obtain the merged community structure.

6. The method for dividing spatial domain alarm communities based on diffusion nodes according to claim 1, characterized in that: Eliminate redundant communities in the merged community structure to obtain the optimized community division result. The specific method is: Traverse the community partition result set. If there is a community whose node set is a subset of another community, eliminate the community as a redundant community.