Multi-machine alarm log association method and device, electronic equipment and storage medium

Abstract

The application provides a multi-machine alarm log association method and device, electronic equipment and a storage medium, and belongs to the technical field of communication. In the method, an internal memory graph is used for association. The internal memory graph can realize multi-machine (between nodes of different entity elements) association. The association points of the internal memory graph are at least one of the sub-nodes, that is, multiple association points are allowed to be set, the dimension of the association is greatly increased, the use of the internal memory graph compresses the internal memory occupation, the merging and multi-machine association are all carried out in the internal memory, the storage and network dependence is removed, and real-time and efficient association can be realized. When the multi-machine association is carried out, the association is combined according to the association point granularity, and the association calculation amount is greatly reduced. In addition, the first internal memory graph and the second internal memory graph can be automatically combined and expanded, the third internal memory graph manually triggered and expanded by the front end can be associated with the current attack event graph, and the expandability is good.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method, apparatus, electronic device, and storage medium for associating alarm logs across multiple machines. Background Technology

[0002] In a real alarm detection process, the rules will trigger a large number of false alarms, which may result in a large number of alarms being reported. It is time-consuming and laborious to investigate and resolve the alarms one by one, and it may also cause you to miss real attack alarms.

[0003] A real attack typically involves multiple machines switching between them. Traditional single-machine alerts aggregated into attack events can only reduce the number of alerts and cannot depict the complete attack process. This provides limited help in tracing the attack source. It is generally necessary to combine other auxiliary information (third-party alerts, threat intelligence, event logs, etc.) to assist in investigation and confirmation.

[0004] Traditional solutions involve constructing a network security event graph and dividing it into multiple alarm clusters. Graph computation is used to extract topological features to associate different alarms, which is computationally intensive and difficult to achieve real-time association. Another solution generates corresponding attack events based on a self-built graph database, Threat Graph (including raw data and alarm data), which requires significant storage and computation costs, and the multi-machine association effect is generally poor. Yet another solution achieves real-time and efficient aggregation and association of alarms on a single machine, but this relies on a graph database, which consumes relatively large resources and cannot support automatic and manual expansion of auxiliary information (third-party alarms, threat intelligence, event logs, etc.).

[0005] In summary, the traditional single-machine alarm real-time efficient aggregation and correlation scheme has the following drawbacks:

[0006] High resource consumption: The relationships exist in a distributed graph database. Each association involves multiple graph query and merge operations, which consumes network and disk resources, resulting in high resource overhead.

[0007] Multi-machine association cannot be supported: This is mainly due to the fact that a single complete attack path involves multiple machines. The attack events associated with a single machine cannot depict the complete attack process and have limited help in tracing the attack source. Generally, it is necessary to combine the event log context and other auxiliary investigation and confirmation.

[0008] Poor scalability: It only involves its own alarm association. The actual attack process tracing requires combining alarms and other auxiliary information (third-party alarms, threat intelligence, event logs, etc.) to customize and manually expand the current overall attack graph to depict the complete attack path and quickly locate and trace the source.

[0009] In summary, traditional single-machine alarm log association methods suffer from technical problems such as high resource consumption, inability to support multi-machine association, and poor scalability. Summary of the Invention

[0010] In view of this, the purpose of the present invention is to provide a multi-machine alarm log association method, apparatus, electronic device and storage medium to alleviate the technical problems of traditional single-machine alarm log association methods, such as large resource consumption, inability to support multi-machine association and poor scalability.

[0011] In a first aspect, embodiments of the present invention provide a method for associating multi-machine alarm logs, including:

[0012] Real-time alarms and / or real-time logs are converted into a first memory graph, and the association points of the first memory graph are calculated according to preset association point rules to obtain a first memory graph with association point information. The first memory graph includes: multiple nodes and edges. The nodes include: parent nodes and child nodes. The edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. The edges are used to represent access relationships or parent-child relationships between nodes. The nodes are used to represent entity elements. The association point is at least one of the child nodes.

[0013] The auxiliary information stored by the third party is converted into a second memory map, and the association points of the second memory map are calculated according to the preset association point rules to obtain a second memory map with association point information;

[0014] The first memory map with associated point information and the second memory map with associated point information are merged according to the node granularity to obtain an association detection map with associated point information.

[0015] The association detection graph and the historical attack event graph with association point information are associated by multiple machines according to the granularity of the association points, or the association detection graph and the historical attack event graph with association point information are associated by multiple machines according to the time dimension to obtain the current attack event graph.

[0016] The current attack event graph is displayed on the front end, and a third memory graph with associated point information that is manually triggered and extended is associated with the current attack event graph on the front end to obtain an updated attack event graph.

[0017] Furthermore, the entity elements include: device entity, IP address entity, process entity, and file path entity, wherein the device entity includes: host, Kubernetes, container, DNS, and IP.

[0018] Furthermore, multi-machine association is performed on the association detection graph and historical attack event graph containing association point information according to the granularity of association points, including:

[0019] If the target associated point in the association detection graph with associated point information is the same as the target associated point in the historical attack event graph, then the target associated point in the association detection graph with associated point information and the target associated point in the historical attack event graph are merged to obtain the current attack event graph.

[0020] Furthermore, multi-machine association is performed on the association detection graph and historical attack event graph with associated point information according to the time dimension, including:

[0021] If the difference between the timestamp of the target node in the association detection graph with associated point information and the timestamp of the target node in the historical attack event graph is less than a preset time threshold, then the target node in the association detection graph with associated point information and the target node in the historical attack event graph are merged to obtain the current attack event graph.

[0022] Furthermore, multi-machine association is performed on the association detection graph and historical attack event graph containing association point information according to the granularity of association points, or multi-machine association is performed on the association detection graph and historical attack event graph containing association point information according to the time dimension, including:

[0023] First, perform multi-machine association on the association detection graph and historical attack event graph containing association point information according to the granularity of association points;

[0024] If no association can be established, then the association detection graph with association point information and the historical attack event graph are further associated by multiple machines according to the time dimension to obtain the current attack event graph.

[0025] Furthermore, the current attack event graph is displayed on the front end, including:

[0026] Display the current attack event graph on the front end according to the front end requirements.

[0027] Furthermore, the method also includes:

[0028] The updated attack event graph is used to update the historical attack event graph, so as to perform subsequent multi-machine alarm log association based on the updated historical attack event graph.

[0029] Secondly, embodiments of the present invention also provide a multi-machine alarm log association device, comprising:

[0030] The first conversion unit is used to convert real-time alarms and / or real-time logs into a first memory graph, and calculate the association points of the first memory graph according to a preset association point rule to obtain a first memory graph with association point information. The first memory graph includes: multiple nodes and edges. The nodes include: parent nodes and child nodes. The edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. The edges are used to represent access relationships or parent-child relationships between nodes. The nodes are used to represent entity elements. The association point is at least one of the child nodes.

[0031] The second conversion unit is used to convert the auxiliary information stored by the third party into a second memory map, and calculate the association points of the second memory map according to the preset association point rules to obtain a second memory map with association point information.

[0032] The merging unit is used to merge the first memory map with associated point information and the second memory map with associated point information according to the node granularity to obtain an association detection map with associated point information;

[0033] The first multi-machine association unit is used to perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the granularity of association points, or to perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the time dimension to obtain the current attack event graph.

[0034] The second multi-machine association unit is used to display the current attack event graph on the front end, and to associate the manually triggered extended third memory graph with the current attack event graph with the current attack event graph on the front end to obtain an updated attack event graph.

[0035] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0036] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to perform the method described in any of the first aspects above.

[0037] In this embodiment of the invention, a multi-machine alarm log association method is provided, comprising: converting real-time alarms and / or real-time logs into a first memory graph, and calculating association points in the first memory graph according to preset association point rules to obtain a first memory graph with association point information, wherein the first memory graph includes: multiple nodes and edges, nodes include: parent nodes and child nodes, edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, edges are used to represent access relationships or parent-child relationships between nodes, nodes are used to represent entity elements, and association points are at least one of the child nodes; converting auxiliary information stored by a third party into a second memory graph, and calculating association points in the first memory graph according to preset rules. The association point rule calculates the association points of the second memory graph, resulting in a second memory graph with association point information. The first and second memory graphs with association point information are merged according to node granularity to obtain an association detection graph with association point information. The association detection graph with association point information and the historical attack event graph are then multi-machine associated according to association point granularity, or multi-machine associated according to the time dimension, to obtain the current attack event graph. The current attack event graph is displayed on the front end, and a manually triggered extended third memory graph with association point information is multi-machine associated with the current attack event graph on the front end to obtain an updated attack event graph. As described above, the multi-machine alarm log association method of the present invention uses a memory graph for association. The edges of this memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. Nodes represent entity elements, thus enabling multi-machine (inter-node association between different entity elements) associations. Each association point in the memory graph is at least one of the child nodes, allowing for multiple association points, significantly increasing the dimensionality of the association. The use of the memory graph compresses memory usage; merging and multi-machine association are all performed in memory, eliminating storage and network dependencies, enabling real-time and efficient association. Simultaneously, it can be based on… The system features a customized memory graph that supports efficient multi-machine association algorithms. During multi-machine association, pairwise associations are no longer performed at the node level; instead, associations are merged at the association point level, significantly reducing computational load. Furthermore, the system automatically merges and expands the first memory graph (with association point information) obtained in real-time and the second memory graph (with association point information) converted from auxiliary information stored in third-party databases. It also supports multi-machine association between the third memory graph (with association point information) and the current attack event graph, which is manually triggered by the front end. This provides excellent scalability and alleviates the technical problems of traditional single-machine alarm log association methods, such as high resource consumption, inability to support multi-machine association, and poor scalability. Attached Figure Description

[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 A flowchart illustrating a multi-machine alarm log association method provided in an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the association engine provided in an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the memory graph structure provided in an embodiment of the present invention;

[0042] Figure 4 This is a multi-machine overall schema diagram provided in an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the merged correlation detection map provided in an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of multi-machine association provided in an embodiment of the present invention;

[0045] Figure 7 A schematic diagram illustrating the multi-machine association results provided in an embodiment of the present invention;

[0046] Figure 8 A schematic diagram of a multi-machine alarm log association device provided in an embodiment of the present invention;

[0047] Figure 9 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0048] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Traditional single-machine alarm log association methods consume a lot of resources, cannot support multi-machine association, and have poor scalability.

[0050] Based on this, the multi-machine alarm log association method of the present invention uses a memory graph for association. The edges of the memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. Nodes are used to represent entity elements, thus realizing multi-machine (nodes of different entity elements) association. The association point of the memory graph is at least one of the child nodes, that is, multiple association points can be set, which greatly increases the dimension of association. The use of memory graphs compresses memory usage. Merging and multi-machine association are all performed in memory, eliminating storage and network dependencies, and enabling real-time and efficient association. At the same time, the memory graph can be customized according to business needs, supporting efficient multi-machine association algorithms. When associationing multiple machines, it is no longer necessary to perform pairwise associations according to node granularity, but to perform association merging according to association point granularity, which greatly reduces the amount of association calculation. In addition, the first memory graph with association point information obtained in real time and the second memory graph with association point information obtained by converting auxiliary information from third-party storage can be automatically merged and expanded. It also supports multi-machine association between the third memory graph with association point information, which is manually triggered by the front end, and the current attack event graph, which has good scalability.

[0051] To facilitate understanding of this embodiment, a multi-machine alarm log association method disclosed in this embodiment of the invention will first be described in detail.

[0052] Example 1:

[0053] According to an embodiment of the present invention, an embodiment of a multi-machine alarm log association method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0054] Figure 1 This is a flowchart of a multi-machine alarm log association method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0055] Step S102: Convert real-time alarms and / or real-time logs into a first memory graph, and calculate the association points of the first memory graph according to the preset association point rules to obtain a first memory graph with association point information. The first memory graph includes: multiple nodes and edges. Nodes include: parent nodes and child nodes. Edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. Edges are used to represent access relationships or parent-child relationships between nodes. Nodes are used to represent entity elements. Association points are at least one of the child nodes.

[0056] In this embodiment of the invention, an alarm (Detection) is relevant attack information that is detected and reported through rule detection. It usually includes element information related to the current attack process, including devices (including: host / container / K8s / IP / DNS, which will not be emphasized separately later), processes (process tree), network access (connect), and files; an event is the host / container event log collected by the agent, including: regular process creation, network connection, user login, etc., and also includes alarm-related element information; an attack event (Incident) refers to a complete intrusion attack process, which usually includes multiple Detections (alarms) / Events (logs) and multiple Devices.

[0057] like Figure 2 As shown, the real-time graph conversion module converts real-time alarms and / or real-time logs into a first memory graph, and calculates the associated points of the first memory graph according to preset association point rules. The specific process of calculating associated points according to preset association point rules is existing technology and will not be described in detail here. The aforementioned first memory graph includes: multiple nodes and edges. Nodes are used to represent entity elements, and edges are used to represent access relationships or parent-child relationships between nodes. Nodes include: parent nodes and child nodes. An associated point is at least one of the child nodes, that is, multiple associated points can be set. Edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes.

[0058] like Figure 3 As shown, the memory graph (which includes: the first memory graph, the second memory graph, the third memory graph, the historical attack event graph, and the current attack event graph) is composed of nodes and edges with different schema names. The multi-machine memory graph structure adds the relationships between different parent nodes (NodePath) and the relationships between different child nodes of different parent nodes (ElementPath) to the single-machine memory graph structure. For example... Figure 3 As shown, Device2 and Device3 each contain multiple child nodes and edges within their parent nodes, and there are multiple NodePath+ElementPath markers indicating the relationship between the two Devices (i.e., the parent nodes).

[0059] The following is a list and description of commonly used nodes and edges. The multi-machine memory graph structure is built upon the single-machine graph structure, as shown below. Figure 4 As shown, this is the overall schema diagram of the multi-machine architecture. In the memory diagram above, the multi-machine architecture has already naturally completed some connections.

[0060] Table 1 Nodes (Entities):

[0061]

[0062] Table 2 Edges (Relationships):

[0063]

[0064] Step S104: Convert the auxiliary information stored by the third party into a second memory map, and calculate the association points of the second memory map according to the preset association point rules to obtain a second memory map with association point information;

[0065] Specifically, the automatic expansion module queries relevant auxiliary information (such as event log context) from third-party storage, calls the batch graph construction module to convert the auxiliary information into point-edge information similar to alarms, i.e., the second memory graph, and calculates the associated points of the second memory graph according to the preset associated point rules.

[0066] Step S106: Merge the first memory map with associated point information and the second memory map with associated point information according to the node granularity to obtain the association detection map with associated point information.

[0067] Specifically, the graph merging module merges the first memory graph with associated point information and the second memory graph with associated point information according to the node granularity to obtain the associated detection graph (i.e., the Detect graph) with associated point information.

[0068] like Figure 5 As shown, before a single alarm association, a Detect graph is first constructed, including: a first memory graph with associated point information converted from the alarm, and a second memory graph with associated point information converted from automatic expansion rules (i.e., auxiliary information stored by a third party). These are then merged according to node granularity to form an association detection graph (i.e., the Detect graph) with associated point information. Each Device may have multiple associated points within the same Detect graph, such as... Figure 5 In the diagram, the circles with a background color are the connection points.

[0069] Step S108: Perform multi-machine association on the association detection graph and historical attack event graph with association point information according to the granularity of association points, or perform multi-machine association on the association detection graph and historical attack event graph with association point information according to the time dimension to obtain the current attack event graph;

[0070] Specifically, the multi-machine association module calculates the association between each association detection graph and historical attack event graph containing association point information. The association includes: association at the granularity of association points and association at the time dimension. The result of the association is naturally a multi-machine structure, and it directly outputs the current attack event graph. The structure of the historical attack event graph is the same as that of the memory graph, and will not be described again here.

[0071] Step S110: Display the current attack event graph on the front end, and perform multi-machine association between the manually triggered extended third memory graph with associated point information and the current attack event graph on the front end to obtain an updated attack event graph.

[0072] Specifically, the current attack event graph output by the multi-machine association module is given to the visualization enrichment and statistics module. The visualization enrichment and statistics module performs visualization enrichment and statistics on the current attack event graph according to the front-end display requirements, and then gives it to the front-end for display. The front-end allows querying other auxiliary information (such as event log context, etc.), that is, manually adding auxiliary information. Specific auxiliary information, such as logs, is sent to the batch graph construction module to be converted into a third memory graph with correlation point information, and then sent to the front-end. The multi-machine association between the third memory graph with correlation point information and the current attack event graph is completed directly on the front-end (the process of multi-machine association is the same as the process in step S108 above), and the updated attack event graph is obtained. After refreshing, a message is sent to the multi-machine association module to refresh the historical attack event graph, thereby achieving the effect of front-end editing.

[0073] In this embodiment of the invention, a multi-machine alarm log association method is provided, comprising: converting real-time alarms and / or real-time logs into a first memory graph, and calculating association points in the first memory graph according to preset association point rules to obtain a first memory graph with association point information, wherein the first memory graph includes: multiple nodes and edges, nodes include: parent nodes and child nodes, edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, edges are used to represent access relationships or parent-child relationships between nodes, nodes are used to represent entity elements, and association points are at least one of the child nodes; converting auxiliary information stored by a third party into a second memory graph, and calculating association points in the first memory graph according to preset rules. The association point rule calculates the association points of the second memory graph, resulting in a second memory graph with association point information. The first and second memory graphs with association point information are merged according to node granularity to obtain an association detection graph with association point information. The association detection graph with association point information and the historical attack event graph are then multi-machine associated according to association point granularity, or multi-machine associated according to the time dimension, to obtain the current attack event graph. The current attack event graph is displayed on the front end, and a manually triggered extended third memory graph with association point information is multi-machine associated with the current attack event graph on the front end to obtain an updated attack event graph. As described above, the multi-machine alarm log association method of the present invention uses a memory graph for association. The edges of this memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. Nodes represent entity elements, thus enabling multi-machine (inter-node association between different entity elements) associations. Each association point in the memory graph is at least one of the child nodes, allowing for multiple association points, significantly increasing the dimensionality of the association. The use of the memory graph compresses memory usage; merging and multi-machine association are all performed in memory, eliminating storage and network dependencies, enabling real-time and efficient association. Simultaneously, it can be based on… The system features a customized memory graph that supports efficient multi-machine association algorithms. During multi-machine association, pairwise associations are no longer performed at the node level; instead, associations are merged at the association point level, significantly reducing computational load. Furthermore, the system automatically merges and expands the first memory graph (with association point information) obtained in real-time and the second memory graph (with association point information) converted from auxiliary information stored in third-party databases. It also supports multi-machine association between the third memory graph (with association point information) and the current attack event graph, which is manually triggered by the front end. This provides excellent scalability and alleviates the technical problems of traditional single-machine alarm log association methods, such as high resource consumption, inability to support multi-machine association, and poor scalability.

[0074] The above provides a brief overview of the multi-machine alarm log association method of the present invention. The specific details involved are described in detail below.

[0075] In an optional embodiment of the present invention, the entity elements include: device entity, IP address entity, process entity, and file path entity. The device entity includes: host, Kubernetes, container, DNS, and IP.

[0076] In an optional embodiment of the present invention, multi-machine association is performed on the association detection graph and the historical attack event graph containing association point information according to the granularity of association points, specifically including the following steps:

[0077] If the target associated point in the association detection graph with associated point information is the same as the target associated point in the historical attack event graph, then the target associated point in the association detection graph with associated point information and the target associated point in the historical attack event graph are merged to obtain the current attack event graph.

[0078] Specifically, the historical attack event graph is matched by finding the corresponding historical attack event graph for each Device in the Detect graph, thus transforming the O(n^2) complexity of the tree association into the O(n) complexity of finding the association of a point. (See reference) Figure 6 Iterate through each associated point in Device2 and Device3 in the Detect graph, and calculate the match between each associated point and the associated point of each Device in the existing historical attack event graph (Incident has a preset index). The process 2 of Device2 in the Detect graph (i.e., associated point 2) matches the process 2 of Device2 (i.e., associated point 2) in the historical attack event graph. Therefore, the two can be merged. It can be seen that the last associated historical attack event graph of the Detect graph is Incident1 (attack event 1). The remaining operations are simply graph merging operations based on associated points.

[0079] As shown above, the merged association results are as follows: Figure 7 As shown, this method can achieve multi-machine association, similar to dynamic programming. Each planning only requires calculation and matching with the existing historical attack event graph according to the association points, and theoretically there is no upper limit to the number of associated hosts.

[0080] In an optional embodiment of the present invention, multi-machine association is performed on the association detection graph with associated point information and the historical attack event graph according to the time dimension, specifically including the following steps:

[0081] If the difference between the timestamp of the target node in the association detection graph with associated point information and the timestamp of the target node in the historical attack event graph is less than a preset time threshold, then the target node in the association detection graph with associated point information and the target node in the historical attack event graph are merged to obtain the current attack event graph.

[0082] Specifically, in addition to multi-machine association based on the granularity of association points, the association detection graph and historical attack event graph reported within a certain time range, which contain association point information, are theoretically also considered an attack event and need to be associated. The implementation is quite simple: each Device node (i.e., the parent node) retains the time span of the attack currently involved. When searching for associated historical attack events, if the association point granularity cannot find a match, the nearest time span difference (i.e., the time span between the timestamp of the target node in the association detection graph containing association point information and the timestamp of the target node in the historical attack event graph) is less than a preset time threshold.

[0083] Time-based aggregation uses configuration parameters to control the size of the time span (supports accuracy to milliseconds, default is 2 days) to prevent excessively large spans from potentially causing large-scale attack events. Data transmission costs and front-end display efficiency need to be considered.

[0084] In an optional embodiment of the present invention, multi-machine association is performed on the association detection graph and the historical attack event graph with association point information according to the granularity of association points, or multi-machine association is performed on the association detection graph and the historical attack event graph with association point information according to the time dimension, specifically including the following steps:

[0085] (1) First, perform multi-machine association on the association detection graph and historical attack event graph with association point information according to the granularity of association points;

[0086] (2) If the association cannot be established, then the association detection graph and the historical attack event graph with association point information are associated with each other according to the time dimension to obtain the current attack event graph.

[0087] In an optional embodiment of the present invention, displaying the current attack event graph on the front end specifically includes the following steps:

[0088] Display the current attack event graph on the front end according to the front end requirements.

[0089] Specifically, the aforementioned front-end requirements can be further categorized into front-end display requirements, which can include display-related requirements such as display styles and specific display data.

[0090] In an optional embodiment of the present invention, the method further includes the following steps:

[0091] The historical attack event graph is updated by updating the attack event graph, so that subsequent multi-machine alarm log association can be performed based on the updated historical attack event graph.

[0092] The method of this invention can significantly optimize resource consumption (because it uses memory graphs for association) and associate different alarms in real time and efficiently, generate attack events (incidents) across multiple machines (i.e., devices), and support automatic and manual flexible expansion of associated scenarios and data, efficiently and flexibly depicting the real multi-machine intrusion attack process.

[0093] In the method of this invention, both alarms and logs can be converted into memory graphs. Different memory graphs contain different combinations of nodes and edges. Memory graphs can accurately express the access relationships within and across devices. These combinations are saved by constructing memory graphs and support expansion. At the same time, based on the memory graphs, a multi-machine association algorithm is designed to accurately and efficiently associate multiple devices, depict the real attack path / attack relationship, and form a multi-machine attack event. Alarms contain clear attack information, which may be omitted. In addition to alarms, it is necessary to allow supplementation of key attack information from other data sources (such as logs), including: automatic supplementation based on rules (i.e., the process in step S104) and manual addition of supplementation on the attack event graph (i.e., the process in step S110). This invention designs a memory graph-based association mechanism and an automatic / manual association process.

[0094] This invention has the following characteristics:

[0095] Self-developed memory graph: Developed based on the Go language, the memory graph compresses memory usage, performs all association calculations in memory, removes storage and network dependencies, and enables real-time and efficient association; it can also be customized for business needs and supports efficient association algorithms.

[0096] Multi-machine association algorithm: Combining business characteristics, the association between multiple machines is transformed from O(n^2) computation of pairwise node association of multiple machines to O(n) dynamic programming based on association points, which can greatly reduce the amount of association computation and complete multi-machine association in real time and efficiently.

[0097] Automatic and manual expansion algorithms: The memory graph design inherently supports dynamic expansion; it allows for automatic data querying and conversion into a memory graph according to rules, as well as manual specification of data conversion into a memory graph, and addresses the node merging problem in the expanded graph.

[0098] The key points of this invention are: the composition and construction method of memory graphs; a multi-machine efficient association algorithm based on memory graphs; automatic and manual expansion algorithms based on memory graphs; and the entire multi-machine alarm log association system (i.e., Figure 2 (The associated engine in the system). Its main advantage is that it can build multi-machine attack events in real time, efficiently and accurately based on alarms and logs, trace the attack path, and support flexible automatic and manual expansion.

[0099] Example 2:

[0100] This invention also provides a multi-machine alarm log association device, which is mainly used to execute the multi-machine alarm log association method provided in Embodiment 1 of this invention. The following is a detailed description of the multi-machine alarm log association device provided in this invention.

[0101] Figure 8 This is a schematic diagram of a multi-machine alarm log association device according to an embodiment of the present invention, such as... Figure 8 As shown, the device mainly includes: a first conversion unit 10, a second conversion unit 20, a merging unit 30, a first multi-machine association unit 40, and a second multi-machine association unit 50, wherein:

[0102] The first conversion unit is used to convert real-time alarms and / or real-time logs into a first memory graph, and calculate the association points of the first memory graph according to the preset association point rules to obtain a first memory graph with association point information. The first memory graph includes: multiple nodes and edges. The nodes include: parent nodes and child nodes. The edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. The edges are used to represent the access relationship or parent-child relationship between nodes. The nodes are used to represent entity elements. The association point is at least one of the child nodes.

[0103] The second conversion unit is used to convert the auxiliary information stored by the third party into a second memory map, and calculate the association points of the second memory map according to the preset association point rules to obtain a second memory map with association point information.

[0104] The merging unit is used to merge the first memory map with associated point information and the second memory map with associated point information according to the node granularity to obtain the association detection map with associated point information.

[0105] The first multi-machine association unit is used to perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the granularity of association points, or to perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the time dimension to obtain the current attack event graph.

[0106] The second multi-machine association unit is used to display the current attack event graph on the front end, and to associate the manually triggered extended third memory graph with the current attack event graph with the current attack event graph on the front end to obtain an updated attack event graph.

[0107] In this embodiment of the invention, a multi-machine alarm log association device is provided, comprising: converting real-time alarms and / or real-time logs into a first memory graph, and calculating association points in the first memory graph according to preset association point rules to obtain a first memory graph with association point information, wherein the first memory graph includes: multiple nodes and edges, nodes include: parent nodes and child nodes, edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, edges are used to represent access relationships or parent-child relationships between nodes, nodes are used to represent entity elements, and association points are at least one of the child nodes; converting auxiliary information stored by a third party into a second memory graph, and calculating association points in the first memory graph according to preset rules. The association point rule calculates the association points of the second memory graph, resulting in a second memory graph with association point information. The first and second memory graphs with association point information are merged according to node granularity to obtain an association detection graph with association point information. The association detection graph with association point information and the historical attack event graph are then multi-machine associated according to association point granularity, or multi-machine associated according to the time dimension, to obtain the current attack event graph. The current attack event graph is displayed on the front end, and a manually triggered extended third memory graph with association point information is multi-machine associated with the current attack event graph on the front end to obtain an updated attack event graph. As described above, the multi-machine alarm log association device of the present invention uses a memory graph for association. The edges of this memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. Nodes represent entity elements, thus enabling multi-machine (nodes of different entity elements) association. The association point in the memory graph is at least one of the child nodes, allowing multiple association points to be set, greatly increasing the dimension of the association. The use of the memory graph compresses memory usage; merging and multi-machine association are all performed in memory, eliminating storage and network dependencies, enabling real-time and efficient association. Simultaneously, it can be based on… The system features a customized memory graph that supports efficient multi-machine association algorithms. During multi-machine association, pairwise associations are no longer performed at the node level; instead, associations are merged at the association point level, significantly reducing computational load. Furthermore, the system automatically merges and expands the first memory graph (with association point information) obtained in real-time and the second memory graph (with association point information) converted from auxiliary information stored in third-party databases. It also supports multi-machine association between the third memory graph (with association point information) and the current attack event graph, which is manually triggered by the front end. This provides excellent scalability and alleviates the technical problems of traditional single-machine alarm log association methods, such as high resource consumption, inability to support multi-machine association, and poor scalability.

[0108] Optionally, entity elements include: device entity, IP address entity, process entity, and file path entity. Device entities include: host, Kubernetes, container, DNS, and IP.

[0109] Optionally, the first multi-machine association unit is further configured to: if the target association point in the association detection graph with association point information is the same as the target association point in the historical attack event graph, then merge the target association point in the association detection graph with association point information with the target association point in the historical attack event graph to obtain the current attack event graph.

[0110] Optionally, the first multi-machine association unit is further configured to: if the difference between the timestamp of the target node in the association detection graph with association point information and the timestamp of the target node in the historical attack event graph is less than a preset time threshold, then merge the target node in the association detection graph with association point information and the target node in the historical attack event graph to obtain the current attack event graph.

[0111] Optionally, the first multi-machine association unit is also used to: first perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the granularity of association points; if association is not possible, then perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the time dimension to obtain the current attack event graph.

[0112] Optionally, the second multi-machine association unit is also used to: display the current attack event graph on the front end according to the front end requirements.

[0113] Optionally, the device is also used to: update the historical attack event graph with an updated attack event graph, so as to perform subsequent multi-machine alarm log association based on the updated historical attack event graph.

[0114] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0115] like Figure 9 As shown in the embodiment of this application, an electronic device 600 includes a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 via the bus. The processor 601 executes the machine-readable instructions to perform the steps of the multi-machine alarm log association method described above.

[0116] Specifically, the memory 602 and processor 601 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned multi-machine alarm log association method.

[0117] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 602, and processor 601 reads the information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0118] Corresponding to the above-described multi-machine alarm log association method, this application embodiment also provides a computer-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and run by a processor, the machine-executable instructions cause the processor to perform the steps of the above-described multi-machine alarm log association method.

[0119] The multi-machine alarm log association device provided in this application embodiment can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0120] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0121] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0124] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the multi-machine alarm log association method described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0126] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for associating alarm logs across multiple machines, characterized in that, Applied to the association engine, including: Real-time alarms and / or real-time logs are converted into a first memory graph, and the association points of the first memory graph are calculated according to preset association point rules to obtain a first memory graph with association point information. The first memory graph adopts a self-developed memory graph structure, including: multiple nodes and edges. The nodes include: parent nodes and child nodes. The edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. The edges between different parent nodes are used to represent cross-device relationships. The edges between different child nodes of different parent nodes are used to represent cross-device and cross-process relationships. The edges are used to represent access relationships or parent-child relationships between nodes. The nodes are used to represent entity elements. The association point is at least one of the child nodes. Multiple association points are allowed to exist in the same device. The memory graph structure supports a dynamic matching mechanism for multiple association points. The auxiliary information stored by the third party is converted into a second memory map, and the association points of the second memory map are calculated according to the preset association point rules to obtain a second memory map with association point information; The first memory map with associated point information and the second memory map with associated point information are merged according to the node granularity to obtain an association detection map with associated point information. The association detection graph and the historical attack event graph with association point information are associated by multiple machines according to the granularity of the association points, or the association detection graph and the historical attack event graph with association point information are associated by multiple machines according to the time dimension to obtain the current attack event graph. The current attack event graph is displayed on the front end, and a third memory graph with associated point information that is manually triggered and extended is associated with the current attack event graph on the front end to obtain an updated attack event graph. The updated attack event graph is used to update the historical attack event graph, so as to perform subsequent multi-machine alarm log association based on the updated historical attack event graph.

2. The method according to claim 1, characterized in that, The entity elements include: device entity, IP address entity, process entity, and file path entity. The device entity includes: host, Kubernetes, container, DNS, and IP.

3. The method according to claim 1, characterized in that, Multi-machine association is performed on the association detection graph and historical attack event graph containing association point information according to the granularity of association points, including: If the target associated point in the association detection graph with associated point information is the same as the target associated point in the historical attack event graph, then the target associated point in the association detection graph with associated point information and the target associated point in the historical attack event graph are merged to obtain the current attack event graph.

4. The method according to claim 1, characterized in that, Multi-machine association is performed on the association detection graph and historical attack event graph with associated point information according to the time dimension, including: If the difference between the timestamp of the target node in the association detection graph with associated point information and the timestamp of the target node in the historical attack event graph is less than a preset time threshold, then the target node in the association detection graph with associated point information and the target node in the historical attack event graph are merged to obtain the current attack event graph.

5. The method according to claim 1, characterized in that, Multi-machine association is performed on the association detection graph and historical attack event graph containing association point information according to the granularity of association points, or multi-machine association is performed on the association detection graph and historical attack event graph containing association point information according to the time dimension, including: First, perform multi-machine association on the association detection graph and historical attack event graph containing association point information according to the granularity of association points; If no association can be established, then the association detection graph with association point information and the historical attack event graph are further associated by multiple machines according to the time dimension to obtain the current attack event graph.

6. The method according to claim 1, characterized in that, Display the current attack event graph on the front end, including: Display the current attack event graph on the front end according to the front end requirements.

7. A multi-machine alarm log association device, characterized in that, Applied to the association engine, including: The first conversion unit is used to convert real-time alarms and / or real-time logs into a first memory graph, and calculate the association points of the first memory graph according to preset association point rules to obtain a first memory graph with association point information. The first memory graph adopts a self-developed memory graph structure, including: multiple nodes and edges. The nodes include: parent nodes and child nodes. The edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. The edges between different parent nodes are used to represent cross-device relationships. The edges between different child nodes of different parent nodes are used to represent cross-device and cross-process relationships. The edges are used to represent access relationships or parent-child relationships between nodes. The nodes are used to represent entity elements. The association point is at least one of the child nodes. Multiple association points are allowed to exist in the same device. The memory graph structure supports a dynamic matching mechanism for multiple association points. The second conversion unit is used to convert the auxiliary information stored by the third party into a second memory map, and calculate the association points of the second memory map according to the preset association point rules to obtain a second memory map with association point information. The merging unit is used to merge the first memory map with associated point information and the second memory map with associated point information according to the node granularity to obtain an association detection map with associated point information; The first multi-machine association unit is used to perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the granularity of association points, or to perform multi-machine association on the association detection graph and the historical attack event graph with association point information according to the time dimension to obtain the current attack event graph. The second multi-machine association unit is used to display the current attack event graph on the front end, and to associate the manually triggered extended third memory graph with the current attack event graph with the current attack event graph on the front end to obtain an updated attack event graph. The device is further configured to: update the historical attack event graph using the updated attack event graph, so as to perform subsequent multi-machine alarm log association based on the updated historical attack event graph.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 6.

Images