Multi-machine alarm log association method and device, electronic equipment and storage medium

Abstract

The invention provides a multi-machine alarm log association method and device, electronic equipment and a storage medium, and belongs to the technical field of communication, in the method, association is performed by adopting a memory graph, the memory graph can realize multi-machine (nodes of different entity elements) association, the association point of the memory graph is at least one of sub-nodes, and the sub-nodes of the memory graph are different. A plurality of association points are allowed to be set, the association dimensionality is greatly increased, memory occupation is compressed by using a memory graph, merging and multi-machine association are all carried out in a memory, storage and network dependence are removed, and real-time and efficient association can be realized; during multi-machine association, association merging is performed according to the association point granularity, so that the association calculation amount is greatly reduced; in addition, the first memory graph and the second memory graph can be automatically combined and expanded, multi-machine association of the third memory graph which is manually triggered and expanded at the front end and the current attack event graph is also supported, and the expandability is good.

Description

Technical Field

[0001] The present invention relates to the technical field of communications, and in particular, to a multi-machine alarm log association method, device, electronic device, and storage medium. Background Art

[0002] During the actual alarm detection process, a large number of false alarms are triggered by rules, which may lead to a large number of alarms being reported. Troubleshooting and resolving alarms one by one is time-consuming and laborious, and may also miss real attack alarms.

[0003] The actual attack process generally involves jump relationships between multiple machines. Traditional single-machine alarm aggregation into attack events can only reduce the number of alarms, and cannot depict the complete attack process, which is limited in helping with attack traceability. Generally, other auxiliary information (such as third-party alarms, threat intelligence, event logs, etc.) is also needed to assist in troubleshooting and confirmation.

[0004] In traditional solutions, by constructing a network security event graph and dividing it into multiple alarm clusters, graph computing methods are used to extract topological features to associate different alarms, with a large amount of calculation and it is difficult to achieve real-time association. In another solution, based on a self-built graph database Threat Graph (including original data and alarm data) to generate corresponding attack events, it requires a large amount of storage and calculation costs, and the multi-machine association effect is average. In yet another solution, through real-time and efficient aggregation and association of single-machine alarms, it relies on a graph database with relatively large resource occupancy, and cannot support the automatic and manual expansion of auxiliary information (such as third-party alarms, threat intelligence, event logs, etc.).

[0005] In summary, the above traditional real-time and efficient aggregation and association scheme of single-machine alarms has the following disadvantages: Large resource occupancy: The relationships exist in a distributed graph database, and each association involves multiple graph query and merge operations, occupying resources on both the network and the disk, with large resource overhead. Unable to support multi-machine association: Mainly considering the association of single-machine alarms themselves, the actual single complete attack path involves multiple machines, and the attack events associated by single machines cannot depict the complete attack process, which is limited in helping with attack traceability. Generally, the event log context, etc. is also needed to assist in troubleshooting and confirmation. Poor scalability: Only involves the association of its own alarms. In actual attack process traceability, it is necessary to combine alarms and other auxiliary information (such as third-party alarms, threat intelligence, event logs, etc.) to customize and manually expand the currently overall constructed attack graph, depict the complete attack path, and quickly locate and trace.

[0006] In summary, the traditional single-machine alarm log association method has technical problems such as large resource occupancy, inability to support multi-machine association, and poor scalability. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a multi-machine alarm log association method, device, electronic device and storage medium, so as to alleviate the technical problems of large resource occupation, inability to support multi-machine association and poor scalability in the traditional single-machine alarm log association method.

[0008] In a first aspect, an embodiment of the present invention provides a multi-machine alarm log association method, including: Convert real-time alarms and / or real-time logs into a first in-memory graph, and calculate the association points of the first in-memory graph according to a preset association point rule to obtain a first in-memory graph with association point information, where the first in-memory graph includes: multiple nodes and edges, the nodes include: parent nodes and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the association points are at least one of the child nodes; Convert the auxiliary information stored in the third party into a second in-memory graph, and calculate the association points of the second in-memory graph according to the preset association point rule to obtain a second in-memory graph with association point information; Merge the first in-memory graph with association point information and the second in-memory graph with association point information according to node granularity to obtain an association detection graph with association point information; Perform multi-machine association on the association detection graph with association point information and the historical attack event graph according to association point granularity, or perform multi-machine association on the association detection graph with association point information and the historical attack event graph according to the time dimension to obtain the current attack event graph; Display the current attack event graph on the front end, and perform multi-machine association on the third in-memory graph with association point information manually triggered and expanded on the front end and the current attack event graph to obtain an updated attack event graph.

[0009] Further, the entity elements include: device entities, IP address entities, process entities, file path entities, and the device entities include: hosts, K8s, containers, DNS, and IPs.

[0010] Further, performing multi-machine association on the association detection graph with association point information and the historical attack event graph according to association point granularity includes: If the target association points in the association detection graph with association point information are the same as the target association points in the historical attack event graph, then merge the target association points in the association detection graph with association point information and the target association points in the historical attack event graph to obtain the current attack event graph.

[0011] Further, multi-machine association of the association detection graph with associated point information and the historical attack event graph is performed according to the time dimension, including: If the difference between the timestamps of the target nodes in the association detection graph with associated point information and the timestamps of the target nodes in the historical attack event graph is less than a preset time threshold, then the target nodes in the association detection graph with associated point information are merged with the target nodes in the historical attack event graph, thereby obtaining the current attack event graph.

[0012] Further, multi-machine association of the association detection graph with associated point information and the historical attack event graph is performed according to the associated point granularity, or multi-machine association of the association detection graph with associated point information and the historical attack event graph is performed according to the time dimension, including: First, multi-machine association of the association detection graph with associated point information and the historical attack event graph is performed according to the associated point granularity; If association cannot be performed, then multi-machine association of the association detection graph with associated point information and the historical attack event graph is performed according to the time dimension to obtain the current attack event graph.

[0013] Further, the current attack event graph is displayed on the front end, including: The current attack event graph is displayed on the front end according to the front-end requirements.

[0014] Further, the method further includes: The historical attack event graph is updated by using the updated attack event graph, so as to perform subsequent multi-machine alarm log association according to the updated historical attack event graph.

[0015] In a second aspect, an embodiment of the present invention further provides a multi-machine alarm log association device, including: A first conversion unit, configured to convert real-time alarms and / or real-time logs into a first memory graph, and calculate the associated points of the first memory graph according to a preset associated point rule, so as to obtain a first memory graph with associated point information, where the first memory graph includes: a plurality of nodes and edges, the nodes include: a parent node and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the associated points are at least one of the child nodes; A second conversion unit, configured to convert auxiliary information stored in a third party into a second memory graph, and calculate the associated points of the second memory graph according to the preset associated point rule, so as to obtain a second memory graph with associated point information; A merging unit, configured to merge the first memory graph with associated point information and the second memory graph with associated point information according to node granularity to obtain an association detection graph with associated point information; A first multi - device association unit, configured to perform multi - device association on the association detection graph with associated point information and a historical attack event graph according to association point granularity, or perform multi - device association on the association detection graph with associated point information and the historical attack event graph according to a time dimension to obtain a current attack event graph; A second multi - device association unit, configured to display the current attack event graph on a front - end, and perform multi - device association on the current attack event graph and a third memory graph with associated point information manually triggered and extended on the front - end to obtain an updated attack event graph.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method according to any one of the above - mentioned first aspects are implemented.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer - readable storage medium. The computer - readable storage medium stores machine - executable instructions. When the machine - executable instructions are called and run by a processor, the machine - executable instructions cause the processor to run the method according to any one of the above - mentioned first aspects.

[0018] In an embodiment of the present invention, a multi-machine alarm log association method is provided, including: converting real-time alarms and / or real-time logs into a first memory graph, and calculating the association points of the first memory graph according to a preset association point rule to obtain a first memory graph with association point information, where the first memory graph includes: multiple nodes and edges, the nodes include: parent nodes and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the association points are at least one of the child nodes; converting auxiliary information stored in a third party into a second memory graph, and calculating the association points of the second memory graph according to a preset association point rule to obtain a second memory graph with association point information; merging the first memory graph with association point information and the second memory graph with association point information according to node granularity to obtain an association detection graph with association point information; performing multi-machine association on the association detection graph with association point information and the historical attack event graph according to association point granularity, or performing multi-machine association on the association detection graph with association point information and the historical attack event graph according to the time dimension to obtain a current attack event graph; displaying the current attack event graph on the front end, and performing multi-machine association on the manually triggered and expanded third memory graph with association point information and the current attack event graph on the front end to obtain an updated attack event graph. Through the above description, it can be seen that in the multi-machine alarm log association method of the present invention, memory graphs are used for association. The edges of the memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, and the nodes are used to represent entity elements, so that multi-machine (between nodes of different entity elements) association can be realized. The association points of the memory graph are at least one of the child nodes, that is, multiple association points are allowed to be set, greatly increasing the association dimension. The use of the memory graph compresses the memory occupancy, and the merging and multi-machine association are all performed in memory, removing the storage and network dependencies, and being able to perform real-time and efficient association; at the same time, the memory graph can be customized according to the business, supporting an efficient multi-machine association algorithm; when performing multi-machine association, it is no longer necessary to perform pairwise association according to node granularity, but to perform association and merging according to association point granularity, greatly reducing the association calculation amount; in addition, the first memory graph with association point information obtained in real time and the second memory graph with association point information converted from auxiliary information stored in a third party can be automatically merged and expanded, and it also supports multi-machine association between the manually triggered and expanded third memory graph with association point information on the front end and the current attack event graph, with good scalability, alleviating the technical problems of large resource occupation, inability to support multi-machine association, and poor scalability in the traditional single-machine alarm log association method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings required for the description of the specific embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0020] Figure 1 Flowchart of a multi-machine alarm log association method provided by an embodiment of the present invention; Figure 2 Schematic diagram of an association engine provided by an embodiment of the present invention; Figure 3 Schematic diagram of a memory graph structure provided by an embodiment of the present invention; Figure 4 Multi-machine overall Schema relationship diagram provided by an embodiment of the present invention; Figure 5 Schematic diagram of an association detection graph obtained by merging provided by an embodiment of the present invention; Figure 6 Schematic diagram of multi-machine association provided by an embodiment of the present invention; Figure 7 Schematic diagram of multi-machine association results provided by an embodiment of the present invention; Figure 8 Schematic diagram of a multi-machine alarm log association device provided by an embodiment of the present invention; Figure 9 Schematic diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0021] The following will clearly and completely describe the technical solutions of the present invention in combination with the embodiments. Obviously, the described embodiments are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0022] Traditional single-machine alarm log association methods have high resource occupancy, cannot support multi-machine association, and have poor scalability.

[0023] Based on this, in the multi-machine alarm log association method of the present invention, an in-memory graph is used for association. The edges of the in-memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. Nodes are used to represent entity elements, enabling multi-machine (between nodes of different entity elements) association. The association points of the in-memory graph are at least one of the child nodes, that is, multiple association points are allowed to be set, greatly increasing the dimension of association. The use of the in-memory graph compresses memory occupancy, and both merging and multi-machine association are performed in memory, eliminating storage and network dependencies and enabling real-time and efficient association. At the same time, the in-memory graph can be customized according to the business, supporting an efficient multi-machine association algorithm. When performing multi-machine association, there is no need to perform pairwise association according to the node granularity, but to perform association merging according to the association point granularity, greatly reducing the association calculation amount. In addition, the first in-memory graph with association point information obtained in real time and the second in-memory graph with association point information converted from the auxiliary information stored in the third party can be automatically merged and extended, and it also supports the multi-machine association of the third in-memory graph with association point information manually triggered by the front end and the current attack event graph, with good scalability.

[0024] To facilitate the understanding of this embodiment, first, a multi-machine alarm log association method disclosed in the embodiments of the present invention will be introduced in detail.

[0025] Embodiment 1: According to the embodiments of the present invention, an embodiment of a multi-machine alarm log association method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0026] Figure 1 is a flowchart of a multi-machine alarm log association method according to the embodiments of the present invention, as Figure 1 shown, the method includes the following steps: Step S102, convert real-time alarms and / or real-time logs into a first in-memory graph, and calculate the association points of the first in-memory graph according to a preset association point rule to obtain a first in-memory graph with association point information, where the first in-memory graph includes: multiple nodes and edges, the nodes include: parent nodes and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the association points are at least one of the child nodes; In the embodiments of the present invention, an alert (Detection) is relevant attack information determined and reported through rule detection, usually including element information related to the current attack process, including devices (Device) (including: hosts / containers / K8s / Ip / Dns, not emphasized separately hereinafter), processes (process tree Process / Process Tree), network access (Connect), and files; a log (Event) is a host / container event log collected through an Agent, including: regular process creation, network connection, user login, etc., and also includes alert-related element information; an attack event (Incident) refers to a complete intrusion attack process, usually including multiple Detections (alerts) / Events (logs) and multiple Devices (devices).

[0027] As Figure 2 shown, the real-time graph conversion module converts real-time alerts and / or real-time logs into a first in-memory graph, and calculates the correlation points of the first in-memory graph according to a preset correlation point rule. The specific process of calculating the correlation points according to the preset correlation point rule is a prior art and will not be elaborated here. The above first in-memory graph includes: multiple nodes and edges. The nodes are used to represent entity elements, and the edges are used to represent the access relationship or parent-child relationship between the nodes. The nodes include: parent nodes and child nodes. The correlation points are at least one of the child nodes, that is, multiple correlation points are allowed to be set. The edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes.

[0028] As Figure 3 shown, the in-memory graph (here the in-memory graph includes: the first in-memory graph, the second in-memory graph, the third in-memory graph, the historical attack event graph, and the current attack event graph) is composed of point-edge combinations with different Schema names into a graph; the multi-machine in-memory graph structure adds the relationship NodePath between different parent nodes and the relationship ElementPath between different child nodes of different parent nodes on the basis of the single-machine in-memory graph structure. As Figure 3 shown, within the respective parent nodes of Device2 and Device3, there are multiple child nodes and edges, and there are multiple NodePath + ElementPath markings between the two Devices (i.e., parent nodes) to mark the relationship between multiple machines.

[0029] The following shows a list of common nodes and edges and their descriptions. The multi-machine in-memory graph structure is constructed based on the single-machine graph structure. As Figure 4 shown, it is a multi-machine overall Schema relationship graph. In the above in-memory graph, partial associations have been naturally completed in the multi-machine architecture.

[0030] Table 1 Nodes (Entities):

[0031] Table 2 Edge (Relationship):

[0032] In step S104, convert the auxiliary information stored in the third party into a second in-memory graph, and calculate the correlation points of the second in-memory graph according to the preset correlation point rule, so as to obtain a second in-memory graph with correlation point information; Specifically, the automatic expansion module queries relevant auxiliary information (such as event log context, etc.) from the third-party storage, and calls the batch graph construction module to convert the auxiliary information into point-edge information similar to the alarm, that is, the second in-memory graph, and calculates the correlation points of the second in-memory graph according to the preset correlation point rule.

[0033] In step S106, merge the first in-memory graph with correlation point information and the second in-memory graph with correlation point information according to the node granularity to obtain a correlation detection graph with correlation point information; Specifically, the graph merging module merges the first in-memory graph with correlation point information and the second in-memory graph with correlation point information according to the node granularity to obtain a correlation detection graph with correlation point information (i.e., the Detect graph).

[0034] As Figure 5 shown, before single-alarm correlation, first construct a Detect graph, including: the first in-memory graph with correlation point information converted from the alarm + the second in-memory graph with correlation point information converted from the automatic expansion rule (i.e., the auxiliary information stored in the third party), and merge them into a correlation detection graph with correlation point information (i.e., the Detect graph) according to the node granularity. Each Device in the same Detect graph may have multiple correlation points. As Figure 5 shown in, the circles with background color are the correlation points.

[0035] In step S108, perform multi-machine correlation on the correlation detection graph with correlation point information and the historical attack event graph according to the correlation point granularity, or perform multi-machine correlation on the correlation detection graph with correlation point information and the historical attack event graph according to the time dimension to obtain the current attack event graph; Specifically, the multi-machine correlation module calculates the correlation between each correlation detection graph with correlation point information and the historical attack event graph. The correlation includes: correlation at the correlation point granularity and correlation in the time dimension; the result of the correlation is naturally a multi-machine structure, and directly outputs the current attack event graph. The structure of the above historical attack event graph is the same as that of the in-memory graph, which will not be elaborated here.

[0036] Step S110: Display the current attack event graph on the front end, and perform multi - entity association between the third memory graph with associated point information triggered manually on the front end and the current attack event graph to obtain an updated attack event graph.

[0037] Specifically, the current attack event graph output by the multi - entity association module is given to the visualization enrichment and statistics module. The visualization enrichment and statistics module performs visualization enrichment and statistics on the current attack event graph according to the front - end display requirements, and then gives it to the front - end display. The front end allows querying other auxiliary information (such as event log context, etc.), that is, manually adding auxiliary information. Specifically, auxiliary information such as logs is passed to the batch graph construction module to be converted into a third memory graph with associated point information and passed to the front end. The multi - entity association between the third memory graph with associated point information and the current attack event graph is directly completed on the front end (the process of this multi - entity association is the same as the process in step S108 above) to obtain an updated attack event graph. After refreshing, the message notifies the multi - entity association module to refresh the historical attack event graph, thereby achieving the effect of front - end editing.

[0038] In an embodiment of the present invention, a multi-machine alarm log association method is provided, including: converting real-time alarms and / or real-time logs into a first memory graph, and calculating the association points of the first memory graph according to a preset association point rule to obtain a first memory graph with association point information, where the first memory graph includes: multiple nodes and edges, the nodes include: parent nodes and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the association points are at least one of the child nodes; converting auxiliary information stored in a third party into a second memory graph, and calculating the association points of the second memory graph according to a preset association point rule to obtain a second memory graph with association point information; merging the first memory graph with association point information and the second memory graph with association point information according to node granularity to obtain an association detection graph with association point information; performing multi-machine association on the association detection graph with association point information and the historical attack event graph according to association point granularity, or performing multi-machine association on the association detection graph with association point information and the historical attack event graph according to the time dimension to obtain a current attack event graph; displaying the current attack event graph on the front end, and performing multi-machine association on the third memory graph with association point information manually triggered on the front end and the current attack event graph to obtain an updated attack event graph. It can be seen from the above description that in the multi-machine alarm log association method of the present invention, memory graphs are used for association. The edges of the memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, and the nodes are used to represent entity elements, so that multi-machine (between nodes of different entity elements) association can be realized. The association points of the memory graph are at least one of the child nodes, that is, multiple association points are allowed to be set, greatly increasing the association dimension. The use of the memory graph compresses the memory occupancy, and the merging and multi-machine association are all performed in the memory, removing the storage and network dependencies, and being able to perform real-time and efficient association; at the same time, the memory graph can be customized according to the business, supporting an efficient multi-machine association algorithm; when performing multi-machine association, there is no need to perform pairwise association according to node granularity anymore, but to perform association and merging according to association point granularity, greatly reducing the association calculation amount; in addition, the first memory graph with association point information obtained in real time and the second memory graph with association point information converted from auxiliary information stored in a third party can be automatically merged and extended, and it also supports multi-machine association between the third memory graph with association point information manually triggered on the front end and the current attack event graph, with good scalability, alleviating the technical problems of large resource occupation, inability to support multi-machine association, and poor scalability of traditional single-machine alarm log association methods.

[0039] The above briefly introduces the multi-machine alarm log association method of the present invention. The following will describe the specific content involved in detail.

[0040] In an alternative embodiment of the present invention, the entity elements include: device entities, IP address entities, process entities, file path entities, and the device entities include: hosts, K8s, containers, DNS, and IP.

[0041] In an alternative embodiment of the present invention, multi-institutional association is performed on the association detection graph with association point information and the historical attack event graph according to the association point granularity, which specifically includes the following steps: If the target association point in the association detection graph with association point information is the same as the target association point in the historical attack event graph, then the target association point in the association detection graph with association point information is merged with the target association point in the historical attack event graph, thereby obtaining the current attack event graph.

[0042] Specifically, use the association points in each Device in the Detect graph to find and match the corresponding historical attack event graph, and convert the association of the tree O(n^2) to the association search of points O(n) using the association points. Refer to Figure 6 , traverse each association point in Device2 and Device3 in the Detect graph, calculate and match each association point with the association points of each Device in the existing historical attack event graph (Incident has a preset index). The process 2 (i.e., the 2nd association point) of Device2 in the Detect graph matches the process 2 (i.e., the 2nd association point) of Device2 in the historical attack event graph. Therefore, the two can be merged. It can be seen that the historical attack event graph finally associated by the Detect graph is Incident1 (attack event 1), and the remaining operation is simply the operation of merging the graph according to the association points.

[0043] As above, after merging, the association result is as Figure 7 shown. In this way, multi-institutional association can be completed. Similar to dynamic programming, each planning only needs to calculate and match with the existing historical attack event graph according to the association points, and the theoretically associated hosts have no upper limit.

[0044] In an alternative embodiment of the present invention, multi-institutional association is performed on the association detection graph with association point information and the historical attack event graph according to the time dimension, which specifically includes the following steps: If the difference between the time stamp of the target node in the association detection graph with association point information and the time stamp of the target node in the historical attack event graph is less than the preset time threshold, then the target node in the association detection graph with association point information is merged with the target node in the historical attack event graph, thereby obtaining the current attack event graph.

[0045] Specifically, in addition to performing multi-machine association according to the granularity of association points, the association detection graph and historical attack event graph with association point information reported within a certain time range are also an attack event in theory and need to be associated. The specific implementation is also very simple. Each Device node (i.e., the parent node) retains the current attack time span involved. When searching for associated historical attack events, if the association cannot be hit according to the granularity of association points, it is sufficient to find those with a time span difference (i.e., the time span between the timestamp of the target node in the association detection graph with association point information and the timestamp of the target node in the historical attack event graph) less than the preset time threshold.

[0046] The time dimension aggregation uses configuration parameters to control the time span size (supporting accuracy to milliseconds, default 2 days) to prevent overly large attack events that may occur due to an overly large span, and it is necessary to consider the data transmission cost and the front-end display efficiency.

[0047] In an alternative embodiment of the present invention, multi-machine association is performed on the association detection graph and historical attack event graph with association point information according to the granularity of association points, or multi-machine association is performed on the association detection graph and historical attack event graph with association point information according to the time dimension. Specifically, the following steps are included: (1) First, perform multi-machine association on the association detection graph and historical attack event graph with association point information according to the granularity of association points; (2) If the association fails, then perform multi-machine association on the association detection graph and historical attack event graph with association point information according to the time dimension to obtain the current attack event graph.

[0048] In an alternative embodiment of the present invention, the current attack event graph is displayed on the front end, specifically including the following steps: Display the current attack event graph on the front end according to the front-end requirements.

[0049] Specifically, the above front-end requirements can specifically be front-end display requirements, and the front-end display requirements can specifically be requirements related to display, such as the display style, specific data to be displayed, etc.

[0050] In an alternative embodiment of the present invention, the method further includes the following steps: Use the updated attack event graph to update the historical attack event graph for subsequent multi-machine alarm log association based on the updated historical attack event graph.

[0051] The method of the present invention can significantly optimize resource occupancy (because memory graphs are used for association) and efficiently and real-time associate different alarms to generate attack events (i.e., Incidents) across multiple machines (i.e., Devices). At the same time, it supports automatic and manual flexible expansion of association scenarios and data, and efficiently and flexibly depicts the real multi-machine intrusion attack process.

[0052] In the method of the present invention, both alarms and logs can be converted into memory graphs. Different memory graphs contain different combinations of nodes and edges. The memory graph can accurately represent the access relationships within the current device and across devices. By constructing the memory graph, these combination relationships are saved and expansion is supported. At the same time, based on the memory graph, by designing a multi-machine association algorithm, multiple devices are accurately and efficiently associated, depicting the real attack path / attack relationship, and forming a multi-machine attack event. Alarms contain clear attack information, which may be missing. In addition to alarms, it is necessary to allow the supplementation of key attack information from other data sources (such as logs), including: automatic supplementation based on rules (i.e., the process of step S104) and manual addition and supplementation on the attack event graph (i.e., the process of step S110). The present invention designs an association mechanism and an automatic / manual association process based on the memory graph.

[0053] The present invention has the following characteristics: Self-developed memory graph: The memory graph is developed based on the Go language, compressing memory occupancy. The association calculation is all performed in memory, removing the dependence on storage and network, and being able to perform real-time and efficient association. At the same time, it can customize the memory graph for the business and support efficient association algorithms; Multi-machine association algorithm: The association between multiple machines combines business characteristics, changing the pairwise node association calculation between multiple machines from O(n^2) to dynamic programming based on association points O(n), which can greatly reduce the association calculation amount and complete multi-machine association in real time and efficiently; Automatic and manual expansion algorithms: The memory graph design naturally supports dynamic expansion; it allows automatically querying data according to rules and converting it into a memory graph and manually specifying data to be converted into a memory graph, expanding the node merging problem of the converted graph.

[0054] The key points of the present invention are: the composition and construction method of the memory graph; the multi-machine efficient association algorithm based on the memory graph; the automatic and manual expansion algorithms based on the memory graph; the entire multi-machine alarm log association system (i.e., Figure 2 the association engine in). The main advantage is that it can construct multi-machine attack events in real time, efficiently, and accurately based on alarms and logs, trace the attack path, and at the same time support flexible automatic and manual expansion.

[0055] Embodiment 2: The embodiment of the present invention also provides a multi-machine alarm log association device, which is mainly used to execute the multi-machine alarm log association method provided in Embodiment 1 of the present invention. The following specifically introduces the multi-machine alarm log association device provided by the embodiment of the present invention.

[0056] Figure 8 is a schematic diagram of a multi-machine alarm log association device according to an embodiment of the present invention, as Figure 8As shown in the figure, the device mainly includes: a first conversion unit 10, a second conversion unit 20, a merging unit 30, a first multi - mechanism association unit 40, and a second multi - mechanism association unit 50, where: The first conversion unit is used to convert real - time alarms and / or real - time logs into a first memory graph, and calculate the association points of the first memory graph according to a preset association point rule, obtaining a first memory graph with association point information. Among them, the first memory graph includes: multiple nodes and edges. The nodes include: parent nodes and child nodes. The edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes. The edges are used to represent the access relationship or parent - child relationship between nodes, and the nodes are used to represent entity elements. The association points are at least one of the child nodes; The second conversion unit is used to convert the auxiliary information stored in the third party into a second memory graph, and calculate the association points of the second memory graph according to a preset association point rule, obtaining a second memory graph with association point information; The merging unit is used to merge the first memory graph with association point information and the second memory graph with association point information according to the node granularity, obtaining an association detection graph with association point information; The first multi - mechanism association unit is used to perform multi - mechanism association on the association detection graph with association point information and the historical attack event graph according to the association point granularity, or perform multi - mechanism association on the association detection graph with association point information and the historical attack event graph according to the time dimension, obtaining the current attack event graph; The second multi - mechanism association unit is used to display the current attack event graph on the front - end, and perform multi - mechanism association on the third memory graph with association point information triggered manually on the front - end and the current attack event graph, obtaining an updated attack event graph.

[0057] In an embodiment of the present invention, a multi-machine alarm log association device is provided, including: converting real-time alarms and / or real-time logs into a first memory graph, and calculating the association points of the first memory graph according to a preset association point rule to obtain a first memory graph with association point information, where the first memory graph includes: multiple nodes and edges, the nodes include: parent nodes and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the association points are at least one of the child nodes; converting auxiliary information stored in a third party into a second memory graph, and calculating the association points of the second memory graph according to a preset association point rule to obtain a second memory graph with association point information; merging the first memory graph with association point information and the second memory graph with association point information according to node granularity to obtain an association detection graph with association point information; performing multi-machine association on the association detection graph with association point information and a historical attack event graph according to association point granularity, or performing multi-machine association on the association detection graph with association point information and a historical attack event graph according to a time dimension to obtain a current attack event graph; displaying the current attack event graph on the front end, and performing multi-machine association on the third memory graph with association point information manually triggered and extended on the front end and the current attack event graph to obtain an updated attack event graph. Through the above description, it can be seen that in the multi-machine alarm log association device of the present invention, memory graphs are used for association. The edges of the memory graph include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, and the nodes are used to represent entity elements, so that multi-machine (between nodes of different entity elements) association can be realized. The association points of the memory graph are at least one of the child nodes, that is, multiple association points are allowed to be set, greatly increasing the dimension of association. The use of the memory graph compresses the memory occupancy, and the merging and multi-machine association are all carried out in the memory, removing the storage and network dependencies, and being able to perform real-time and efficient association; at the same time, the memory graph can be customized according to the business, supporting an efficient multi-machine association algorithm; when performing multi-machine association, there is no need to perform pairwise association according to node granularity, but to perform association and merging according to association point granularity, greatly reducing the association calculation amount; in addition, the first memory graph with association point information obtained in real time and the second memory graph with association point information converted from auxiliary information stored in a third party can be automatically merged and extended, and it also supports the multi-machine association of the third memory graph with association point information manually triggered and extended on the front end and the current attack event graph, with good scalability, alleviating the technical problems of large resource occupancy, inability to support multi-machine association, and poor scalability in traditional single-machine alarm log association methods.

[0058] Optionally, the entity elements include: device entities, IP address entities, process entities, file path entities, and the device entities include: hosts, K8s, containers, DNSs, and IPs.

[0059] Optionally, the first multi-machine association unit is further configured to: if the target association point in the association detection graph with association point information is the same as the target association point in the historical attack event graph, merge the target association point in the association detection graph with association point information and the target association point in the historical attack event graph, so as to obtain the current attack event graph.

[0060] Optionally, the first multi-machine association unit is further configured to: if the difference between the time stamp of the target node in the association detection graph with association point information and the time stamp of the target node in the historical attack event graph is less than a preset time threshold, merge the target node in the association detection graph with association point information and the target node in the historical attack event graph, so as to obtain the current attack event graph.

[0061] Optionally, the first multi-machine association unit is further configured to: first perform multi-machine association on the association detection graph with association point information and the historical attack event graph according to the association point granularity; if the association fails, then perform multi-machine association on the association detection graph with association point information and the historical attack event graph according to the time dimension, so as to obtain the current attack event graph.

[0062] Optionally, the second multi-machine association unit is further configured to: display the current attack event graph on the front end according to the front-end requirements.

[0063] Optionally, the device is further configured to: update the historical attack event graph by using the updated attack event graph, so as to perform subsequent multi-machine alarm log association according to the updated historical attack event graph.

[0064] The device provided by the embodiment of the present invention has the same implementation principle and the same technical effects as those of the foregoing method embodiment. For the sake of brief description, for the parts not mentioned in the device embodiment, reference may be made to the corresponding content in the foregoing method embodiment.

[0065] As Figure 9 shown, an electronic device 600 provided by an embodiment of the present application includes: a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device runs, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to execute the steps of the multi-machine alarm log association method as described above.

[0066] Specifically, the foregoing memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited herein. When the processor 601 runs the computer program stored in the memory 602, it can execute the multi-machine alarm log association method as described above.

[0067] The processor 601 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 601 or the instructions in the form of software. The above-mentioned processor 601 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and combines its hardware to complete the steps of the above method.

[0068] Corresponding to the above multi-machine alarm log association method, an embodiment of the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores machine-executable instructions. When the computer-executable instructions are called and run by a processor, the computer-executable instructions cause the processor to run the steps of the above multi-machine alarm log association method.

[0069] The multi-machine alarm log association device provided by the embodiments of the present application may be specific hardware on the device or software or firmware installed on the device, etc. For the device provided by the embodiments of the present application, the implementation principle and the technical effects produced are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can all refer to the corresponding processes in the above method embodiments, and will not be repeated here.

[0070] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0071] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0072] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0073] In addition, the various functional units in the embodiments provided in the present application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0074] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the multi-machine alarm log association method described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM for short), random access memories (RAM for short), magnetic disks, or optical discs.

[0075] It should be noted that: similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0076] Finally, it should be noted that: the above-mentioned embodiments are only specific implementation manners of this application, used to illustrate the technical solution of this application, rather than limiting it. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed in this application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of this application. All should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A multi-machine alarm log association method, characterized in that Including: Converting real-time alerts and / or real-time logs into a first memory graph, and calculating the correlation points of the first memory graph according to a preset correlation point rule to obtain a first memory graph with correlation point information, where the first memory graph includes: multiple nodes and edges, the nodes include: parent nodes and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the correlation points are at least one of the child nodes; Converting the auxiliary information stored in the third party into a second memory graph, and calculating the correlation points of the second memory graph according to the preset correlation point rule to obtain a second memory graph with correlation point information; Merging the first memory graph with correlation point information and the second memory graph with correlation point information according to node granularity to obtain a correlation detection graph with correlation point information; Performing multi-institution correlation on the correlation detection graph with correlation point information and the historical attack event graph according to correlation point granularity, or performing multi-institution correlation on the correlation detection graph with correlation point information and the historical attack event graph according to the time dimension to obtain the current attack event graph; Displaying the current attack event graph on the front end, and performing multi-institution correlation on the third memory graph with correlation point information triggered manually on the front end and the current attack event graph to obtain an updated attack event graph.

2. The method according to claim 1, wherein The entity elements include: device entities, IP address entities, process entities, file path entities, and the device entities include: hosts, K8s, containers, DNSs, and IPs.

3. The method according to claim 1, wherein Performing multi-institution correlation on the correlation detection graph with correlation point information and the historical attack event graph according to correlation point granularity includes: If the target correlation points in the correlation detection graph with correlation point information are the same as the target correlation points in the historical attack event graph, then merging the target correlation points in the correlation detection graph with correlation point information and the target correlation points in the historical attack event graph to obtain the current attack event graph.

4. The method according to claim 1, wherein Performing multi-institution correlation on the correlation detection graph with correlation point information and the historical attack event graph according to the time dimension includes: If the difference between the time stamps of the target nodes in the correlation detection graph with correlation point information and the time stamps of the target nodes in the historical attack event graph is less than a preset time threshold, then merging the target nodes in the correlation detection graph with correlation point information and the target nodes in the historical attack event graph to obtain the current attack event graph.

5. The method according to claim 1, characterized in that, Performing multi-institution correlation on the correlation detection graph with correlation point information and the historical attack event graph according to correlation point granularity, or performing multi-institution correlation on the correlation detection graph with correlation point information and the historical attack event graph according to the time dimension includes: First performing multi-institution correlation on the correlation detection graph with correlation point information and the historical attack event graph according to correlation point granularity; If no association can be made, then multi-machine association is performed on the association detection graph with association point information and the historical attack event graph according to the time dimension to obtain the current attack event graph.

6. The method according to claim 1, wherein Display the current attack event graph on the front end, including: Display the current attack event graph on the front end according to the front-end requirements.

7. The method according to claim 1, wherein The method further includes: Use the updated attack event graph to update the historical attack event graph, so as to perform subsequent multi-machine alarm log association based on the updated historical attack event graph.

8. A multi-machine alarm log correlation device, characterized in that Including: A first conversion unit, configured to convert real-time alarms and / or real-time logs into a first memory graph, and calculate the association points of the first memory graph according to a preset association point rule, to obtain a first memory graph with association point information, where the first memory graph includes: multiple nodes and edges, the nodes include: parent nodes and child nodes, the edges include: edges between different child nodes within the same parent node, edges between different parent nodes, and edges between different child nodes of different parent nodes, the edges are used to represent the access relationship or parent-child relationship between nodes, the nodes are used to represent entity elements, and the association points are at least one of the child nodes; A second conversion unit, configured to convert the auxiliary information stored in the third party into a second memory graph, and calculate the association points of the second memory graph according to the preset association point rule, to obtain a second memory graph with association point information; A merging unit, configured to merge the first memory graph with association point information and the second memory graph with association point information according to the node granularity, to obtain an association detection graph with association point information; A first multi-machine association unit, configured to perform multi-machine association on the association detection graph with association point information and the historical attack event graph according to the association point granularity, or perform multi-machine association on the association detection graph with association point information and the historical attack event graph according to the time dimension, to obtain the current attack event graph; A second multi-machine association unit, configured to display the current attack event graph on the front end, and perform multi-machine association on the current attack event graph and a third memory graph with association point information triggered and extended manually on the front end, to obtain an updated attack event graph.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 above are implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and run by the processor, the machine-executable instructions cause the processor to run the method described in any one of claims 1 to 7 above.

Images