An alarm noise reduction method based on long-term and short-term knowledge graphs and fine-grained edge aggregation

Abstract

This invention relates to an alarm denoising method based on long-term and short-term knowledge graphs and fine-grained edge aggregation, belonging to the field of network security technology. The method includes: collecting alarm logs from security devices and extracting alarm data from the logs; combining the standardized data with a predefined knowledge graph to extract entities and relationships between them, forming alarm logs; constructing a short-term dynamic alarm graph based on short-term time windows; performing fine-grained aggregation on the short-term dynamic alarm graph, calculating similarity between alarm entities and their relationships, and clustering alarm data of the same type; dynamically fusing the short-term alarm graph using a time decay factor based on long-term time windows to capture real-time and historical alarm patterns; and outputting the fused long-term and short-term dynamic alarm graph. This invention establishes alarm graphs based on long-term and short-term knowledge graphs while performing fine-grained clustering analysis, achieving the goal of complex alarm denoising.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to an alarm noise reduction method based on long-term short-term knowledge graphs and fine-grained edge aggregation. Background Technology

[0002] As cyberattack methods continue to evolve and become more complex, the volume of security alerts is also increasing dramatically. Various security devices generate a large amount of alert information to alert on security incidents, abnormal behavior, or potential security threats. However, the sheer volume of alert data has become a major problem for security operations centers. Security devices generate tens of thousands of alert logs daily, and the vast majority of these are likely false alarms. Therefore, the main purpose of intelligent noise reduction for alert information is to compress redundant alerts from this massive amount of data, providing a foundation for identifying truly effective alerts, while simultaneously improving the analytical efficiency of security operations personnel.

[0003] Traditional noise reduction methods mainly rely on static rule filtering or simple aggregation, which are limited to single-dimensional matching. They are difficult to effectively compress redundant alarms and cannot capture the complex semantic and behavioral relationships between alarms, resulting in low analysis efficiency. In recent years, the application of knowledge graph technology in the field of cybersecurity has provided a new approach to solving this problem. By modeling entities (such as processes and files) and relationships (such as creation and access), it is possible to reveal attack behavior patterns in a structured way, while achieving the purpose of aggregation and noise reduction.

[0004] However, statically modeling alert information using existing knowledge graph solutions ignores the dynamic evolution of alert behavior over time, making it difficult to adapt to the detection needs of both short-term real-time threats and long-term covert attacks. Furthermore, existing knowledge graph-based alert aggregation solutions typically focus on coarse-grained entity aggregation, lacking fine-grained analysis of similar alerts, thus limiting noise reduction effectiveness.

[0005] Current alarm deduplication techniques mainly include three methods: alarm deduplication, alarm aggregation, and alarm rule-based methods, but all have limitations. Alarm deduplication identifies and deletes identical or similar alarms, but its deduplication effect is limited and it cannot handle redundant alarms with similar semantics. Alarm aggregation uses related fields as key values ​​to cluster and merge alarms within a fixed time window, which may filter out key details or disrupt attack paths, affecting source tracing analysis. Alarm rule-based methods are based on pre-defined rules from historical experience, which are efficient but have high maintenance costs. These methods mostly use static modeling, ignoring the correlation between alarms and their dynamic evolution over time, making it difficult to meet the needs of complex scenarios.

[0006] In addition, existing knowledge graph noise reduction schemes are mostly based on coarse-grained aggregation of entities (such as processes and files), ignoring further fine-grained aggregation. The existing methods result in a huge number of entities and edges in the aggregated knowledge graph, which is not conducive to analysis and judgment by security operations personnel. Summary of the Invention

[0007] The purpose of this invention is to at least address one of the aforementioned technical deficiencies.

[0008] Therefore, the purpose of this invention is to propose an alarm noise reduction method based on long-term and short-term knowledge graphs and fine-grained edge aggregation, which can be based on long-term and short-term alarm graphs and perform fine-grained cluster analysis to achieve the purpose of complex alarm noise reduction.

[0009] To achieve the above objectives, embodiments of the present invention provide an alarm noise reduction method based on long short-term knowledge graphs and fine-grained edge aggregation, comprising the following steps:

[0010] S1, collect alarm logs from security devices, extract alarm data from the alarm logs, and standardize the alarm data;

[0011] S2, combines the standardized data with a predefined knowledge graph to extract entities and relationships between entities from the alarm data, forming the alarm log (alert_log);

[0012] S3 constructs a short-term dynamic alarm graph based on a short-term time window, displaying the correlation and temporal evolution of modeled alarms, including:

[0013] S31, Divide the short-term time window, including: defining the short-term time window Δt s Extracting time range [t-Δt] s The alarm subset D within t] s ;

[0014] S32, Construct a short-term dynamic alarm graph, including: based on the alarm subset D s Generate short-term dynamic alarm map G s =(V s E s W s A s ), where V s Let E be a set of nodes. s Let A be the set of edges. s W is a set of alarm attributes. s The edge weights; the short-term dynamic alarm graph G s Used to reflect real-time alarm correlations while performing complex structural noise reduction;

[0015] S4. Based on the short-term dynamic alarm map, perform fine-grained aggregation, calculate the similarity between alarm entities and the relationships between alarm entities, and cluster alarm data of the same type according to the similarity calculation results to compress the number of alarms while retaining behavioral differences.

[0016] S5, based on a long-term time window, dynamically fuses short-term alarm graphs using a time decay factor to capture real-time and historical alarm patterns, including:

[0017] S51, Divide the long-term window, including: defining the alarm subset D corresponding to the long-term window. l ;

[0018] S52, Constructing a Long-Term Knowledge Graph: First, from the initial D... l The alarm data within the database is used to construct the initial long-term graph G. l Then, the historical alarm weights are dynamically adjusted using a time decay factor, and the short-term dynamic alarm map G is periodically updated. s Integrating long-term map G l This enables incremental updates, updating nodes and edges and adjusting weights, thus achieving both a long-term knowledge graph and a short-term dynamic alert graph.

[0019] Dynamic fusion of spectra generates a fused long-term and short-term dynamic alarm spectrum;

[0020] S6, output the fused long-term and short-term dynamic alarm map, which integrates the noise reduction results to support response and source tracing.

[0021] Furthermore, in S1, the data in the alarm log includes: alarm log ID, attack source IP, attack source MAC address, attack source port, target IP, target MAC address, target port, alarm time, alarm type, alarm product, alarm level, operation object type, operation object, alarm object type, alarm object, executed operation, attribute information, and alarm process tree.

[0022] Furthermore, in S2, the alarm entities include: processes, files, registry entries, host IPs, external IPs, and web URLs; the relationships between the alarm entities include: process creation, file-related operations, external connections, and web-related operations.

[0023] Furthermore, in S2, the subject and object of each alarm data are extracted based on a predefined knowledge graph to construct the entities of the knowledge graph. The alarm type of each alarm data is extracted as the relationship between alarm entities, and the attribute information of each alarm entity is obtained. The alarm log (alert_log) is...<src,dst,relation,attribute> ,

[0024] Where src is the subject, dst is the object, relation is the relationship between alarm entities, and attribute is the attribute information of the alarm entities.

[0025] Furthermore, in S3, when aggregating the alarm graph, repeated alarms are aggregated according to time windows, retaining the earliest and latest times of alarm occurrence and storing them in attribute A. s In the process, the frequency information of repeated alarms is stored in the weight W. s This is used to achieve initial noise reduction.

[0026] Furthermore, in S3, the edge weights of the short-term dynamic alarm graph are:

[0027]

[0028] Where p is the frequency, representing the number of alarm events between entity i and entity j within the time window; the larger p is, the more frequent the alarms for this pair of entities occur in the short term, and p is used to reflect the intensity of the behavior pattern.

[0029] β is a smoothing parameter, which mathematically acts as a Laplace smoothing mechanism to adjust the model's sensitivity to frequency.

[0030] γ is a time coefficient that controls the rate at which the time interval decays the edge weights. γ determines the degree of influence of the time difference on the association strength.

[0031] Adjust the semantic information and behavior of alarm data by controlling p, β, and γ;

[0032] i and j represent entity i and entity j in the short-term dynamic alarm graph, respectively. This represents the edge weight between entity i and entity j in the current short-term dynamic alarm graph. All edge weights constitute the current short-term dynamic alarm graph G. s The set of edge weights Ws;

[0033] t max The latest time the alarm occurs, t min The earliest time the alarm occurs;

[0034] Short-term dynamic alarm graph G s It reflects real-time alarm correlations and performs complex structural noise reduction, making it suitable for rapid threat response.

[0035] Furthermore, in S4, the method for calculating the similarity between alarm entities and the relationships between alarm entities is as follows:

[0036] s ij =α·cos(a i ,a j )+(1-α)·L(b i ,b j )

[0037] Among them, s ijLet cos be the similarity between alarm i and alarm j after short-term aggregation, L be the cosine similarity, and a be the edit distance similarity. i ,a j b is a semantic feature vector. i ,b j Let be the set of behavioral features, and α∈[0,1] be the weight coefficients.

[0038] Furthermore, in S4, the DBSCAN algorithm is used to cluster alarm data of the same type based on the similarity calculation results, including:

[0039]

[0040] s ij ≥∈,

[0041] |N ∈ (a i )|≥min_samples

[0042] Among them, C k Let N be the k-th cluster, ∈ be the neighborhood radius, and N be the number of clusters. ∈ (a i (For alarm a) i The set of neighborhood points, min_samples is the minimum number of alarm samples for each cluster, a i For the i-th alarm, a j For the j-th alarm; Output: Aggregated alarm set A' = C1, C2, ..., C K Each cluster C k This includes representative alerts and their context.

[0043] Furthermore, in S5, the time decay factor is designed as follows:

[0044]

[0045] Where λ is the time decay factor, used to assign time decay weights to historical alarms, φ(t) ij ) is the decay function, and These are the latest alarm time and the historical alarm time for the current entity i and entity j, respectively.

[0046] Furthermore, in S5, the short-term graph nodes are checked; if the short-term graph node v i Existing in the long-term graph node set V l In the middle, the node attributes and statistical attributes are updated to capture the real-time pattern of historical alarm entities triggering alarms in the current state. Simultaneously, the node attributes retain both past and updated attribute values ​​to preserve historical alarm information. If the node v in the short-term graph...i The set of nodes V that does not exist in the long-term graph l If the condition is met, then add the node to V. l middle;

[0047] The edge weights of the long-term and short-term graphs are dynamically fused using a time decay factor λ as follows:

[0048]

[0049] Among them, w' ij For the weights after fusion, The item is used to amplify the short-term dynamic alarm map G. s The influence of medium-to-high weighted edges, This item is used to protect historically associated edges. Durability, t ij This indicates the current time of the alarm formed by entities i and j. This represents the edge between entity i and entity j in the short-term dynamic alarm graph. The edge represents the edge between entity i and entity j in the current long-term alarm graph, and l indicates that it is a long-term alarm graph.

[0050] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0051] The alarm noise reduction method based on knowledge graphs and fine-grained edge aggregation based on long and short time windows provided by this invention establishes an alarm graph based on long and short time windows for alarm data, and further performs fine-grained cluster analysis on this basis to achieve the purpose of complex alarm noise reduction, providing a foundation for the detection of short-term threats and long-term covert attacks in the later stage.

[0052] 1. The alarm graph constructed by combining alarm data with knowledge graph in this invention can not only achieve the effect of noise reduction, but also intuitively visualize the relationship between alarms, providing support for the judgment of security operations personnel.

[0053] 2. The design scheme of the long and short alarm graph proposed in this invention can capture real-time and historical patterns by utilizing long and short time windows. While effectively reducing noise, it retains key correlation information through a fusion mechanism, and the incremental update method also reduces the computational cost.

[0054] 3. This invention proposes fine-grained aggregation focusing on edge relationships, which can further aggregate alarm data based on alarm graphs to achieve fine-grained alarm noise reduction while retaining key behavioral features and avoiding information loss.

[0055] 4. This invention supports attack path analysis and root cause localization while achieving efficient noise reduction, making it suitable for security operations, IT maintenance, and cloud-native environments, demonstrating significant application potential.

[0056] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0057] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0058] Figure 1 This is a flowchart of an alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of an alarm graph paradigm according to an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram illustrating the construction of an alarm map according to an embodiment of the present invention;

[0061] Figure 4 This is a diagram illustrating the overall architecture of the alarm noise reduction method based on long-term and short-term knowledge graphs and fine-grained edge aggregation according to an embodiment of the present invention. Detailed Implementation

[0062] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0063] The alarm noise reduction method proposed in this invention is based on long and short term knowledge graphs and fine-grained edge aggregation. This method constructs a dynamic knowledge graph to explicitly model the causal and dependency relationships of alarms, captures the dynamic evolution characteristics of alarms by combining long and short term time windows, and uses multi-dimensional fine-grained clustering to reduce redundancy while retaining key context and related paths.

[0064] The core of this invention lies in constructing a dynamically evolving long-term and short-term alarm graph, maintained through a continuous, incremental fusion mechanism. The short-term graph captures real-time alarm patterns, while the long-term graph dynamically fuses the short-term graph using a time decay factor to construct a comprehensive historical behavior panorama. This invention achieves a dynamic time-series graph model that simultaneously considers real-time and historical alarm patterns, realizing both noise reduction and correlation discovery. Building upon the graph, this invention further clusters edges with similar semantics and behavioral patterns to achieve alarm noise reduction; this is a finer-grained noise reduction method that focuses on the similarity of behavioral patterns.

[0065] like Figure 1 and Figure 4 As shown, the alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation in this embodiment of the invention includes the following steps:

[0066] S1 collects alarm logs from security devices, extracts alarm data from the alarm logs, and performs standardized processing on the alarm data.

[0067] Specifically, collect alarm logs from security devices to provide data input for subsequent modeling. The log content needs to include multiple fields such as timestamp, entity (IP, host), and behavior description, and the alarm data should be standardized into JSON format.

[0068] In this invention, the data in the alarm log includes: alarm log ID, attack source IP, attack source MAC address, attack source port, target IP, target MAC address, target port, alarm time, alarm type, alarm product, alarm level, operation object type, operation object, alarm object type, alarm object, executed operation, attribute information, and alarm process tree.

[0069] The fields to be collected and their meanings are explained in Table 1 below:

[0070] Table 1

[0071]

[0072]

[0073] S2 combines the standardized data with a predefined knowledge graph to extract alarm entities and the relationships between entities from the alarm data, forming the alarm log (alert_log).

[0074] The relationships between alert entities are represented as edge relationships in the knowledge graph. The alert log (alert_log) is a triple, including: the alert entity, the edge relationship (i.e., the relationship between alert entities), and the attribute information of the alert entity.

[0075] Based on a predefined knowledge graph, the subject and object of each alarm data are extracted to construct the entities of the knowledge graph. The alarm type of each alarm data is extracted as the relationship between alarm entities, and the attribute information of each alarm entity is obtained. Among these, the alarm log (alert_log) is...<src,dst,relation,attribute> .

[0076] Where src is the subject, dst is the object, relation is the relationship between alarm entities, and attribute is the attribute information of the alarm entities.

[0077] This invention is based on the principles of knowledge graphs and combines them with data from conventional host products, defining, as follows: Figure 2 The alarm graph paradigm shown. Figure 2 It shows different types of entities and the relationships between them.

[0078] In this invention, alarm entities include: processes, files, registry entries, host IPs, external IPs, and web URLs, etc.

[0079] Alarm edges (i.e., relationships between entities) include: process creation (process operations), file operations (writing dangerous files / reading sensitive files, creating binary files, etc.), external connections and web operations (illegal requests, SQL injection, etc.).

[0080] (1) Entity recognition: Based on predefined host alarm data, the subject and object of each alarm data are extracted to construct the entities of the knowledge graph. For example, taking the creation process as an example, the parent process A and the child process B are both alarm entities, where the parent process is the subject (src) and the child process is the object (dst).

[0081] (2) Relationship extraction: The alarm type of each alarm data, that is, the edge relationship between entities. For example, the creation of a process is the edge relationship between process A and process B.

[0082] (3) Attributes: The attribute information contained varies depending on the entity type. For example, if the entity is a process, the attribute information may include process tree, process hash, etc.; if the entity is a file, the attribute information may include file hash, etc.

[0083] (4) Define the extracted alarm log: alert_log =<src,dst,relation,attribute> .

[0084] (5) The extracted alarm logs (alert_log) are used to form an alarm set D.

[0085] S3 constructs a short-term dynamic alarm graph based on a short-term time window, displaying the correlation and temporal evolution of modeled alarms.

[0086] In this step, short-term knowledge graph modeling is performed based on the extracted triples. That is, a dynamic alarm graph is constructed based on short-term time windows to explicitly model the correlation and temporal evolution of alarms.

[0087] The alert_log extracted in step S2 =<src,dst,relation,attribute> This forms a graph G(V,E,A). Here, V is the set of entities, E is the set of edges, and A is the set of alarm attributes.

[0088] This alarm graph represents an overview of alarms occurring on a specific host over a given period. It effectively aggregates alarm information for noise reduction and reflects the overall alarm structure on that host, allowing for further root cause analysis and threat assessment. Creating an alarm graph from alarm logs avoids the limitations of conventional alarm noise reduction methods, which only offer single-point analysis and neglect structural analysis.

[0089] The process of constructing a short-term alarm graph is explained below. Based on session-level or short time windows (minutes to hours, such as 5 minutes), high-frequency alarms are quickly aggregated to generate a real-time short-term dynamic alarm graph.

[0090] S31, Divide the short-term time window, including: defining the short-term time window Δt s Extracting time range [t-Δt] s The alarm subset D within t] s Wherein, Δt s For example, 5 minutes.

[0091] S32, Construct a short-term dynamic alarm graph, including: based on alarm subset D s Generate short-term dynamic alarm map G s =(V s E s W s A s ), where V s Let E be a set of nodes. s Let A be the set of edges. s W is a set of alarm attributes. s This refers to edge weights. The purpose of setting weights is to assist security operations personnel in subsequent analysis and assessment.

[0092] When aggregating alarm graphs, duplicate alarms are aggregated according to time windows, retaining the earliest and latest times of alarm occurrence and storing them in attribute A. s In the process, the frequency information of repeated alarms is stored in the weight W. s This is done to achieve the initial goal of noise reduction.

[0093] The short-term alarm graph construction retains the subject-object information and key attribute information of the alarms, achieving lossless compression. Time and frequency smoothing parameters are applied here to ensure that short-term, high-frequency alarms have higher edge weights. The edge weights of the short-term dynamic alarm graph designed in this invention are:

[0094]

[0095] Where p represents frequency, indicating the number of alarm events occurring between entity i and entity j within the time window. A larger p indicates more frequent alarms for these entities in the short term. It reflects the strength of the behavioral pattern.

[0096] β is a smoothing parameter, which mathematically functions as a Laplace smoother and is used to adjust the model's sensitivity to frequency.

[0097] γ is a time coefficient that controls the rate at which the time interval decays the edge weights. It determines how much the time difference affects the association strength.

[0098] The semantic information and behavior of alarm data can be adjusted by controlling p, β, and γ.

[0099] i and j represent entity i and entity j in the short-term dynamic alarm graph, respectively. This represents the edge weight between entity i and entity j in the current short-term dynamic alarm graph. All edge weights constitute the current short-term dynamic alarm graph G. s The set of edge weights Ws.

[0100] t max The latest time the alarm occurs, t min The earliest time the alarm occurs.

[0101] Short-term dynamic alarm graph G s It reflects real-time alarm correlations and performs complex structural noise reduction, making it suitable for rapid threat response.

[0102] Figure 3 An example of alarm graph construction is given. For example... Figure 3 As shown, different alarm types are recorded, along with the earliest occurrence time, latest occurrence time, and frequency of each alarm type.

[0103] S4 performs fine-grained aggregation based on short-term dynamic alarm graphs, calculates the similarity between alarm entities and the relationships between alarm entities, and clusters alarm data of the same type according to the similarity calculation results, compressing the number of alarms while retaining behavioral differences.

[0104] In this step, fine-grained edge aggregation is adopted. For similar alarm information, similarity calculation combined with clustering algorithm is used for aggregation to achieve fine-grained noise reduction.

[0105] (1) Similarity calculation based on entity and edge relationship

[0106] Semantic similarity utilizes a vector model to generate vector representations of alarm triples, calculates cosine similarity, and captures the semantic consistency of the descriptive text. Examples of vector models include shibing624 / text2vec-base-chinese.

[0107] Behavioral similarity: For process tree structure, file type, or IP pattern, similarity is calculated using edit distance to assess the consistency of behavioral patterns. Therefore, this invention proposes the following similarity calculation method:

[0108] s ij =α·cos(a i ,a j )+(1-α)·L(b i ,b j (2)

[0109] Among them, s ij Let cos be the similarity between alarm i and alarm j after short-term aggregation, L be the cosine similarity, and a be the edit distance similarity. i ,a j b is a semantic feature vector. i ,b j Let be the set of behavioral features, and α∈[0,1] be the weight coefficients.

[0110] (2) Clustering

[0111] Based on the semantic similarity of edge relationships, alarms of the same type are clustered. For example, all "creation process" alarms are clustered.

[0112] In this invention, the DBSCAN clustering algorithm is used, which supports the identification of clusters of arbitrary shapes and the filtering of noise points, ensuring that the clustering results are accurate and robust. The aggregated triplet set significantly reduces the number of alarms while retaining key behavioral differences.

[0113]

[0114] Among them, C k Let N be the k-th cluster, ∈ be the neighborhood radius, and N be the number of clusters. ∈ (a i (For alarm a) i The set of neighborhood points, min_samples is the minimum number of alarm samples for each cluster, a i For the i-th alarm, a j This is the j-th warning.

[0115] Output: Aggregated alarm set A' = C1, C2, ..., C K Each cluster C k Includes representative alarms and context.

[0116] (3) Update edge information: Select the representative alarm edge from each cluster from the edges that are retained after aggregation.

[0117] This invention focuses on fine-grained aggregation of edge relationships, enabling further aggregation based on the alarm graph to achieve fine-grained alarm noise reduction, providing a foundation for operational personnel's analysis. Fine-grained aggregation significantly improves the noise reduction rate while retaining key behavioral characteristics, avoiding information loss.

[0118] S5, based on a long-term time window, dynamically fuses short-term alarm maps using a time decay factor to capture real-time and historical alarm patterns.

[0119] This step involves constructing a long-term attack graph: based on a longer time window (days to weeks, such as 24 hours or 7 days), a panoramic view of historical behavior is built by incrementally fusing short-term attack graphs, capturing slow, covert attack patterns. The fusion mechanism here uses a time decay factor to dynamically adjust the weight of historical alerts and periodically merges short-term attack graphs G... s Integrating long-term map G l This enables incremental updates.

[0120] S51, Divide the long-term window, including: defining the alarm subset D corresponding to the long-term window. l .

[0121] Among them, the alarm subset D corresponding to the long-term window l For example, 24 hours or 7 days.

[0122] It should be noted that this is only applied when constructing the initial long-term map G. l The complete subset of alarms will only be needed occasionally, and the long-term graph will be updated periodically by fusion of short-term dynamic graphs.

[0123] S52, Constructing a Long-Term Knowledge Graph: First, from the initial D... l The alarm data within the database is used to construct the initial long-term graph G. l Then, the historical alarm weights are dynamically adjusted using a time decay factor, and the short-term dynamic alarm map G is periodically updated. s Integrating long-term map G l This enables incremental updates, updating nodes and edges and adjusting weights, and achieves dynamic fusion of long-term knowledge graph and short-term dynamic alarm graph to generate a fused long-term and short-term dynamic alarm graph.

[0124] Incremental fusion: Periodically (e.g., hourly) merging short-term map G s Integrating long-term map G l Update nodes and edges, and adjust weights. The goal of the long-term and short-term graph fusion mechanism is to dynamically fuse short-term and long-term graphs through time decay and incremental updates, ensuring that the graph takes into account both real-time performance and historical integrity.

[0125] (1) Time decay factor

[0126] Time dimension processing: Time is the core driving force of the model. The time decay mechanism enables the topology and weights of the graph to dynamically reflect the real-time nature and historical importance of alarm information.

[0127] The decay function is:

[0128]

[0129] Where λ is the time decay factor, used to assign time decay weights to historical alarms; typically, more recent alarms have a higher weight. φ(t) ij ) is the decay function, and These are the latest alarm time and the historical alarm time for the current entity i and entity j, respectively.

[0130] (2) Node fusion

[0131] Check the short-term graph nodes; if the short-term graph node v i Existing in the long-term graph node set V l In the middle, the node attributes and statistical attributes are updated to capture the real-time pattern of historical alarm entities triggering alarms in the current state. Simultaneously, the node attributes retain both past and updated attribute values ​​to preserve historical alarm information. If the short-term graph node v... i The set of nodes V that does not exist in the long-term graph l If the condition is met, then add the node to V. l middle.

[0132] Among them, v i ∈V s Node attributes include: timestamp, severity, etc. Statistical attributes of nodes include: alarm type distribution, frequency, etc.

[0133] The following will explain how to capture real-time and historical alarm patterns from two aspects: updating existing nodes and adding new nodes:

[0134] 1. Update existing nodes: If v i ∈V s Existing in the long-term graph node set V l This allows the system to capture the real-time patterns of historical entities triggering alarms by updating node attributes such as timestamps and severity. Simultaneously, the node's attributes retain both past and updated values, thus preserving historical alarm information.

[0135] 2. Add a new node: When a node in the short-term graph is not in the node set V of the long-term graph.l If the condition is met, then add the node to V. l This method is effective at capturing real-time alarm information.

[0136] (3) Edge integration

[0137] Edge fusion: Merging short-term edges E s and long-term edge E l Update the edge weights. The edge weights of the long-term and short-term graphs are dynamically fused using the time decay factor λ as follows:

[0138]

[0139] Among them, w' ij For the weights after fusion, The item is used to amplify the short-term dynamic alarm map G. s The influence of medium-to-high weighted edges, This item is used to protect historically associated edges. Durability, t ij This indicates the current time of the alarm formed by entities i and j. This represents the edge between entity i and entity j in the short-term dynamic alarm graph. The edge represents the edge between entity i and entity j in the current long-term alarm graph, and l indicates that it is a long-term alarm graph.

[0140] Specifically, the edge fusion method dynamically merges the edge weights of long-term and short-term graphs using a time decay factor. The following explanation focuses on the weight calculation for edges that need to be merged to illustrate how to capture real-time and historical alarm patterns. Newly added edges, like newly added nodes, are directly integrated into the long-term graph.

[0141] Item: Enlarged short-term spectrum G s The influence of medium- and high-weight edges, and the selection mechanism of the max function, ensure that newly occurring, high-weight alarm entities are associated with each other. This method can clearly show the real-time alarm chain.

[0142] Item: Safeguarding historical connections Its persistence ensures that historically formed association structures are not cleared when no new alarms occur, but rather decay slowly in a controlled manner.

[0143] In summary, firstly, the short-term alarm graph already includes the latest, denoised alarm correlations, therefore G s This ensures real-time performance, enabling the capture of any newly emerging alarms or entities (nodes / edges). Therefore, the aforementioned periodic incremental fusion mechanism ensures the long-term graph G... l Capable of rapidly reflecting short-term spectra Gs The latest changes captured in the process.

[0144] Long-term map G l By continuously incorporating new nodes and edges through the above fusion scheme, a longer-term historical alarm behavior view is constructed. Incremental fusion involves adding or updating the changed parts, G... l The main structure and historical information are fully preserved. This approach avoids the computational overhead and potential information loss caused by full reconstruction, smoothly integrating real-time "increments" into historical "stocks" and ensuring the continuity of historical alarm information.

[0145] The design scheme of the long-term and short-term alarm graph proposed in this invention can capture real-time and historical patterns using long-term and short-term time windows, effectively reduce noise, retain key correlation information through a fusion mechanism, and reduce computational costs through incremental updates.

[0146] S6 outputs the fused long-term and short-term dynamic alarm maps, which integrate the noise reduction results to support response and source tracing.

[0147] In this step, the fused long-term and short-term dynamic alarm maps are integrated into Neo4j for easier analysis. This step ultimately generates a denoised dynamic map, significantly reducing the number of alarms. Furthermore, the alarm map constructed using the long-term and short-term models adapts to complex scenarios while preserving correlations and behavioral differences.

[0148] In summary, the alarm denoising method based on long-term and short-term knowledge graphs and fine-grained edge aggregation of this invention effectively aggregates and denoises alarm information by constructing an alarm graph on the host over a period of time. This comprehensively reflects the alarm structure overview on the host, providing strong support for root cause analysis and threat assessment. Unlike traditional alarm denoising methods that are limited to single-point analysis, this invention introduces structured analysis through knowledge graphs, innovatively focusing on fine-grained aggregation of edge relationships. Combined with long-term and short-term time window design, it captures real-time and historical behavior patterns, significantly improving the denoising rate while retaining key correlation information, reducing computational costs, and providing security operations personnel with an efficient basis for analysis and judgment.

[0149] The output of this invention is a highly compressed, redundancy-free, and key-relevance-preserving alarm knowledge graph, applicable to network security operations scenarios, specifically handling security alarms. The combination of long-term and short-term models with fine-grained edge aggregation effectively reduces alarm noise, providing a comprehensive view of host alarms for easier analysis by security operations personnel. It is also suitable for detecting slow, covert attack patterns, a unique challenge in the security field and less common in routine IT failures. This invention enables complex alarm noise reduction, facilitating analysis by security personnel and providing support for subsequent threat detection.

[0150] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0151] It will be readily understood by those skilled in the art that this invention includes any combination of the inventive description and specific embodiments outlined in the foregoing specification, as well as the various parts shown in the accompanying drawings. Due to space limitations and for the sake of brevity, not all of these combinations have been described in detail. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0152] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An alarm noise reduction method based on long short-term knowledge graphs and fine-grained edge aggregation, characterized in that, Includes the following steps: S1, collect alarm logs from security devices, extract alarm data from the alarm logs, and standardize the alarm data; S2, combines the standardized data with a predefined knowledge graph to extract entities and relationships between entities from the alarm data, forming the alarm log (alert_log); S3 constructs a short-term dynamic alarm graph based on a short-term time window, displaying the correlation and temporal evolution of modeled alarms, including: S31, Divide the short-term time window, including: defining the short-term time window Δt s Extracting time range [t-Δt] s The alarm subset D within t] s ; S32, Construct a short-term dynamic alarm graph, including: based on the alarm subset D s Generate short-term dynamic alarm map G s =(V s E s W s A s ), where V s Let E be a set of nodes. s Let A be the set of edges. s W is a set of alarm attributes. s The edge weights; the short-term dynamic alarm graph G s Used to reflect real-time alarm correlations while performing complex structural noise reduction; S4. Based on the short-term dynamic alarm map, perform fine-grained aggregation, calculate the similarity between alarm entities and the relationships between alarm entities, and cluster alarm data of the same type according to the similarity calculation results to compress the number of alarms while retaining behavioral differences. S5, based on a long-term time window, dynamically fuses short-term alarm graphs using a time decay factor to capture real-time and historical alarm patterns, including: S51, Divide the long-term window, including: defining the alarm subset D corresponding to the long-term window. l ; S52, Constructing a Long-Term Knowledge Graph: First, from the initial D... l The alarm data within the database is used to construct the initial long-term graph G. l Then, the historical alarm weights are dynamically adjusted using a time decay factor, and the short-term dynamic alarm map G is periodically updated. s Integrating long-term map G l This enables incremental updates, updating nodes and edges and adjusting weights, and achieves dynamic fusion of long-term knowledge graph and short-term dynamic alarm graph to generate a fused long-term and short-term dynamic alarm graph. S6, output the fused long-term and short-term dynamic alarm map, which integrates the noise reduction results to support response and source tracing.

2. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1, characterized in that, In S1, the data in the alarm log includes: alarm log ID, attack source IP, attack source MAC address, attack source port, target IP, target MAC address, target port, alarm time, alarm type, alarm product, alarm level, operation object type, operation object, alarm object type, alarm object, executed operation, attribute information, and alarm process tree.

3. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1, characterized in that, In S2, the alarm entities include: processes, files, registry entries, host IPs, external IPs, and web URLs; the relationships between the alarm entities include: process creation, file operations, external connections, and web operations.

4. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1, characterized in that, In S2, the subject and object of each alarm data are extracted based on a predefined knowledge graph to construct the entities of the knowledge graph. The alarm type of each alarm data is extracted as the relationship between alarm entities, and the attribute information of each alarm entity is obtained. The alarm log (alert_log) is...<src,dst,relation,attribute> , Wherein, src is the subject, dst is the object, relation is the relationship between alarm entities, and attribute is the attribute information of the alarm entities.

5. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1, characterized in that, In S3, when aggregating alarm graphs, repeated alarms are aggregated according to time windows, retaining the earliest and latest times of alarm occurrence and storing them in attribute A. s In the process, the frequency information of repeated alarms is stored in the weight W. s This is used to achieve initial noise reduction.

6. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1 or 5, characterized in that, In S3, the edge weights of the short-term dynamic alarm graph are: Where p is the frequency, representing the number of alarm events between entity i and entity j within the time window; the larger p is, the more frequent the alarms between the entities occur in the short term, and p is used to reflect the intensity of the behavior pattern. β is a smoothing parameter, which mathematically acts as a Laplace smoothing mechanism to adjust the model's sensitivity to frequency. γ is a time coefficient that controls the rate at which the time interval decays the edge weights. γ determines the degree of influence of the time difference on the association strength. Adjust the semantic information and behavior of alarm data by controlling p, β, and γ; i and j represent entity i and entity j in the short-term dynamic alarm graph, respectively. This represents the edge weight between entity i and entity j in the current short-term dynamic alarm graph. All edge weights constitute the current short-term dynamic alarm graph G. s The set of edge weights Ws; t max The latest time the alarm occurs, t min The earliest time the alarm occurs; Short-term dynamic alarm graph G s It reflects real-time alarm correlations and performs complex structural noise reduction, making it suitable for rapid threat response.

7. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1, characterized in that, In S4, the method for calculating the similarity between alarm entities and the relationships between alarm entities is as follows: s ij =α·cos(a i ,a j )+(1-α)·L(b i ,b j ) Among them, s ij Let cos be the similarity between alarm i and alarm j after short-term aggregation, L be the cosine similarity, and a be the edit distance similarity. i ,a j b is a semantic feature vector. i ,b j Let be the set of behavioral features, and α∈[0,1] be the weight coefficients.

8. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 7, characterized in that, In S4, the DBSCAN algorithm is used to cluster alarm data of the same type based on the similarity calculation results, including: s ij ≥∈, |N ∈ (a i )|≥min_samples Among them, C k Let N be the k-th cluster, ∈ be the neighborhood radius, and N ∈ (a i (For alarm a) i The set of neighborhood points, min_samples is the minimum number of alarm samples for each cluster, a i For the i-th alarm, a j For the j-th alarm; Output: Aggregated alarm set A ' =C1,C2,…,C K Each cluster C k This includes representative alerts and their context.

9. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1, characterized in that, In S5, the time decay factor is designed as follows: Where λ is the time decay factor, used to assign time decay weights to historical alarms, φ(t) ij ) is the decay function, and These are the latest alarm time and the historical alarm time for the current entity i and entity j, respectively.

10. The alarm noise reduction method based on long short-term knowledge graph and fine-grained edge aggregation according to claim 1, characterized in that, In S5, Check the short-term graph nodes; if the short-term graph node v i Existing in the long-term graph node set V l In the middle, the node attributes and statistical attributes are updated to capture the real-time pattern of historical alarm entities triggering alarms in the current state. Simultaneously, the node attributes retain both past and updated attribute values ​​to preserve historical alarm information. If the node v in the short-term graph... i The set of nodes V that does not exist in the long-term graph l If the condition is met, then add the node to V. l middle; The edge weights of the long-term and short-term graphs are dynamically fused using a time decay factor λ as follows: Among them, w ' ij For the weights after fusion, The item is used to amplify the short-term dynamic alarm map G. s The influence of medium-to-high weighted edges This item is used to protect historically associated edges. Durability, t ij This indicates the current time of the alarm formed by entities i and j. This represents the edge between entity i and entity j in the short-term dynamic alarm graph. The edge represents the edge between entity i and entity j in the current long-term alarm graph, and l indicates that it is a long-term alarm graph.

Images