Security alarm association analysis method and device

Abstract

The invention discloses a security alarm association analysis method and device, and relates to the technical field of alarm analysis. The method comprises the following steps: acquiring security alarm data; based on the first time window, performing anomaly analysis on the security alarm frequency and the security alarm type on the security alarm data to obtain a first anomaly source set; based on a second time window, performing frequent pattern mining on the security alarm data, and based on a frequent pattern mining result, performing frequent pattern anomaly analysis on the security alarm data to obtain a second anomaly source set; and based on the first abnormal source set and the second abnormal source set, performing fusion calculation to obtain a target abnormal source set. The method does not need to be based on a preset rule or scene template, the limitation in the prior art is improved, the method is suitable for medium / low safety alarm data, the applicability to the medium / low safety alarm data is improved, and the generalization degree is high.

Description

Technical Field

[0001] The present application relates to the technical field of alarm analysis, and in particular to a method and device for security alarm correlation analysis. Background Art

[0002] With the rapid development of network technology, the scale of the network is becoming larger and larger, the structure of the network is becoming more and more complex, and people are becoming more and more attached to the network. As a result, network attacks are becoming more and more numerous and complex, and security equipment generates a large amount of security alarm data every day. Network operation and maintenance personnel need to analyze various security alarm data to ensure the security of the network and information system.

[0003] In current technology, there are two common methods of correlation analysis of security alert data: correlation analysis based on preset rules and correlation analysis based on scenario templates. The correlation analysis based on preset rules relies on predefined logical rules to identify, filter and correlate security alert data to generate higher-level security alerts or output information. The correlation analysis based on scenario templates performs correlation analysis on security alert data by simulating real attack processes or using mature attack architectures (such as the MITRE ATT&CK matrix or the KillChain model).

[0004] It can be seen from this that the common correlation analysis method of security alarm data in current technology has great limitations, and is even not applicable to medium / low-risk security alarm data, and has a low degree of generalization. Summary of the invention

[0005] Based on the above problems, the present application provides a security alarm correlation analysis method and device, which improves the limitations of the current technology, enhances the applicability to medium / low security alarm data, and has a high degree of generalization.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, the present application provides a security alarm correlation analysis method, comprising:

[0008] Obtain security alert data;

[0009] Based on the first time window, performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data to obtain a first abnormal source set;

[0010] Based on a second time window, frequent pattern mining is performed on the security alarm data, and based on the result of the frequent pattern mining, the abnormality degree of the frequent pattern is analyzed on the security alarm data to obtain a second abnormal source set; wherein the second time window is larger than the first time window;

[0011] Based on the first abnormal source set and the second abnormal source set, a target abnormal source set is obtained by fusion calculation.

[0012] Optionally, based on the first time window, performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data to obtain a first abnormal source set includes:

[0013] Based on the first time window, segmenting the security warning data;

[0014] For each of the first time windows, performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data within the first time window to obtain a corresponding first sub-abnormal source set;

[0015] Based on the first sub-anomaly source sets corresponding to the multiple first time windows, a first anomaly source set is obtained.

[0016] Optionally, performing abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data within the first time window to obtain a corresponding first sub-abnormal source set includes:

[0017] Based on the security alarm data within the first time window, counting the number of occurrences of the same type of security alarm corresponding to each source within the first time window;

[0018] When the number of occurrences of the same type of security alarms at a source exceeds a first preset threshold, the source is marked as a frequency abnormal source, and a frequency abnormal source set is obtained based on all frequency abnormal sources within the first time window;

[0019] Based on the security alarm data within the first time window, counting the number of types of security alarms corresponding to each source within the first time window;

[0020] When the number of types of security alarms corresponding to a source exceeds a second preset threshold, the source is marked as a type abnormal source, and a type abnormal source set is obtained based on all type abnormal sources within the first time window;

[0021] Based on the frequency anomaly source set and the type anomaly source set, a fusion calculation is performed to obtain a corresponding first sub-anomaly source set.

[0022] Optionally, the performing of frequent pattern mining on the security alarm data based on the second time window, and performing frequent pattern abnormality analysis on the security alarm data based on the result of the frequent pattern mining to obtain a second abnormal source set includes:

[0023] Based on the second time window, segmenting the security warning data;

[0024] For each of the second time windows, performing frequent pattern mining on the security alarm data in the second time window, and based on the result of the frequent pattern mining, performing abnormality analysis of the frequent pattern on the security alarm data to obtain a corresponding second sub-abnormal source set;

[0025] A second abnormal source set is obtained based on the second sub-anomaly source sets corresponding to one or more of the second time windows.

[0026] Optionally, the performing frequent pattern mining on the security alarm data in the second time window, and based on the result of the frequent pattern mining, performing frequent pattern abnormality analysis on the security alarm data to obtain a corresponding second sub-abnormal source set, includes:

[0027] Preprocess the security alarm data within the second time window to obtain a type data table and a target data table; wherein the type data table is a data table in which the source identifier is used as a key to divide the rows, and the type of the security alarm corresponding to the source identifier is used as the value of the corresponding row; the target data table is a data table in which the source identifier is used as a key to divide the rows, and the type of the security alarm corresponding to the source identifier and the number of occurrences of the type of security alarm are used as the value of the corresponding row;

[0028] Based on the number of rows in the type data table or the target data table, a support threshold for frequent pattern mining is determined, and based on the support threshold, frequent pattern mining is performed on the type data table to obtain a frequent pattern set; wherein the frequent pattern set is a result of the frequent pattern mining, and the frequent pattern set includes: one or more frequent patterns, and the frequent patterns are types or type combinations of security alarms that meet the support threshold;

[0029] Based on the frequent pattern set, calculating the abnormality count of the source corresponding to the source identifier of each row in the target data table;

[0030] Based on the abnormality degree count of the source corresponding to the source identifier of each row, a second sub-abnormal source set is obtained.

[0031] Optionally, the preprocessing of the security alarm data in the second time window to obtain a type data table and a target data table includes:

[0032] Based on the security alarm data in the second time window, divide the rows by using the source identifier as a key, and extract the type of the security alarm corresponding to the source identifier as the value of the corresponding row to obtain a basic data table;

[0033] De-duplication and aggregation are performed on the types of security alerts in each row of the basic data table to obtain a type data table;

[0034] Based on the basic data table, the number of occurrences of each type of security alarm in each row is recorded in the type data table to obtain a target data table.

[0035] Optionally, the calculating, based on the frequent pattern set, the abnormality count of the source corresponding to the source identifier of each row in the target data table includes:

[0036] Traverse each row in the target data table;

[0037] For the row currently traversed, matching the frequent pattern set for the source corresponding to the row currently traversed, and determining a matched frequent pattern; determining the number of occurrences of the matched frequent pattern based on the row currently traversed in the target data table, and calculating the abnormality count of the source corresponding to the source identifier of the row currently traversed based on the number of occurrences of the matched frequent pattern;

[0038] Until all rows in the target data table are traversed, the abnormality count of the source corresponding to the source identifier of each row in the target data table is obtained.

[0039] Optionally, obtaining a second sub-set of abnormal sources based on the abnormality degree count of the source corresponding to the source identifier of each row includes:

[0040] Based on the abnormality counts of the sources corresponding to the source identifiers of each row, a mean abnormality is calculated, and based on the abnormality counts of the sources corresponding to the source identifiers of each row and the mean abnormality, a standard deviation of the abnormality is calculated;

[0041] Based on the abnormality mean and the abnormality standard deviation, obtaining an abnormality screening value;

[0042] Based on the abnormality screening value, the source whose abnormality count is greater than the abnormality screening value is marked as a frequent abnormal source, and based on all frequent abnormal sources in the second time window, a second sub-abnormal source set is obtained.

[0043] Optionally, the performing frequent pattern mining on the security warning data based on the second time window includes:

[0044] Based on the second time window, frequent pattern mining is performed on the security alarm data by using an Apriori algorithm or an FP-growth algorithm.

[0045] In a second aspect, the present application provides a security alarm correlation analysis device, comprising:

[0046] A data acquisition module, used to acquire security alarm data;

[0047] A first analysis module, configured to perform an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data based on a first time window to obtain a first abnormal source set;

[0048] A second analysis module is used to perform frequent pattern mining on the security alarm data based on a second time window, and perform abnormality analysis of the frequent pattern on the security alarm data based on the result of the frequent pattern mining to obtain a second abnormal source set; wherein the second time window is larger than the first time window;

[0049] A fusion calculation module is used to obtain a target abnormal source set by fusion calculation based on the first abnormal source set and the second abnormal source set.

[0050] Compared with the prior art, the present application has the following beneficial effects: the present application does not need to be based on preset rules or scenario templates, which improves the limitations of the current technology, and the present application is applicable to medium / low safety warning data, improves the applicability to medium / low safety warning data, and has a high degree of generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 A schematic diagram of a flow chart of a security alarm correlation analysis method provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a process for obtaining a first sub-anomaly source set provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a process for obtaining a second sub-anomaly source set provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the structure of a security alarm correlation analysis device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] As described above, the current technology relies on pre-defined logic rules to identify, filter and analyze security alert data. Specifically, logic rules are usually specified based on known attack patterns or features, which means that they can only identify or analyze security alert types that have been defined and recorded. For new or variant security alert data, especially those without clear specific or behavioral patterns, current rules cannot be effectively analyzed, so they have great limitations and low generalization. Furthermore, the preset rules pre-set in the security equipment in the current technology are usually for high-risk security alert data, and the consideration and coverage of medium / low-risk security alert data are generally low, and the applicability is poor.

[0057] The current technology uses scenario template-based correlation analysis methods that rely on simulating real attack processes or mature attack architectures (such as the MITRE ATT&CK matrix or the KillChain model) to perform correlation analysis on security alert data. However, the actual attack scenario link usually has reasons such as scenario step jumps and missing steps, which results in a low matching degree based on the preset scenario template, making it impossible to perform correlation analysis. Therefore, it has great limitations and a low degree of generalization. Furthermore, the scenario templates preset in the security equipment in the current technology are also designed for high-risk security alert data (high-risk attack scenarios), and the consideration and coverage of medium / low-risk security alert data are generally low, and the applicability is poor.

[0058] In summary, the correlation analysis method of security warning data in the current technology has great limitations, and is even not applicable to medium / low-risk security warning data, and has a low degree of generalization.

[0059] The present application provides a security alarm correlation analysis method, including: obtaining security alarm data; based on a first time window, performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data to obtain a first abnormal source set; based on a second time window, performing frequent pattern mining on the security alarm data, and based on the result of the frequent pattern mining, performing an abnormality analysis of the frequent pattern on the security alarm data to obtain a second abnormal source set; based on the first abnormal source set and the second abnormal source set, a fusion calculation is performed to obtain a target abnormal source set. The present application does not need to be based on preset rules or scenario templates, improves the limitations of the current technology, and is applicable to medium / low security alarm data, improves the applicability to medium / low security alarm data, and has a high degree of generalization.

[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] Embodiment 1:

[0062] Combine the following Figure 1-Figure 3 , a security alarm correlation analysis method provided in an embodiment of the present application is introduced in detail.

[0063] It should be noted that the security alarm correlation analysis method provided in the embodiment of the present application is mainly suitable for correlation analysis of medium / low security alarm data.

[0064] like Figure 1 As shown, a security alarm correlation analysis method provided in an embodiment of the present application includes the following steps:

[0065] S101. Obtain security alarm data.

[0066] Among them, security alarm data is the recorded data generated by security devices or systems when potential threats or attack behaviors are detected. Security alarm data is crucial for analyzing and responding to security incidents.

[0067] Specifically, the security alarm data includes: security alarm type, level, source identifier, timestamp, etc.

[0068] Among them, the security alert type (Alert Type) is used to indicate the specific reason or condition of the alert, such as attempted intrusion, malware activity, brute force cracking, etc.

[0069] The severity level is used to identify the importance and urgency of the security alert, and is generally divided into three levels: high, medium, and low. In the embodiment of the present application, the security alert data for joint analysis is security alert data of medium and low levels.

[0070] The source identifier is a unique identifier used to indicate the origin of the suspicious activity or attack. Exemplarily, the source identifier may be a source IP address.

[0071] The timestamp is used to identify the exact time when the security alarm occurs.

[0072] In one possible implementation, after obtaining the security alarm data, the security alarm data is preprocessed, that is, the security alarm data is cleaned, integrated, and formatted, so as to improve the efficiency and accuracy of subsequent correlation analysis and ensure that valuable information can be extracted from the security alarm data.

[0073] It should be noted that S102 is performed for security warning data of medium or low level. That is, the embodiment of the present application is applicable to security warning data of medium or low level, so the security warning data is assumed to be security warning data of medium or low level below.

[0074] S102: Based on the first time window, perform an abnormality analysis on the number of security alarms and the type of security alarms on the security alarm data to obtain a first abnormal source set.

[0075] The first time window is a time window determined by a shorter, adjacent time, for example, 1 minute or 5 minutes as a time window. It can be understood that the first time window is an adjacent time window.

[0076] The abnormality analysis of the number of security alarms on the security alarm data is to analyze whether the number of security alarms of the same type initiated by the same source is abnormal. If abnormal, the source is added to the first abnormal source set.

[0077] The abnormality analysis of the security alarm type on the security alarm data is to analyze whether the number of types of security alarms initiated by the same source is abnormal. If abnormal, the source is added to the first abnormal source set.

[0078] Specifically, based on the first time window, the security alarm data is processed in segments; for each first time window, the number of security alarms and the type of security alarms are analyzed for abnormality of the security alarm data within the first time window to obtain a corresponding first sub-abnormal source set; based on the first sub-abnormal source sets corresponding to multiple first time windows, a first abnormal source set is obtained.

[0079] Wherein, based on the timestamp of the security alarm data, the security alarm data is processed in sections according to the first time window to obtain security alarm data within multiple first time windows.

[0080] For ease of understanding, the following Figure 2 The following describes in detail how to obtain the corresponding first sub-anomaly source set in the embodiment of the present application.

[0081] S201. Based on security alarm data in a first time window, count the number of occurrences of the same type of security alarm corresponding to each source in the first time window.

[0082] Specifically, based on the source identifier and type of the security alarm data within the first time window, the number of security alarms of the same type initiated / triggered by a source corresponding to each source identifier within the first time window is counted.

[0083] Exemplarily, the data carried by the security alarm data is taken as (source identifier, type), and the security alarm data in the first time window includes: (IP1, type A), (IP1, type A), (IP2, type A), (IP2, type B), (IP2, type B). Then the number of occurrences of the same type of security alarms corresponding to each source in the first time window is counted as follows: the number of occurrences of type A security alarms corresponding to IP1 is 2, the number of occurrences of type A security alarms corresponding to IP2 is 1, and the number of occurrences of type B security alarms corresponding to IP2 is 2.

[0084] S202: When the number of occurrences of the same type of security alarms at a source exceeds a first preset threshold, the source is marked as a frequency abnormality source, and a frequency abnormality source set is obtained based on all frequency abnormality sources within a first time window.

[0085] The frequency anomaly source set includes: all frequency anomaly sources within the first time window.

[0086] Exemplarily, the number of occurrences of the same type of security alarm corresponding to each source in the first time window is counted as follows: the number of occurrences of type C security alarms corresponding to IP3 is 5, and the number of occurrences of type D security alarms corresponding to IP3 is 2; the number of occurrences of type C security alarms corresponding to IP4 is 2, and the number of occurrences of type D security alarms corresponding to IP4 is 2, and when the first preset threshold is 3, then the number of occurrences of type C security alarms for IP3 exceeds 3, then IP3 is marked as a frequency abnormality source, and if the number of occurrences of the same type of security alarms for IP4 does not exceed 3, then IP4 is not a frequency abnormality source.

[0087] The above steps S201 and S202 perform an abnormality analysis of the number of security alarms on the security alarm data within the first time window. By counting the number of occurrences of the same type of security alarms corresponding to each source, it is possible to identify the attack initiator (i.e., source) who frequently repeats a specific attack (security alarm) to detect potential threats as early as possible.

[0088] S203: Based on the security alarm data in the first time window, count the number of types of security alarms corresponding to each source in the first time window.

[0089] Specifically, based on the source identifier and type of the security alarm data in the first time window, the number of types of security alarms initiated by the source corresponding to each source identifier in the first time window is counted.

[0090] Exemplarily, the data carried by the security alarm data is taken as (source identifier, type). The security alarm data in the first time window includes: (IP1, type A), (IP1, type A), (IP1, type B), (IP1, type C), (IP2, type A), (IP2, type B), (IP2, type B). Then, the number of types of security alarms corresponding to each source in the first time window is statistically as follows: the number of types of security alarms corresponding to IP1 is 3 (Type A, Type B and Type C), and the number of types of security alarms corresponding to IP2 is 2 (Type A and Type B).

[0091] S204: When the number of types of security alarms corresponding to a source exceeds a second preset threshold, the source is marked as a type abnormal source, and a type abnormal source set is obtained based on all type abnormal sources within the first time window.

[0092] The type anomaly source set includes: all type anomaly sources within the first time window.

[0093] Exemplarily, the number of types of security alarms corresponding to each source in the first time window is counted as follows: the number of types of security alarms corresponding to IP3 is 5, the number of types of security alarms corresponding to IP4 is 6, the number of types of security alarms corresponding to IP5 is 3, and the second preset threshold is 4, then if the number of types of security alarms corresponding to IP3 and IP4 exceeds 4, IP3 and IP4 are marked as type abnormal sources.

[0094] The above steps S203 and S204 perform abnormality analysis of the security alarm types on the security alarm data in the first time window. By counting the number of types of security alarms corresponding to each source, it is possible to identify that the attack initiator (ie, the source) is trying multiple means / types of attacks to detect potential threats as early as possible.

[0095] It should be noted that there is no specific time sequence between S201-S202 and S203-S204. S201-S202 can be performed first, and then S203-S204; or S203-S204 can be performed first, and then S201-S202. In addition, S201-S202 and S203-S204 can also be performed simultaneously / almost simultaneously, and this application does not make any specific limitations.

[0096] S205: Based on the frequency anomaly source set and the type anomaly source set, a fusion calculation is performed to obtain a corresponding first sub-anomaly source set.

[0097] Specifically, the frequency anomaly source set and the type anomaly source set are subjected to a union process to obtain a corresponding first sub-anomaly source set, that is, the first sub-anomaly source set is the union of the frequency anomaly source set and the type anomaly source set.

[0098] A union is the result of merging two or more sets, where each element belongs to at least one of the sets. For example, for any two sets S1 and S2, their union S1∪S2 contains all the elements that belong to S1 and S2, while removing duplicates.

[0099] By combining the frequency anomaly source set and the type anomaly source set, we ensure that no anomaly source is missed. Even if a source is an anomaly source only in one anomaly analysis (i.e., the number of security alarms or the type of security alarms), it will be included in the final first sub-anomaly source set. And redundancy is reduced. If a source is an anomaly source in both anomaly analysis, it will only appear once in the first sub-anomaly source set to avoid repeated processing.

[0100] The above combination Figure 2 The first sub-anomaly source set is obtained in detail in the embodiment of the present application. Figure 1 A security alarm joint analysis method provided in an embodiment of the present application is introduced.

[0101] S103: Based on the second time window, perform frequent pattern mining on the security alarm data, and based on the result of the frequent pattern mining, perform abnormality analysis on the security alarm data of the frequent pattern to obtain a second abnormal source set.

[0102] The second time window is larger than the first time window. The second time window is a time window determined by a longer, non-adjacent time. For example, the second time window is a time window of 1 week or 2 weeks.

[0103] Frequent pattern mining is an important task in data mining, which aims to find frequently occurring items, subsequences or substructures from large data sets. In the embodiment of the present application, frequent pattern mining aims to find frequently occurring types of security alerts or types of security alerts in a large amount of data security data.

[0104] Among them, the abnormality analysis of frequent patterns of security alarm data is to analyze whether there are frequent patterns that appear frequently and may indicate malicious activities or abnormal actions.

[0105] In a possible implementation, based on the second time window, frequent pattern mining is performed on the security alarm data by using an Apriori algorithm or an FP-growth algorithm.

[0106] Among them, the Apriori algorithm is one of the classic algorithms for frequent pattern mining in the field of data mining. It is an iterative algorithm based on the prior principle (AprioriProperty).

[0107] Among them, the FP-growth algorithm efficiently stores and processes the mining of frequent patterns by constructing a data result called FP tree.

[0108] Specifically, based on the second time window, the security alarm data is processed in segments; for each second time window, frequent pattern mining is performed on the security alarm data within the second time window, and based on the results of the frequent pattern mining, the abnormality of the frequent pattern of the security alarm data is analyzed to obtain the corresponding second sub-abnormal source set; based on the second sub-abnormal source sets corresponding to one or more second time windows, a second abnormal source set is obtained.

[0109] For ease of understanding, the following Figure 3 The following describes in detail how to obtain the corresponding second sub-anomaly source set in the embodiment of the present application.

[0110] S301. Preprocess the security alarm data in the second time window to obtain a type data table and a target data table.

[0111] The type data table is a data table that is divided into rows using source identifiers as keys and uses the types of security alerts corresponding to the source identifiers as values ​​of corresponding rows.

[0112] The target data table is a data table that is divided into rows using the source identifier as a key, and the type of security alert corresponding to the source identifier and the number of occurrences of the type of security alert are used as values ​​of the corresponding rows.

[0113] In one possible implementation, based on the security alarm data in the second time window, the rows are divided with the source identifier as the key, and the type of the security alarm corresponding to the source identifier is extracted as the value of the corresponding row to obtain a basic data table; the types of security alarms in each row in the basic data table are deduplicated and aggregated to obtain a type data table; based on the basic data table, the number of occurrences of each type of security alarm in each row is recorded in the type data table to obtain a target data table.

[0114] Exemplarily, the security alarm data in the second time window is divided into rows according to the source identifier as the key, and the type of the security alarm corresponding to the source identifier (ie, initiated) is extracted as the value of the corresponding row, and the basic data table is obtained as shown in Table 1:

[0115] Table 1

[0116]

[0117] As shown in Table 1, the security alarm data in the second time window is divided into three rows according to IP1, IP2 and IP3 as keys, and the types A, B, C, A and B of the security alarm corresponding to (initiated) IP1 are extracted as the values ​​of the row corresponding to IP1; the types B, C, C and B of the security alarm corresponding to (initiated) IP2 are extracted as the values ​​of the row corresponding to IP2; the types A, B and A of the security alarm corresponding to (initiated) IP3 are extracted as the values ​​of the row corresponding to IP3, thereby obtaining the basic data table shown in Table 1.

[0118] The types of security alerts in each row of the basic data table are deduplicated and aggregated to obtain the type data table shown in Table 2:

[0119] Table 2

[0120]

[0121] As shown in Table 2, the types of security alarms in each row of the basic data table (i.e., Table 1) are deduplicated and aggregated to obtain a type data table. That is, the types of security alarms corresponding to (or initiated by) IP1 are clustered and deduplicated into A, B, and C; the types of security alarms corresponding to (or initiated by) IP2 are clustered and deduplicated into B and C; and the types of security alarms corresponding to (or initiated by) IP3 are clustered and deduplicated into A and B. As shown in Table 2, the types of security alarms corresponding to (or initiated by) IP1 are A, B, and C; the types of security alarms corresponding to (or initiated by) IP2 are B and C; and the types of security alarms corresponding to (or initiated by) IP3 are A and B.

[0122] Based on the basic data table, the number of occurrences of each type of security alarm in each row is recorded in the type data table, and the target data table is obtained as shown in Table 3:

[0123] Table 3

[0124]

[0125] As shown in Table 3, based on the basic data table (Table 1), the number of occurrences of each type of security alarm in each row is recorded in the type data table (Table 2), that is, the number of occurrences of type A security alarms (initiated) corresponding to IP1 is recorded as 2 in the type data table, the number of occurrences of type B security alarms is recorded as 2 in the type data table, the number of occurrences of type C security alarms is recorded as 1 in the type data table; the number of occurrences of type B security alarms (initiated) corresponding to IP2 is recorded as 2 in the type data table, and the number of occurrences of type C security alarms is recorded as 1 in the type data table; the number of occurrences of type A security alarms (initiated) corresponding to IP3 is recorded as 2 in the type data table, and the number of occurrences of type B security alarms is recorded as 1 in the type data table, and the target data table (i.e., Table 3) is obtained.

[0126] S302: Determine a support threshold for frequent pattern mining based on the number of rows in the type data table or the target data table, and perform frequent pattern mining on the type data table based on the support threshold to obtain a frequent pattern set.

[0127] Among them, the frequent pattern set is the result of frequent pattern mining, and the frequent pattern set includes: one or more frequent patterns, and the frequent patterns are the types or type combinations of security alarms that meet the support threshold, that is, the types or type combinations of security alarms that exceed the support threshold.

[0128] Specifically, based on the number of rows n in the type data table or the target data table, the support threshold d is obtained by the formula d=n*0.3.

[0129] S303: Based on the frequent pattern set, calculate the abnormality count of the source corresponding to the source identifier of each row in the target data table.

[0130] Specifically, each row in the target data table is traversed; for the currently traversed row, the source matching frequent pattern set corresponding to the currently traversed row is determined to determine the matching frequent pattern; based on the currently traversed row in the target data table, the number of occurrences of the matching frequent pattern is determined, and based on the number of occurrences of the matching frequent pattern, the source anomaly count corresponding to the source identifier of the currently traversed row is calculated; until all rows in the target data table are traversed, the source anomaly count corresponding to the source identifier of each row in the target data table is obtained.

[0131] For ease of understanding, the following is a detailed description of how to calculate the abnormality count of the source corresponding to the source identifier of each row in the target data table in combination with formula (1). The abnormality count of the source corresponding to the source identifier of each row in the target data table is denoted as C i , where C i is the abnormality count corresponding to the source with source identifier IPi, so the abnormality count of the source corresponding to the source identifier of each row is C 1 , C 2 , C 3 , ..., C i .

[0132] For the source abnormality count Ci corresponding to the source identifier IPi, initialize C i =0; match the source corresponding to the source identifier IPi with the frequent pattern set P={P1, P2, ..., Ps}, determine the matched frequent pattern, and determine the number of occurrences of the matched frequent pattern; and based on the number of occurrences of the matched frequent pattern, accumulate the abnormality count C of the source corresponding to the source identifier IPi i .

[0133] Specifically, the abnormality count is obtained by formula (1).

[0134]

[0135] Among them, C i is the abnormality count of the source corresponding to the source identifier IPi, P j is the jth frequent pattern in the frequent pattern set, cnt(P j ) is P j The number of occurrences of IPi in the target data table, K belongs to P j , and |K|=1, that is, K is the frequent pattern P j is a type of security alarm in , and cnt(K) is the number of occurrences of type K of security alarm.

[0136] Exemplarily, taking Table 2 as an example, the frequent pattern set is P = {AB, C}.

[0137] The abnormality count C corresponding to IP1 1 =min{2,2}+1=3, that is, the abnormality count C corresponding to IP1 1 It is the sum of the smaller number of occurrences of type A security alarms and type B security alarms, and the number of occurrences of type C security alarms.

[0138] The abnormality count C corresponding to IP2 2 =min{0,2}+2=2, that is, the abnormality count C corresponding to IP2 2 It is the sum of the smaller number of occurrences of the security alarm of type A and the number of occurrences of the security alarm of type B, and the number of occurrences of the security alarm of type C. It can be understood that the frequent pattern AB is not a frequent pattern matched by IP2.

[0139] The abnormality count C corresponding to IP3 3 =min{2,1}+0=2, that is, the abnormality count C corresponding to IP3 3 It is the sum of the smaller number of occurrences of the security alarm of type A and the number of occurrences of the security alarm of type B, and the number of occurrences of the security alarm of type C. It can be understood that the frequent pattern C is not a frequent pattern matched by IP3.

[0140] S304: Obtain a second sub-abnormal source set based on the abnormality degree count of the source corresponding to the source identifier of each row.

[0141] Specifically, based on the anomaly degree count of the source corresponding to the source identifier of each row, the mean anomaly degree is calculated, and based on the anomaly degree count and the mean anomaly degree of the source corresponding to the source identifier of each row, the standard deviation of the anomaly degree is calculated; based on the mean anomaly degree and the standard deviation of the anomaly degree, an anomaly degree screening value is obtained; based on the anomaly degree screening value, the source whose anomaly degree count is greater than the anomaly degree screening value is marked as a frequent anomaly source, and based on all frequent anomaly sources in the second time window, a second sub-anomaly source set is obtained.

[0142] For ease of understanding, the following describes in detail how to obtain the abnormality screening value in combination with formula (2) to formula (4).

[0143] Based on the anomaly count of the source corresponding to the source identifier of each row, the anomaly mean is calculated using formula (2).

[0144]

[0145] Among them, u is the mean of abnormality, n is the number of rows in the target data table, C i is the abnormality count of the source corresponding to the i-th row (i.e., IPi).

[0146] Based on the anomaly count and anomaly mean of the source corresponding to the source identifier of each row, the anomaly standard deviation is calculated using formula (3).

[0147]

[0148] Among them, σ is the standard deviation of abnormality, u is the mean of abnormality, n is the number of rows in the target data table, C i (i.e., IPi) is the abnormality count of the source corresponding to the i-th row.

[0149] Based on the abnormality mean and the abnormality standard deviation, the abnormality screening value is calculated by formula (4).

[0150] T=u+3σ (4)

[0151] Among them, T is the outlier screening value, u is the outlier mean, and σ is the outlier standard deviation.

[0152] The anomaly mean and three times the standard deviation of the anomaly are used as anomaly screening values ​​to identify anomalies or extreme values ​​based on the characteristics of the normal distribution. For example, in the embodiment of the present application, if the anomaly count of a source exceeds μ+3σ, it may indicate a potential security threat or attack behavior, that is, the source is a frequent anomaly source.

[0153] The above combination Figure 3 This article describes in detail how to obtain the corresponding second sub-anomaly source set in the embodiment of the present application. Figure 1 , introduces a security alarm correlation analysis method provided in an embodiment of the present application.

[0154] S104: Based on the first abnormal source set and the second abnormal source set, a fusion calculation is performed to obtain a target abnormal source set.

[0155] Specifically, the first abnormal source set and the second abnormal source set are combined to obtain a target abnormal source set.

[0156] A security alarm correlation analysis method provided by an embodiment of the present application includes: obtaining security alarm data; based on a first time window, performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data to obtain a first abnormal source set; based on a second time window, performing frequent pattern mining on the security alarm data, and based on the result of the frequent pattern mining, performing an abnormality analysis of the frequent pattern on the security alarm data to obtain a second abnormal source set; based on the first abnormal source set and the second abnormal source set, a fusion calculation is performed to obtain a target abnormal source set. The embodiment of the present application does not need to be based on preset rules or scenario templates, which improves the limitations of the current technology, and the embodiment of the present application is applicable to medium / low security alarm data, improves the applicability to medium / low security alarm data, and has a high degree of generalization.

[0157] Moreover, the embodiments of the present application are mainly aimed at medium / low-level security warning data, and the number and types of medium and low-level security warning data are large. Since the second time window is larger than the first time window, that is, the abnormality analysis of the adjacent time of the security warning data is performed, the abnormal source of the basic and fast abnormal attack can be identified, and then the abnormality analysis of the security warning data is performed for a medium and long time, and the abnormal source of the slow and intermittent abnormal attack can be identified. Finally, the abnormal source sets obtained under the two time windows are integrated to obtain a comprehensive and accurate identification of the abnormal source.

[0158] Embodiment 2:

[0159] Combine the following Figure 4 , a security alarm correlation analysis device provided in an embodiment of the present application is introduced in detail.

[0160] like Figure 4 As shown, a security alarm correlation analysis device provided in an embodiment of the present application includes the following modules:

[0161] Data acquisition module 401, used to acquire security alarm data;

[0162] A first analysis module 402 is used to perform an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data based on the first time window to obtain a first abnormal source set;

[0163] The second analysis module 403 is used to perform frequent pattern mining on the security alarm data based on the second time window, and perform abnormality analysis of the frequent pattern on the security alarm data based on the result of the frequent pattern mining to obtain a second abnormal source set; wherein the second time window is larger than the first time window;

[0164] The fusion calculation module 404 is used to obtain a target abnormal source set by fusion calculation based on the first abnormal source set and the second abnormal source set.

[0165] In one possible implementation, the first analysis module 402 is specifically used to perform segmented processing on the security alarm data based on the first time window; for each first time window, perform an abnormality analysis on the number of security alarms and the type of security alarms on the security alarm data within the first time window to obtain a corresponding first sub-abnormal source set; and obtain a first abnormal source set based on the first sub-abnormal source sets corresponding to multiple first time windows.

[0166] In a possible implementation, the first analysis module 402 includes: a frequency analysis module, a type analysis module, and a set fusion module.

[0167] The frequency analysis module is used to count the number of occurrences of the same type of security alarms corresponding to each source in the first time window based on the security alarm data in the first time window; when the number of occurrences of the same type of security alarms of the source exceeds the first preset threshold, the source is marked as a frequency anomaly source, and based on all frequency anomaly sources in the first time window, a frequency anomaly source set is obtained.

[0168] The type analysis module is used to count the number of types of security alarms corresponding to each source in the first time window based on the security alarm data in the first time window; when the number of types of security alarms corresponding to the source exceeds a second preset threshold, the source is marked as a type abnormal source, and based on all type abnormal sources in the first time window, a type abnormal source set is obtained.

[0169] The set fusion module is used to obtain the corresponding first sub-anomaly source set by fusion calculation based on the frequency anomaly source set and the type anomaly source set.

[0170] In one possible implementation, the second analysis module 403 is specifically used to perform segmented processing on the security alarm data based on the second time window; for each second time window, perform frequent pattern mining on the security alarm data within the second time window, and based on the results of the frequent pattern mining, perform frequent pattern abnormality analysis on the security alarm data to obtain a corresponding second sub-abnormal source set; based on the second sub-abnormal source sets corresponding to one or more second time windows, obtain a second abnormal source set.

[0171] In a possible implementation, the second analysis module 403 includes: a preprocessing module, a pattern mining module, an abnormality module and a counting screening module.

[0172] The preprocessing module is used to preprocess the security alarm data in the second time window to obtain a type data table and a target data table.

[0173] The pattern mining module is used to determine the support threshold of frequent pattern mining based on the number of rows in the target data table or the type data table, and based on the support threshold, perform frequent pattern mining on the type data table to obtain a frequent pattern set.

[0174] The anomaly module is used to calculate the anomaly count of the source corresponding to the source identifier of each row in the target data table based on the frequent pattern set.

[0175] The counting and filtering module is used to obtain a second sub-abnormal source set based on the abnormality degree count of the source corresponding to the source identifier of each row.

[0176] In one possible implementation, the preprocessing module is specifically used to divide the security alarm data within the second time window into rows based on the source identifier as the key, and extract the type of security alarm corresponding to the source identifier as the value of the corresponding row to obtain a basic data table; deduplicate and aggregate the types of security alarms in each row in the basic data table to obtain a type data table; based on the basic data table, record the number of occurrences of each type of security alarm in each row in the type data table to obtain a target data table.

[0177] In a possible implementation, the anomaly degree module is specifically used to traverse each row in the target data table; for the currently traversed row, the source matching frequent pattern set corresponding to the currently traversed row is determined to determine the matching frequent pattern; based on the currently traversed row in the target data table, the number of occurrences of the matching frequent pattern is determined, and based on the number of occurrences of the matching frequent pattern, the source anomaly degree count corresponding to the source identifier of the currently traversed row is calculated; until all rows in the target data table are traversed, the source anomaly degree count corresponding to the source identifier of each row in the target data table is obtained.

[0178] In one possible implementation, the counting and filtering module is specifically used to calculate the mean anomaly based on the anomaly count of the source corresponding to the source identifier of each row, and calculate the standard deviation of the anomaly based on the anomaly count and the mean anomaly of the source corresponding to the source identifier of each row; obtain the anomaly filtering value based on the anomaly mean and the anomaly standard deviation; based on the anomaly filtering value, mark the source whose anomaly count is greater than the anomaly filtering value as a frequent anomaly source, and obtain a second sub-anomaly source set based on all frequent anomaly sources in the second time window.

[0179] In a possible implementation, the second analysis module is specifically configured to perform frequent pattern mining on the security alarm data based on the second time window by using an Apriori algorithm or an FP-growth algorithm.

[0180] The embodiment of the present application provides a security alarm correlation analysis device, including: a data acquisition module 401, used to acquire security alarm data; a first analysis module 402, used to perform an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data based on a first time window, to obtain a first abnormal source set; a second analysis module 403, used to perform frequent pattern mining on the security alarm data based on a second time window, and based on the result of the frequent pattern mining, perform an abnormality analysis of the frequent pattern on the security alarm data to obtain a second abnormal source set; a fusion calculation module 404, used to perform a fusion calculation based on the first abnormal source set and the second abnormal source set to obtain a target abnormal source set. The embodiment of the present application does not need to be based on preset rules or scenario templates, improves the limitations of the current technology, and the embodiment of the present application is applicable to medium / low security alarm data, improves the applicability to medium / low security alarm data, and has a high degree of generalization.

[0181] Moreover, the embodiments of the present application are mainly aimed at medium / low-level security warning data, and the number and types of medium and low-level security warning data are large. Since the second time window is larger than the first time window, that is, the abnormality analysis of the adjacent time of the security warning data is performed, the abnormal source of the basic and fast abnormal attack can be identified, and then the abnormality analysis of the security warning data is performed for a medium and long time, and the abnormal source of the slow and intermittent abnormal attack can be identified. Finally, the abnormal source sets obtained under the two time windows are integrated to obtain a comprehensive and accurate identification of the abnormal source.

[0182] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method and device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The method and device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A security alarm correlation analysis method, characterized in that: include: Obtain security alert data; Based on the first time window, performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data to obtain a first abnormal source set; Based on a second time window, frequent pattern mining is performed on the security alarm data, and based on the result of the frequent pattern mining, an abnormality degree analysis of the frequent pattern is performed on the security alarm data to obtain a second abnormal source set; wherein the second time window is larger than the first time window; Based on the first abnormal source set and the second abnormal source set, a target abnormal source set is obtained by fusion calculation.

2. The method according to claim 1, characterized in that The step of performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data based on the first time window to obtain a first abnormal source set includes: Based on the first time window, segmenting the security warning data; For each of the first time windows, performing an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data within the first time window to obtain a corresponding first sub-abnormal source set; Based on the first sub-anomaly source sets corresponding to the multiple first time windows, a first anomaly source set is obtained.

3. The method according to claim 2, characterized in that The performing of abnormality analysis on the number of security alarms and the type of security alarms on the security alarm data within the first time window to obtain a corresponding first sub-abnormal source set includes: Based on the security alarm data within the first time window, counting the number of occurrences of the same type of security alarm corresponding to each source within the first time window; When the number of occurrences of the same type of security alarms at a source exceeds a first preset threshold, the source is marked as a frequency abnormal source, and a frequency abnormal source set is obtained based on all frequency abnormal sources within the first time window; Based on the security alarm data within the first time window, counting the number of types of security alarms corresponding to each source within the first time window; When the number of types of security alarms corresponding to a source exceeds a second preset threshold, the source is marked as a type abnormal source, and a type abnormal source set is obtained based on all type abnormal sources within the first time window; Based on the frequency anomaly source set and the type anomaly source set, a fusion calculation is performed to obtain a corresponding first sub-anomaly source set.

4. The method according to claim 1, characterized in that: The step of performing frequent pattern mining on the security alarm data based on the second time window, and performing abnormality analysis of the frequent pattern on the security alarm data based on the result of the frequent pattern mining to obtain a second abnormal source set includes: Based on the second time window, segmenting the security warning data; For each of the second time windows, performing frequent pattern mining on the security alarm data in the second time window, and based on the result of the frequent pattern mining, performing abnormality analysis of the frequent pattern on the security alarm data to obtain a corresponding second sub-abnormal source set; A second abnormal source set is obtained based on the second sub-anomaly source sets corresponding to one or more of the second time windows.

5. The method according to claim 4, characterized in that The performing frequent pattern mining on the security alarm data in the second time window, and performing frequent pattern abnormality analysis on the security alarm data based on the result of the frequent pattern mining to obtain a corresponding second sub-abnormal source set, includes: Preprocess the security alarm data within the second time window to obtain a type data table and a target data table; wherein the type data table is a data table in which the source identifier is used as a key to divide the rows, and the type of the security alarm corresponding to the source identifier is used as the value of the corresponding row; the target data table is a data table in which the source identifier is used as a key to divide the rows, and the type of the security alarm corresponding to the source identifier and the number of occurrences of the type of security alarm are used as the value of the corresponding row; Based on the number of rows in the type data table or the target data table, a support threshold for frequent pattern mining is determined, and based on the support threshold, frequent pattern mining is performed on the type data table to obtain a frequent pattern set; wherein the frequent pattern set is a result of the frequent pattern mining, and the frequent pattern set includes: one or more frequent patterns, and the frequent patterns are types or type combinations of security alarms that meet the support threshold; Based on the frequent pattern set, calculating the abnormality count of the source corresponding to the source identifier of each row in the target data table; Based on the abnormality degree count of the source corresponding to the source identifier of each row, a second sub-abnormal source set is obtained.

6. The method according to claim 5, characterized in that The preprocessing of the security alarm data in the second time window to obtain a type data table and a target data table includes: Based on the security alarm data in the second time window, divide the rows by using the source identifier as a key, and extract the type of the security alarm corresponding to the source identifier as the value of the corresponding row to obtain a basic data table; De-duplication and aggregation are performed on the types of security alerts in each row of the basic data table to obtain a type data table; Based on the basic data table, the number of occurrences of each type of security alarm in each row is recorded in the type data table to obtain a target data table.

7. The method according to claim 5, characterized in that The calculating, based on the frequent pattern set, the abnormality count of the source corresponding to the source identifier of each row in the target data table includes: Traverse each row in the target data table; For the row currently traversed, matching the frequent pattern set for the source corresponding to the row currently traversed, and determining a matching frequent pattern; determining the number of occurrences of the matching frequent pattern based on the row currently traversed in the target data table, and calculating the abnormality count of the source corresponding to the source identifier of the row currently traversed based on the number of occurrences of the matching frequent pattern; Until all rows in the target data table are traversed, the abnormality count of the source corresponding to the source identifier of each row in the target data table is obtained.

8. The method according to claim 5, characterized in that The second sub-abnormal source set is obtained based on the abnormality degree count of the source corresponding to the source identifier of each row, including: Based on the abnormality counts of the sources corresponding to the source identifiers of each row, a mean abnormality is calculated, and based on the abnormality counts of the sources corresponding to the source identifiers of each row and the mean abnormality, a standard deviation of the abnormality is calculated; Based on the abnormality mean and the abnormality standard deviation, obtaining an abnormality screening value; Based on the abnormality screening value, the source whose abnormality count is greater than the abnormality screening value is marked as a frequent abnormal source, and based on all frequent abnormal sources in the second time window, a second sub-abnormal source set is obtained.

9. The method according to any one of claims 1 to 8, characterized in that: The performing frequent pattern mining on the security warning data based on the second time window includes: Based on the second time window, frequent pattern mining is performed on the security alarm data by using an Apriori algorithm or an FP-growth algorithm.

10. A security alarm correlation analysis device, characterized in that: include: A data acquisition module, used to acquire security alarm data; A first analysis module, configured to perform an abnormality analysis of the number of security alarms and the type of security alarms on the security alarm data based on a first time window to obtain a first abnormal source set; A second analysis module is used to perform frequent pattern mining on the security alarm data based on a second time window, and perform abnormality analysis of the frequent pattern on the security alarm data based on the result of the frequent pattern mining to obtain a second abnormal source set; wherein the second time window is larger than the first time window; A fusion calculation module is used to obtain a target abnormal source set by fusion calculation based on the first abnormal source set and the second abnormal source set.

Images