A method and device for quantitative analysis of network security threats

Abstract

The application discloses a network security threat quantitative analysis method and device, the quantitative analysis method comprises the following steps: collecting threat data and preprocessing the threat data; analyzing the preprocessed threat data, obtaining the correlation between all security threats of a same source IP; calculating the threat risk value of the source IP by using the correlation, taking the threat risk value as the evaluation result of the source IP; and outputting the evaluation result. The application analyzes the security threats of the same source IP, obtains the threat risk value of the source IP, quantizes the risk of the security threats, and accurately evaluates the risk degree of the threats.

Description

Technical Field

[0001] This application relates to the field of cybersecurity technology, and more specifically, to a method and apparatus for quantitative analysis of cybersecurity threats. Background Technology

[0002] With the development of information technology and the continuous growth of business scale, the resulting cybersecurity risks and threats are increasing rapidly. As business systems become increasingly complex and interconnected, the forms and types of cybersecurity threats are also rising, especially with the emergence of new attack methods and techniques. Advanced persistent threats (APs) are having a very serious impact on current cybersecurity. Current cybersecurity protection efforts face unprecedented difficulties and challenges. Many devices in existing cybersecurity protection systems (such as firewalls, intrusion detection systems, IPS (Intrusion Prevention Systems), and WAFs (Web Application Firewalls)) generate massive amounts of alert logs, recording cybersecurity events on devices and within the network. Querying and statistically analyzing these logs can help understand the current security threat situation of the network.

[0003] However, all these alerts suffer from a serious problem: they only show the result, not the cause, and these results often contain significant redundancy and false alarms. The root cause of all cybersecurity incidents lies in security threats; security alerts without in-depth threat analysis are practically useless. Especially in the current security environment, isolated evidence is insufficient; without cross-validation and a complete chain of evidence, the responses and analyses are essentially for single, isolated security incidents, thus having significant limitations. Furthermore, because they are single-incident analyses, they lead to massive data accumulation, obscuring the true threats and making it difficult for cybersecurity managers to extract valuable information, thus posing a significant challenge to current cybersecurity threat analysis.

[0004] Cybersecurity threats are growing with the proliferation of data-driven technologies, and mitigating both routine and targeted threats without disrupting operations and growth has become an increasingly important business requirement. To better provide relevant managers with the information they need to make impactful and far-reaching strategic decisions, a clear understanding of all remaining and evolving business threats is essential. Currently, in the field of cybersecurity risk assessment, domestic security risk management primarily focuses on vulnerability assessment, while international approaches emphasize holistic security risk management. However, regardless of domestic or international context, current security risk management, especially threat management, is largely implemented qualitatively, failing to quantify threat risks and thus hindering accurate assessment of their severity. Summary of the Invention

[0005] This application provides a method and apparatus for quantitative analysis of network security threats, which analyzes security threats from the same source IP, obtains the threat risk value of the source IP, and quantifies the risk of the security threat to accurately assess the degree of threat risk.

[0006] This application provides a method for quantitative analysis of cybersecurity threats, including:

[0007] Collect threat data and preprocess it;

[0008] Analyze the pre-processed threat data to obtain the correlation between all security threats from the same source IP;

[0009] The threat risk value of the source IP is calculated using the correlation relationship, which serves as the assessment result for the source IP;

[0010] Output the evaluation results.

[0011] Preferably, the output data is a visualization of the threat attack chain of the source IP.

[0012] Preferably, a frequent pattern tree is used to obtain the correlation between all security threats from the same source IP.

[0013] Preferably, the preprocessed threat data is analyzed to obtain the correlation between all security threats from the same source IP, specifically including:

[0014] The preprocessed threat data is cleaned to transform the security threats into a transaction database. Each security threat is treated as an item in the transaction database. The time and space of the security threat are used as attributes of the security threat. The attribute values ​​of different security threats are combined into an itemset.

[0015] A frequent pattern tree is constructed based on a transaction database, wherein frequent itemsets for each security threat are selected based on minimum support and minimum confidence.

[0016] On the frequent pattern tree, the frequent pattern growth algorithm is used to mine frequent itemsets of security threats from the same source IP;

[0017] Based on the results of frequent itemset mining, correlation analysis is performed on all security threats from the same source IP to obtain the correlation and anomalies between different security threats from the same source IP.

[0018] Preferably, the threat risk value of the source IP is calculated using correlation relationships, specifically including:

[0019] Based on correlation analysis, quantitative threat indicators from IP addresses originating from the same source are used.

[0020] The threat risk value of the source IP is calculated based on the quantitative threat analysis indicators.

[0021] This application also provides a quantitative analysis device for network security threats, including a data acquisition and preprocessing module, a threat perception module, a threat quantification module, and an output module;

[0022] The data acquisition and preprocessing module is used to acquire threat data and preprocess it.

[0023] The threat awareness module is used to analyze preprocessed threat data to obtain the correlation between all security threats from the same source IP.

[0024] The threat quantification module is used to calculate the threat risk value of the source IP using correlation relationships, as an assessment result of the source IP;

[0025] The output module is used to output the evaluation results.

[0026] Preferably, the output data of the output module is a visualization of the threat attack chain of the source IP.

[0027] Preferably, the threat perception module is used to obtain the correlation between all security threats from the same source IP by employing a frequent pattern tree.

[0028] Preferably, the threat perception module includes a conversion module, a construction module, a mining module, and a correlation analysis module;

[0029] The conversion module is used to clean the preprocessed threat data, converting security threats into a transaction database. Each security threat is an item in the transaction database, and the time and space of the security threat are used as attributes of the security threat. The attribute values ​​of different security threats are combined into an itemset.

[0030] The building module is used to construct a frequent pattern tree based on the transaction database, where frequent itemsets for each security threat are filtered according to minimum support and minimum confidence.

[0031] The mining module is used to mine frequent itemsets of security threats from the same source IP on the frequent pattern tree using a frequent pattern growth algorithm;

[0032] The association analysis module is used to perform association analysis on all security threats from the same source IP based on the results of frequent itemset mining, and to obtain the association and anomalies between different security threats from the same source IP.

[0033] Preferably, the threat quantification module includes a statistics module and a calculation module;

[0034] The statistics module is used to calculate the total duration of threat attacks from the same source IP, the number of targets of threat attacks, the number of threat attack methods, and the total number of threat attacks based on the correlation.

[0035] The calculation module is used to calculate the threat risk value of the source IP based on the total duration of the threat attack, the number of targets of the threat attack, the number of threat attack methods, and the total number of threat attacks.

[0036] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0038] Figure 1 A flowchart illustrating the quantitative analysis method for cybersecurity threats provided in this application;

[0039] Figure 2 An example of an FP-tree provided for this application;

[0040] Figure 3 A structural diagram of the network security threat quantification analysis device provided in this application. Detailed Implementation

[0041] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0042] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0043] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0044] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0045] This application provides a method and apparatus for quantitative analysis of network security threats. It analyzes security threats from the same source IP address to obtain the threat risk value of that source IP, quantifying the risk of the security threat to accurately assess its severity. Furthermore, this application employs a Frequent Pattern Tree (FP-tree) approach to correlate and statistically analyze security threats from the same source IP address. The compressibility of FP-tree significantly reduces the time and space complexity of mining frequent itemsets, thereby improving data analysis efficiency. Moreover, this application considers the importance of the attacked information assets when calculating the total number of threat attacks, enabling focused protection of core information assets. It also effectively converges noisy network security data, achieving proactive security risk prevention, in-process monitoring and analysis, and post-attack backtesting and evidence collection. This helps enterprises meet security compliance requirements, improve network security governance, effectively ensure the secure operation of the network, improve operational efficiency, and enhance security management.

[0046] Example 1

[0047] like Figure 1 As shown, the quantitative analysis method for cybersecurity threats provided in this application includes:

[0048] S110: Collect threat data and preprocess the threat data.

[0049] Specifically, it enables high-speed collection of outgoing log data and security alarm data generated by network devices, security devices, operating systems, databases, middleware, and business systems, achieving full lifecycle management of threat data.

[0050] Threat data collection tasks include offline collection, real-time collection, incremental collection, and full collection.

[0051] Threat data can include Log, Metric, TraceID, Flow, and mirrored traffic.

[0052] Threat data collection methods include syslog, Snmp-Trap, text file collection, agent collection, Kafka collection, and database read operations.

[0053] This application supports the parsing of UTF-8, UTF-16, GB2312, GBK, ASCII, and CP936 encodings, and supports the recognition of the above encoding fields. It also supports the independent recognition and parsing of nested encodings.

[0054] After obtaining threat data, the preprocessing of the threat data includes:

[0055] 1) Data format standardization: Data parsing rules are configured automatically or manually to achieve formatted parsing and standardized storage of data.

[0056] 2) Standardization of data content

[0057] For the same type of security threat (i.e., the content of threat data), different device manufacturers may classify it into different threat types, which makes it impossible to unify the standards for subsequent data analysis. Therefore, in the preprocessing stage, it is necessary to use unified parsing rules to format and centrally store the threat data classification.

[0058] If the threat data is in a standardized format, threat content can be parsed and identified through methods such as Keypairs parsing, JSON format parsing, and CSV format parsing. This application supports custom extensions for the data type of the data content.

[0059] By writing identification rules using regular expressions (including nested sub-regular expressions), threat data can be matched precisely at each level in a pipeline manner. It can also handle complex scenarios such as transcoding and semantic conversion, thereby standardizing the data content.

[0060] S120: Analyzes pre-processed threat data to obtain the correlation between all security threats from the same source IP, which is used to perform threat perception on the overall network operation status, information assets, and security devices.

[0061] As an example, a Frequent Pattern Tree (FP-tree) is used to obtain the associations among all security threats from the same source IP. The Frequent Pattern Growth (FP-Growth) algorithm is an alternative method for finding frequent itemsets without using candidate generation, thus improving performance. It reduces search costs by recursively finding short patterns and then concatenating them to grow into longer frequent patterns. The core of this method is the use of a special data structure, the FP-tree, which preserves itemset association information. The Frequent Pattern Tree (FP-tree) is a compact data structure used to store quantitative information about frequent patterns in a database. The FP-tree is created from the initial itemsets of the database. The purpose of the FP-tree is to mine the most frequent patterns. Each node in the FP-tree represents an item in the itemset.

[0062] In this embodiment, the preprocessed threat data is analyzed to obtain the correlation between all security threats from the same source IP, specifically including:

[0063] S1201: Perform data cleaning on the preprocessed threat data, transforming the security threats into a transactional database format, with each security threat representing a transaction. Convert the threat data into an FP-tree format. Specifically, treat each security threat as an item in the transactional database, use the time and location of the security threat as attributes, and combine the attribute values ​​of different security threats into an itemset.

[0064] S1202: Construct a frequent pattern tree (FP-tree) based on a transaction database. During the construction process, frequent itemsets that have a certain level of support and confidence for each security threat are selected based on minimum support and minimum confidence.

[0065] S1203: On a frequent pattern tree (FP-tree), use the frequent pattern growth (FP-growth) algorithm to mine frequent itemsets of security threats from the same source IP.

[0066] S1204: Based on the results of frequent itemset mining, perform correlation analysis on all security threats from the same source IP to obtain the correlation and anomalies between different security threats from the same source IP.

[0067] Figure 2 An example of an FP-tree is shown, where the root node represents null, indicating that no security threat was detected, while the other nodes represent different elements of the security threat. Frequent occurrences are mined within the threat tree to establish logical relationships between threats, and then the threats are merged.

[0068] As an example, the meanings of the markings in the diagram are as follows:

[0069] A: 5 indicates that there were 5 attacks from the same source IP;

[0070] B:2 indicates that a certain important asset was attacked twice;

[0071] C:3 indicates that a high-level attack consists of 3 attacks;

[0072] C:2 indicates that a high-level attack consists of 2 attacks;

[0073] D:1 represents one SQL (Structured Query Language) injection attack;

[0074] D:2 indicates that there were 2 SQL injection attacks;

[0075] E:1 indicates that the attack duration is 1 day;

[0076] Path ABC represents a high-risk attack from a specific attack source on a critical asset.

[0077] The path ABCD represents a high-risk SQL injection attack targeting a critical asset from a specific attack source.

[0078] Path AC represents a high-risk attack launched by a certain attack source against a non-critical asset;

[0079] Path ACD represents a high-risk SQL injection attack launched by a certain attack source against non-critical assets;

[0080] The path ACE represents the duration of a high-risk attack launched by a particular attack source against a non-critical asset.

[0081] S130: Calculate the threat risk value of the source IP using the correlation relationship, and use it as the assessment result of the source IP to classify the threat, determine the severity level of the threat and the corresponding countermeasures.

[0082] Specifically, the associated threat data is quantitatively analyzed through model algorithms, based on the probability of the threat occurring and the severity of the damage to the system's confidentiality, availability, and integrity.

[0083] The risk analysis principles related to threat elements include three points, as follows:

[0084] 1) Utilize threat sources, types, motives, etc., and combine them with historical statistical data such as threat-related security incidents and logs to determine the capability and frequency of threats. This item is related to the number w of threat attack methods described below.

[0085] Threat sources include environmental, accidental, and human-caused factors. Threat types include physical damage, natural disasters, information damage, technological failure, unauthorized actions, functional impairment, and supply chain failure. Threat motives include malicious and non-malicious; this application only describes and quantifies malicious threat motives. Table 1 below is an example of a threat statistics table.

[0086] Table 1

[0087]

[0088] 2) The extent of the impact on assets after a security incident is caused by the exploitation of vulnerabilities, which is related to the number of targets t of the threat attack described below;

[0089] 3) Determine the likelihood of a security incident occurring based on the threat’s capability and frequency, combined with the ease with which the vulnerability can be exploited. This is related to the total duration of the threat attack, d, and the number of targets of the threat attack, t, as described below.

[0090] As one example, calculating the threat risk value of a source IP using correlation relationships specifically includes:

[0091] S1301: Quantitative analysis indicators of threats from IPs of the same source based on correlation.

[0092] As an example, the "4W3H" principle is used to obtain quantitative threat analysis indicators. The "4W3H" principle is shown in Table 2 below:

[0093] Table 2

[0094]

[0095]

[0096] As an example, the quantitative threat analysis indicators include the total duration d of threat attacks from the same source IP, the number of targets t of threat attacks, the number of threat attack methods w, and the total number of threat attacks f(n).

[0097] The total number of threat attacks, f(n), depends on the weights V of the various threat attack methods. w and the number of attacks (Times) w ,Right now

[0098]

[0099] Among them, the weight V of the threat attack method w The security value V of the attacked business system is positively linearly related; the greater the security value V of the attacked business system, the greater the weight V of the threat attack method. wThe larger the scale, the better. Within a business system, a set of information assets designed to achieve the same business objective are managed and maintained as a single unit.

[0100] Determining the security value V of a business system includes the following steps:

[0101] P1: Information Asset Acquisition

[0102] There are two methods for data collection: active and passive.

[0103] Active acquisition: Active acquisition collects live information assets in the network through port scanning. To solve the problem of massive assets in large networks, this application adopts a parallel working mode of Masscan + NMAP + built-in fingerprint database. Masscan (high-speed scanner) performs rapid device location and port collection, NMAP (Network Mapper) is used for fine scanning and asset type identification, and the built-in fingerprint database is used to supplement dedicated equipment and correct results.

[0104] Passive acquisition: The passive acquisition mode collects network traffic passively by mirroring traffic and identifies information assets in network communication through DPI (Deep Packet Analysis) technology.

[0105] P2: Security Assignment of Information Assets

[0106] It should be noted that the security value of a business system differs from the inherent value of the business system itself. Its security value changes continuously in the network environment as network security threats evolve, primarily depending on the confidentiality of all information assets within the business system. i Integrity I i and availability A i For example, if an attacker obtains important information from an information asset, the confidentiality of the information asset is weakened. If the attacker modifies the management rules or other aspects of the information asset, causing its functionality to differ from the default settings, the integrity of the information asset is weakened. If the attacker damages the normal functionality of the information asset, its availability is reduced.

[0107] Specifically, the security value V of the business system is calculated using the following formula:

[0108]

[0109] Among them, C i I i A i These represent the confidentiality, integrity, and availability of the i-th information asset, respectively, and N represents the number of information assets within the business system.

[0110] S1302: Calculate the threat risk value T of the source IP based on quantitative threat analysis indicators.

[0111] T=f(d, t, w, f(n)) (3)

[0112] This embodiment not only considers the time (attack duration, number of times) and spatial dimensions (attack path, source, target) of cybersecurity threats, but also their associated security attributes, thereby significantly improving the accuracy and practicality of the analysis. It can improve the work efficiency of cybersecurity managers and effectively enhance the level of cybersecurity risk management.

[0113] S140: Output assessment results to demonstrate the methods and pathways through which current security threats are attacked and spread in cyberspace.

[0114] An attack path is a visual representation of the continuous flow that occurs during an attack. It emphasizes the "connection points" and examines the entire context of the risk being exploited. This includes starting with the network exposure of the relevant assets, continuing through the risky roles and privilege escalation of those assets, and all the way to the successful execution of attacks on sensitive assets if the attack is exploited or affected.

[0115] As an example, the output data is a visualization of the threat attack chain of the source IP, namely a threat attack chain graph. A threat attack chain graph is a visual attack graph that displays all attack sequences that a malicious user can execute in a concise way to achieve the desired target.

[0116] Example 2

[0117] Based on the above-mentioned quantitative analysis method, this application also provides a quantitative analysis device for network security threats. For example... Figure 3 As shown, the quantitative analysis device includes a data acquisition and preprocessing module 310, a threat perception module 320, a threat quantification module 330, and an output module 340.

[0118] The data acquisition and preprocessing module 310 is used to acquire threat data and preprocess the threat data.

[0119] The threat perception module 320 is used to analyze the pre-processed threat data to obtain the correlation between all security threats from the same source IP.

[0120] The threat quantification module 330 is used to calculate the threat risk value of the source IP using correlation relationships, as an assessment result of the source IP.

[0121] Output module 340 is used to output the evaluation results.

[0122] Preferably, the output data of the output module 340 is a visualization result of the threat attack chain of the source IP.

[0123] Preferably, the threat perception module 320 is used to obtain the correlation between all security threats from the same source IP by employing a frequent pattern tree.

[0124] Preferably, the threat perception module 320 includes a conversion module 3201, a construction module 3202, a mining module 3203, and a correlation analysis module 3204.

[0125] The conversion module 3201 is used to clean the preprocessed threat data, convert the security threats into a transaction database, with each security threat as an item in the transaction database, and the time and space of the security threat as its attributes, and the attribute values ​​of different security threats are combined into an itemset.

[0126] Module 3202 is used to build a frequent pattern tree based on a transaction database, wherein frequent itemsets for each security threat are filtered based on minimum support and minimum confidence.

[0127] The mining module 3203 is used to mine frequent itemsets of security threats from the same source IP on the frequent pattern tree using a frequent pattern growth algorithm.

[0128] The association analysis module 3204 is used to perform association analysis on all security threats from the same source IP based on the mining results of frequent itemsets, and to obtain the association and anomalies between different security threats from the same source IP.

[0129] Preferably, the threat quantification module 330 includes a statistics module 3301 and a calculation module 3302.

[0130] The statistics module 3301 is used to statistically analyze the total duration of threat attacks from the same source IP, the number of targets of threat attacks, the number of threat attack methods, and the total number of threat attacks based on the correlation relationship.

[0131] The calculation module 3302 is used to calculate the threat risk value of the source IP based on the total duration of the threat attack, the number of targets of the threat attack, the number of threat attack methods, and the total number of threat attacks.

[0132] In summary, this application can not only accurately perceive the current real risks and threats in the network, effectively improving network security managers' ability to perceive and respond to risks, but the system also helps to improve the overall level of security governance.

[0133] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. A method for quantitative analysis of network security threats, characterized in that, include: Collect threat data and preprocess the threat data; Analyzing the preprocessed threat data reveals the correlations among all security threats originating from the same IP address. This includes using a frequent pattern tree to identify these correlations. The preprocessed threat data is cleaned to transform the security threats into a transaction database. Each security threat is an item in the transaction database. The time and space of the security threat are used as attributes of the security threat. The attribute values ​​of different security threats are combined into an itemset. A frequent pattern tree is constructed based on the transaction database, wherein frequent itemsets for each security threat are filtered according to minimum support and minimum confidence. On the frequent pattern tree, the frequent pattern growth algorithm is used to mine frequent itemsets of security threats from the same source IP; Based on the results of frequent itemset mining, correlation analysis is performed on all security threats from the same source IP to obtain the correlation and anomalies between different security threats from the same source IP; The threat risk value of the source IP is calculated using the aforementioned correlation, and this value is used as the assessment result for the source IP. Output the evaluation results.

2. The method for quantitative analysis of network security threats according to claim 1, characterized in that, The output data is a visualization of the threat attack chain of the source IP.

3. The method for quantitative analysis of network security threats according to claim 1 or 2, characterized in that, Calculating the threat risk value of the source IP using the aforementioned correlation specifically includes: Based on the aforementioned correlation, quantitative analysis indicators of threats from IP addresses originating from the same source are compiled. The threat risk value of the source IP is calculated based on the aforementioned quantitative threat analysis indicators.

4. A device for quantitative analysis of network security threats, characterized in that, It includes a data acquisition and preprocessing module, a threat perception module, a threat quantification module, and an output module; The data acquisition and preprocessing module is used to acquire threat data and preprocess the threat data; The threat perception module is used to analyze the preprocessed threat data to obtain the correlation between all security threats from the same source IP. The threat perception module is used to obtain the correlation between all security threats from the same source IP using a frequent pattern tree; the threat perception module specifically includes: a conversion module, a construction module, a mining module, and a correlation analysis module; The conversion module is used to clean the preprocessed threat data, convert the security threats into a transaction database, with each security threat as an item in the transaction database, and the time and space of the security threat as attributes of the security threat, and the attribute values ​​of different security threats are combined into an itemset; The construction module is used to construct a frequent pattern tree based on the transaction database, wherein frequent itemsets for each security threat are filtered according to minimum support and minimum confidence. The mining module is used to mine frequent itemsets of security threats from the same source IP on the frequent pattern tree using a frequent pattern growth algorithm; The association analysis module is used to perform association analysis on all security threats from the same source IP based on the mining results of frequent itemsets, and to obtain the association and anomalies between different security threats from the same source IP. Furthermore, the threat quantification module is used to calculate the threat risk value of the source IP using the correlation relationship, as an assessment result of the source IP; The output module is used to output the evaluation results.

5. The quantitative analysis device for network security threats according to claim 4, characterized in that, The output data of the output module is a visualization of the threat attack chain of the source IP.

6. The quantitative analysis device for network security threats according to claim 4 or 5, characterized in that, The threat quantification module includes a statistics module and a calculation module; The statistics module is used to calculate the total duration of threat attacks from the same source IP, the number of targets of threat attacks, the number of threat attack methods, and the total number of threat attacks based on the correlation. The calculation module is used to calculate the threat risk value of the source IP based on the total duration of the threat attack, the number of targets of the threat attack, the number of threat attack methods, and the total number of threat attacks.

Images