Methods for identifying effective threat intelligence and methods for evaluating data providers.

Abstract

This application provides a method for determining effective threat intelligence and an evaluation method for data providers. The method for determining effective threat intelligence includes: receiving pending threat intelligence; identifying the target provider of the pending threat intelligence; determining whether the target provider is a qualified provider, with qualified providers determined based on the number of times their provided threat intelligence has been cited in historical security event analysis; if so, the pending threat intelligence is determined as effective threat intelligence. This method avoids the use of complex classification algorithms when determining effective threat intelligence from multiple threat intelligences, and improves the efficiency of effective threat intelligence determination while ensuring accurate identification.

Description

Technical Field

[0001] This application relates to the field of network information security technology, and in particular to a method for determining effective threat intelligence and a method for evaluating data providers, as well as corresponding devices, electronic devices and storage media. Background Technology

[0002] With the development of the internet, cybersecurity has risen to the level of a national strategy. Threat intelligence plays an increasingly important role in the analysis and defense of cybersecurity. Threat intelligence is knowledge based on known security events, which can include context, mechanisms, indicators, meanings, and actionable recommendations. By analyzing network data through threat intelligence, existing or impending security events can be discovered, allowing for the implementation of corresponding measures to maximize network security protection.

[0003] To more accurately detect security incidents in the network, more accurate threat intelligence is needed for analysis; that is, effective threat intelligence is required. Only effective threat intelligence can uncover truly threatening security incidents in the network. Using ineffective threat intelligence, even if an incident is detected, it may be a mistaken operation or a non-threatening, unrelated incident. Currently, the specific method used to identify effective threat intelligence from a large pool of information is as follows: after acquiring multiple threat intelligence reports, a Bayesian classification algorithm is used to categorize them into two main types: effective threat intelligence and ineffective threat intelligence. In this way, effective threat intelligence is identified from the vast amount of threat intelligence available.

[0004] However, the Bayesian algorithm used in the process of classifying threat intelligence is relatively complex, which leads to a slow classification process and makes it difficult to quickly identify effective threat intelligence from a large number of threat intelligence, thereby reducing the efficiency of identifying effective threat intelligence. Summary of the Invention

[0005] The purpose of this application is to provide a method for determining effective threat intelligence and a method for evaluating data providers, as well as corresponding apparatus, electronic devices and storage media, to improve the efficiency of determining effective threat intelligence.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions:

[0007] The first aspect of this application provides a method for determining effective threat intelligence, the method comprising: receiving pending threat intelligence; determining a target provider of the pending threat intelligence; determining whether the target provider is a qualified provider, wherein the qualified provider is determined based on the number of times the threat intelligence it provides is cited in historical security event analysis; if so, determining the pending threat intelligence as effective threat intelligence.

[0008] A second aspect of this application provides a method for evaluating data providers, the method comprising: obtaining the number of times threat intelligence provided by different data providers is cited in historical security event analysis; determining data providers whose threat intelligence is cited more than a preset number of times as qualified providers, wherein the threat intelligence provided by qualified providers is valid threat intelligence.

[0009] A third aspect of this application provides an apparatus for determining effective threat intelligence. The apparatus includes: a receiving module for receiving pending threat intelligence; a first determining module for determining a target provider of the pending threat intelligence; a judging module for judging whether the target provider is a qualified provider, and if so, proceeding to a second determining module, wherein the qualified provider is determined based on the number of times the threat intelligence it provides is cited in historical security event analysis; and a second determining module for determining the pending threat intelligence as effective threat intelligence.

[0010] A fourth aspect of this application provides an evaluation device for data providers, the device comprising: an acquisition module for acquiring the number of times threat intelligence provided by different data providers is cited in historical security event analysis; and a third determination module for determining data providers whose threat intelligence is cited more than a preset number of times as qualified providers, wherein the threat intelligence provided by qualified providers is valid threat intelligence.

[0011] The fifth aspect of this application provides an electronic device, the electronic device comprising: a processor, a memory, and a bus; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the method of the first aspect or the second aspect.

[0012] A sixth aspect of this application provides a computer-readable storage medium comprising: a stored program; wherein, when the program is executed, it controls the device on which the storage medium is located to perform the method of the first aspect or the second aspect.

[0013] Compared to existing technologies, the method for determining valid threat intelligence provided in the first aspect of this application, upon receiving pending threat intelligence, identifies the target provider of the pending threat intelligence. If the target provider is a qualified provider, the pending threat intelligence provided by the target provider is directly regarded as valid threat intelligence. Since qualified providers are determined based on the number of times their threat intelligence has been cited in historical security event analyses—that is, providers whose threat intelligence has been cited more frequently in historical security event analyses are considered qualified providers, indicating higher utilization value of their threat intelligence—this avoids the use of complex classification algorithms when determining valid threat intelligence from multiple threat intelligences. The number of times threat intelligence has been cited in security event analyses is relatively easy to obtain, and the process of filtering threat intelligence based on citation counts is simple. Furthermore, the filtered threat intelligence is all high-value threat intelligence; data providers offering high-value threat intelligence can also be considered high-quality data providers. Threat intelligence provided by high-quality data providers can be regarded as valid threat intelligence. This method improves the efficiency of determining valid threat intelligence while ensuring its accurate identification.

[0014] The data provider evaluation method provided in the second aspect of this application, the effective threat intelligence determination device provided in the third aspect, the data provider evaluation device provided in the fourth aspect, the electronic device provided in the fifth aspect, and the computer-readable storage medium provided in the sixth aspect have the same or similar beneficial effects as the effective threat intelligence determination method provided in the first aspect. Attached Figure Description

[0015] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:

[0016] Figure 1 This is a flowchart illustrating the method for determining effective threat intelligence in the embodiments of this application;

[0017] Figure 2 This is a flowchart illustrating the evaluation method for the data provider in the embodiments of this application;

[0018] Figure 3 This is a schematic diagram of the structure of the device for determining effective threat intelligence in the embodiments of this application;

[0019] Figure 4 This is a schematic diagram of the structure of the evaluation device of the data provider in the embodiments of this application;

[0020] Figure 5This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0021] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0022] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0023] Currently, to obtain effective threat intelligence, classification algorithms such as Bayesian methods are mainly used to divide multiple threat intelligence into two categories: invalid threat intelligence and valid threat intelligence. However, the computational process of classification algorithms such as Bayesian methods is relatively complex, and the time required to classify multiple threat intelligence into invalid and valid threat intelligence is relatively long, thus reducing the efficiency of determining valid threat intelligence.

[0024] The inventors discovered that the low efficiency of identifying valid threat intelligence stems from the complexity and time-consuming process of classifying multiple threat intelligence entries into valid or invalid ones. Instead of using these complex classification algorithms, a new approach can be adopted: selecting high-quality data providers for threat intelligence. These providers can be identified as valid threat intelligence. The selection of high-quality data providers can be based on the frequency with which their threat intelligence has been used in historical analysis; providers with frequently used threat intelligence are considered high-quality. This bypasses the need for complex algorithms for classifying threat intelligence as valid or invalid, and the acquisition and sorting of usage frequency are simple and easy, thus improving the efficiency of identifying valid threat intelligence.

[0025] In view of this, embodiments of this application provide a method for determining effective threat intelligence and a method for evaluating data providers, as well as corresponding devices, electronic devices, and storage media. Upon receiving pending threat intelligence, the method determines the target provider of the pending threat intelligence. If the target provider is a qualified provider, the pending threat intelligence provided by the target provider is directly regarded as effective threat intelligence. Since qualified providers are determined based on the number of times their provided threat intelligence has been cited in historical security event analysis—that is, providers whose provided threat intelligence has been cited more times in historical security event analysis are considered qualified providers, and their provided threat intelligence has higher utilization value—this avoids the use of complex classification algorithms when determining effective threat intelligence from multiple threat intelligences. The number of times threat intelligence has been cited in security event analysis is relatively easy to obtain, and the process of filtering threat intelligence based on the number of citations is simple. Furthermore, the filtered threat intelligence is all high-value threat intelligence. Data providers providing high-value threat intelligence can also be considered high-quality data providers, and threat intelligence provided by high-quality data providers can be regarded as effective threat intelligence. This improves the efficiency of determining effective threat intelligence while ensuring that it can be accurately determined.

[0026] First, the method for determining effective threat intelligence provided in the embodiments of this application will be described in detail.

[0027] Figure 1 This is a flowchart illustrating the method for determining effective threat intelligence in the embodiments of this application. See [link / reference]. Figure 1 As shown, the method may include:

[0028] S11: Receive pending threat intelligence.

[0029] Pending threat intelligence refers to threat intelligence provided by a particular provider that needs to be assessed for its validity.

[0030] S12: Identify the target provider of pending threat intelligence.

[0031] Each threat intelligence is provided by a provider. The same threat intelligence may be provided by one or more providers. By using the information carried in the threat intelligence or the interface through which the threat intelligence is reported, it is possible to determine which provider provided the pending threat intelligence and identify that provider as the target provider.

[0032] S13: Determine whether the target provider is a qualified provider. If yes, proceed to step S14; otherwise, proceed to step S15.

[0033] Among them, qualified providers are determined based on the number of times their threat intelligence is cited in historical security incident analysis.

[0034] Before assessing the effectiveness of threat intelligence, the capabilities of each provider can be evaluated. This can be done by counting the number of times threat intelligence from each provider has been cited in historical security incident analysis, and storing providers whose threat intelligence has been cited more than a preset number of times as qualified providers. Subsequently, after identifying target providers for potential threat intelligence, they are matched with qualified target providers. If a match is found, the target provider is considered a qualified provider; otherwise, it is not.

[0035] S14: Determine pending threat intelligence as valid threat intelligence.

[0036] Once the target provider is identified as a qualified provider, it indicates that the threat intelligence provided by the target provider has high utilization value in subsequent security incident analysis. Therefore, the pending threat intelligence provided by the target provider can be directly identified as valid threat intelligence.

[0037] S15: Use a preset algorithm to process pending threat intelligence to determine whether the pending threat intelligence is valid intelligence.

[0038] After determining that the target provider is not a qualified provider, it means that most of the threat intelligence provided by the target provider may not have much value in subsequent security incident analysis. However, it cannot be ruled out that all the threat intelligence provided by the target provider is of low value. Therefore, a preset algorithm can be used to process the pending threat intelligence to determine whether the pending threat intelligence is valid.

[0039] As described above, the method for determining valid threat intelligence provided in this application, upon receiving pending threat intelligence, identifies the target provider of the pending threat intelligence. If the target provider is a qualified provider, the pending threat intelligence provided by the target provider is directly regarded as valid threat intelligence. Since qualified providers are determined based on the number of times their threat intelligence has been cited in historical security event analysis—that is, providers whose threat intelligence has been cited more frequently in historical security event analysis are considered qualified providers, indicating higher utilization value of their threat intelligence—this avoids the use of complex classification algorithms when determining valid threat intelligence from multiple threat intelligences. The number of times threat intelligence has been cited in security event analysis is relatively easy to obtain, and the process of filtering threat intelligence based on the number of citations is simple. Furthermore, the filtered threat intelligence is all high-value threat intelligence, and data providers offering high-value threat intelligence can also be considered high-quality data providers. Threat intelligence provided by high-quality data providers can be regarded as valid threat intelligence. This method improves the efficiency of determining valid threat intelligence while ensuring its accurate identification.

[0040] Furthermore, as a response to Figure 1 An extension of the method shown is that, since qualified providers are determined based on the number of times their threat intelligence is cited in historical security incident analysis, qualified providers need to be identified from among the multiple providers of threat intelligence before determining whether the pending threat intelligence is valid.

[0041] Specifically, prior to step S11 above, the method may further include:

[0042] S101: Obtain the number of times threat intelligence from different data providers has been cited in historical security incident analysis.

[0043] Different data providers refer to the various data providers currently offering threat intelligence services to users. The threat intelligence provided by different data providers may be the same or different. For example: Data provider A provides the user with threat intelligence 1 and threat intelligence 2, data provider B provides the user with threat intelligence 1 and threat intelligence 3, and data provider C provides the user with threat intelligence 4.

[0044] Once the data provider offers threat intelligence, analysts can use this intelligence to analyze network data and identify security incidents. Each time an analyst uses a threat intelligence during the analysis, the citation count for that intelligence is incremented by one. For example, given threat intelligences 1, 2, and 3, if analyst A uses threat intelligences 1 and 2 at time a, and uses threat intelligence 3 at time b, while analyst B uses threat intelligence 1 at time b, then during the period from time a to time b, threat intelligence 1 was cited 3 times, threat intelligence 2 was cited 1 time, and threat intelligence 3 was cited 1 time. Of course, the citation count can be adjusted based on the results of historical security incident analysis. The period from time a to time b can be considered part of the historical security incident analysis.

[0045] S102: Determine the data provider of threat intelligence that has been cited more than a preset number of times as a qualified provider.

[0046] After obtaining the number of times each threat intelligence was cited in historical security incident analysis, these citations can be sorted from highest to lowest. The data providers corresponding to the highest preset number of citations are then selected as qualified providers. Here, any value between the preset number and the next highest can be considered a preset number. After obtaining multiple citation counts, each citation count can be compared to the preset number. If it is greater, the data provider corresponding to that citation count is selected as a qualified provider; otherwise, it is skipped.

[0047] The number of target data providers can be one or more. When the quality requirements for effective threat intelligence are high, the preset number of attempts can be set higher, resulting in fewer target data providers (all of whom will be top-tier data providers). When the quality requirements for effective threat intelligence are not high, the preset number of attempts can be set lower, resulting in more target data providers.

[0048] S103: Determine the threat intelligence provided by qualified providers as valid threat intelligence in order to identify security incidents through valid threat intelligence analysis.

[0049] After identifying the target data provider, since the threat intelligence provided by the target data provider has been used frequently by the assessors in historical security incident analysis, it indicates that this threat intelligence is of considerable value in the security incident assessment process. Therefore, the corresponding data provider is capable of providing highly valuable, i.e., effective threat intelligence. Thus, all threat intelligence provided by the target data provider is considered effective threat intelligence, and using this effective intelligence to analyze network data can quickly and accurately assess network security incidents.

[0050] As described above, after obtaining threat intelligence from different data providers, the number of times this threat intelligence has been cited in historical security incident analysis is determined. Data providers whose threat intelligence has been cited more than a preset number of times are identified as qualified providers, and the threat intelligence provided by qualified providers is then identified as valid threat intelligence. This method allows for the rapid and accurate identification of qualified providers, thereby improving the efficiency of identifying valid threat intelligence.

[0051] Furthermore, as a refinement of step S101 above, although some threat intelligence is cited multiple times in historical security incident analysis, the resulting events may not be events posing a security threat, i.e., non-security events. To differentiate between threat intelligence producing different results—that is, to prioritize threat intelligence that can generate genuine security events without completely ignoring the intelligence production work that produces non-security events—each piece of threat intelligence can be evaluated from two dimensions: effectiveness and ineffectiveness, i.e., merit and effort.

[0052] Specifically, step S101 above may include:

[0053] Step A1: Obtain the number of times threat intelligence from different data providers was effectively cited and invalidally cited in historical security incident analysis.

[0054] Valid citations are used to characterize events that assist in identifying security incidents during historical security incident analysis. Invalid citations are used to characterize events that assist in identifying non-security incidents during historical security incident analysis.

[0055] Each threat intelligence piece of information is used multiple times in historical security incident analysis. Each use may result in either a security incident or a non-security incident. For example, the threat intelligence might be "repeatedly changing passwords in numerical order to log in." In one analysis, a security incident might be identified as "multiple login attempts were made on a device belonging to Company A." However, after verification with the employee using the device at Company A, it was found that they had forgotten a digit of their password and attempted to log in; this is a non-security incident. In another analysis, a security incident might be identified as "multiple login attempts were made on multiple devices belonging to Company A." After verification with Company B, it was confirmed that their devices were indeed compromised multiple times; this is a security incident. Therefore, when counting the number of times threat intelligence is cited in historical security incident analysis, it is necessary to separate the number of valid citations from the number of invalid citations. In the example above, the number of valid citations of the intelligence data can be counted as 1, and the number of invalid citations can be counted as 1.

[0056] Step A2: Assign a first weight to the number of valid citations of each threat intelligence, and a second weight to the number of invalid citations of each threat intelligence.

[0057] The first weight is greater than the second weight.

[0058] A threat intelligence being effectively referenced once indicates that the reference generated a security event. Conversely, a threat intelligence being invalidally referenced once indicates that the reference generated a non-security event. To emphasize the role of threat intelligence in generating security events, the number of times threat intelligence is effectively referenced can be appropriately increased. A large parameter, the first weight, can be attached to the number of times threat intelligence is effectively referenced, while a smaller parameter, the second weight, can be attached to the number of times threat intelligence is invalidally referenced.

[0059] When assigning a first weight to the number of valid citations of threat intelligence and a second weight to the number of invalid citations of threat intelligence, the parameter can be multiplied by the corresponding number of citations, or the parameter can be added to the corresponding number of citations, as long as the method of assigning weights to the number of valid citations and the number of invalid citations is the same.

[0060] In practical applications, the first weight can be a value greater than 1, while the second weight can be empty or a value smaller than the first weight. Alternatively, the first weight can be empty, while the second weight can be a value greater than 0 and less than 1.

[0061] Step A3: In each threat intelligence, add the number of valid citations after adding the first weight and the number of invalid citations after adding the second weight to get the number of times multiple threat intelligences are cited.

[0062] For each piece of threat intelligence, its use may sometimes result in a security incident and sometimes in a non-security incident. In other words, each piece of threat intelligence may contain both valid and invalid citations. Since the use of threat intelligence results in either a security incident or a non-security incident, adding the valid and invalid citations gives the total number of times that threat intelligence has been cited in historical security analysis.

[0063] The above analysis shows that the number of times each threat intelligence is cited is divided into the number of valid citations and the number of invalid citations. A larger first weight is assigned to the number of valid citations, and a smaller second weight is assigned to the number of invalid citations. This approach highlights the contribution of threat intelligence in identifying security incidents without ignoring its role in identifying non-security incidents. It more accurately represents the differences between different threat intelligences, thereby more accurately identifying the target data provider and ultimately more accurately determining the validity of the threat intelligence.

[0064] Furthermore, as an extension of step S102 above, when determining the target data provider, not only the threat intelligence provided by each data provider can be referenced, but also other data related to the threat intelligence provided by each data provider can be referenced, because these other data will also be used by the assessors when conducting security incident assessments.

[0065] Specifically, prior to step S102 above, the method may further include:

[0066] Step B1: Obtain the number of times the analytical data provided by different data providers has been used in the analysis of historical security incidents.

[0067] The assessment data refers to data used in security incident analysis, excluding threat intelligence.

[0068] In the process of security incident analysis, analysts do not rely solely on threat intelligence; they also combine it with other data, known as analysis data, to form a comprehensive assessment. This analysis data is provided by data providers. Therefore, obtaining the frequency with which analysis data from different providers has been used in historical security incident analysis can provide a reference for selecting a target data provider.

[0069] It should be noted that the number of times the analysis data is used can also be calculated based on the number of times it is used effectively and the number of times it is used ineffectively. For the specific calculation method, please refer to the calculation method of the number of times threat intelligence is used effectively and the number of times it is used ineffectively, which will not be repeated here.

[0070] Accordingly, step S102 above can be: selecting the data provider of threat intelligence that has been cited more than a first preset number of times and the analysis data that has been used more than a second preset number of times as a qualified provider.

[0071] After obtaining the number of times each threat intelligence was cited and the number of times each assessment data was used, the target data provider can be selected according to the ranking of the number of times each threat intelligence was cited and the number of times each assessment data was used.

[0072] As can be seen from the above, in addition to threat intelligence, the frequency with which analytical data used in security incident analysis is used can more comprehensively evaluate the data provider's ability to provide high-quality data, thereby improving the accuracy of identifying effective threat intelligence.

[0073] Some threat intelligence and analysis data may be provided by the same provider, while others may be provided by different providers. Whether it is based on the number of times threat intelligence is cited in security incident analysis or the number of times analysis data is used in security incident analysis, it is to evaluate the ability of each data provider to provide high-quality data. Therefore, the number of times threat intelligence provided by the same data provider is cited and the number of times analysis data is used can be added together to determine the ability of each data provider to provide high-quality data.

[0074] Specifically, prior to step S102 above, the method may further include:

[0075] Step B2: Add the number of times the threat intelligence provided by the same data provider is cited and the number of times the analysis data is used to obtain the total count for the corresponding data provider.

[0076] In other words, among multiple threat intelligence and multiple assessment data, the data provider for each threat intelligence and each assessment data is identified. Then, the number of times threat intelligence belonging to the same data provider is cited and the number of times assessment data is used are added together to obtain the total number of citations of threat intelligence and assessment data provided by each data provider, i.e., the total count.

[0077] For example, Threat Intelligence 1 is cited twice, provided by data provider A; Threat Intelligence 2 is cited four times, provided by data provider A; Threat Intelligence 3 is cited eight times, provided by data provider B; Analysis Data 1 is used three times, provided by data provider A; Analysis Data 2 is used once, provided by data provider B; and Analysis Data 3 is used seven times, provided by data provider C. Therefore, data provider A provided Threat Intelligence 1, Threat Intelligence 2, and Analysis Data 1, which were cited / used 2, 4, and 3 times respectively, for a total count of 9. Data provider B provided Threat Intelligence 3 and Analysis Data 2, which were cited / used 8 and 1 times respectively, for a total count of 9. Data provider C provided Analysis Data 3, which was used seven times, for a total count of 3. This indicates that data provided by data provider A was used 9 times in the security incident analysis, data provided by data provider B was used 9 times in the security incident analysis, and data provided by data provider C was used 7 times in the security incident analysis.

[0078] Accordingly, step S102 above can be: selecting data providers whose total count is greater than a preset count as qualified providers.

[0079] After obtaining the total count from each data provider, we know how many times their data has been cited / used in security incident analysis. The more times data is cited / used, the higher its value in security incident analysis, and the greater its ability to provide high-quality data. Therefore, data providers with a total count greater than a preset count are the target data providers capable of providing high-quality data in subsequent security incident analysis.

[0080] Continuing the example above, data provider A has a total count of 9, data provider B has a total count of 9, data provider C has a total count of 7, and the preset count is 8. Therefore, data providers A and B are the target data providers capable of providing high-quality data. The data provided by these two providers will then be used for security incident analysis.

[0081] As shown above, when selecting target data providers, ranking them by adding together the number of times their threat intelligence is cited and the number of times their analytical data is used allows for a comprehensive evaluation of the quality of the data provided by each provider. This helps select data providers that offer high-quality data across various aspects, thus improving the selection of target data providers with stronger overall capabilities.

[0082] The importance of various data provided by different data providers in security incident analysis varies. Simply adding up the frequency of different data types would either diminish the importance of critical data or elevate the importance of less important data. Therefore, when adding up the frequency of various data types from the same provider (i.e., threat intelligence and various analytical data), different weights can be assigned to each type of data.

[0083] Specifically, prior to step B2 above, the method may further include:

[0084] Step B02: Determine the weights corresponding to threat intelligence and analysis data respectively.

[0085] The higher the importance of threat intelligence and assessment data in security incident analysis, the greater its corresponding weight.

[0086] Security incident analysis utilizes various types of data, and these data are perceived by analysts to have varying degrees of importance. Generally, analysts prioritize threat intelligence over other analytical data. Therefore, threat intelligence can be given a higher weighting.

[0087] Threat intelligence includes various types, such as Indicator of Compromise (IOC) intelligence and threat radar. The weights assigned to different types of threat intelligence can be the same or different.

[0088] Analysis data includes various types, such as logs, alerts, and clues. The weight of each type of data needs to be determined based on its importance in security incident analysis. For example, clues are equivalent to threat intelligence, and their weight can be the same. Alerts originate from logs. Compared to clues and threat intelligence, analysts tend to prefer using clues and threat intelligence; therefore, alerts have a lower weight than clues, and logs have a lower weight than alerts.

[0089] Accordingly, step B2 above could be: weighting the number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used, according to the weights corresponding to threat intelligence and analysis data respectively.

[0090] That is, the total count of data providers = the weight of threat intelligence × the number of times threat intelligence is cited + the weight of analysis data × the number of times analysis data is used.

[0091] As can be seen from the above, when summing the number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used, assigning corresponding weights to threat intelligence and analysis data can distinguish different types of data provided by the data provider according to their importance in the analysis. This makes the total count of data providers more consistent with the scenario of security incident analysis, thereby enabling the identified target data provider to provide more applicable data for security incident analysis and improving the efficiency and accuracy of security incident analysis.

[0092] When summing various data from the same provider, the more times a data point is cited or used, the larger the sum becomes, resulting in a large total count for all data providers, which is detrimental to comparative calculations. Therefore, the frequency of each data point provided by a data provider can be limited to prevent the total count from becoming excessively large.

[0093] Specifically, step B2 above may include:

[0094] Step B21: According to the first preset classification, classify the number of times the threat intelligence provided by the same data provider is cited into the corresponding level.

[0095] In the first preset classification, multiple adjacent citation counts correspond to a level. The more citations a level has, the higher the level and the higher the score corresponding to that level.

[0096] In other words, multiple levels are pre-defined, with each level corresponding to a certain number of citations. The higher the level, the greater the number of citations within that level, and the higher the corresponding score. For example, in a pre-defined hierarchy, seven levels are set: Level 1 includes 1 citation and scores 1 point; Level 2 includes 2-5 citations and scores 2 points; Level 3 includes 6-10 citations and scores 4 points; Level 4 includes 11-20 citations and scores 8 points; Level 5 includes 21-50 citations and scores 16 points; Level 6 includes 50-100 citations and scores 32 points; and Level 7 includes over 100 citations and scores 64 points.

[0097] For each threat intelligence and analysis data provided by the data provider, the number of times it is cited or used is mapped to a specific level in the first preset classification, and the score of that level is used as the score of the threat intelligence or analysis data.

[0098] Step B22: According to the second preset classification, classify the number of times the analysis data provided by the same data provider is used into the corresponding level.

[0099] In the second preset classification, multiple adjacent usage counts correspond to a level, so that the more times it is cited, the higher the level, and the higher the score corresponding to the level.

[0100] Step B22 here is implemented in the same way as step B21 above, and will not be repeated here.

[0101] It should be noted here that the first preset level and the second preset level can be the same or different.

[0102] Step B23: Determine the first score corresponding to the level of threat intelligence provided by the same data provider in the first preset classification.

[0103] In the first preset classification, different levels correspond to different scores. After determining the level of threat intelligence provided by the same data provider, the level is located in the first preset classification, and the score corresponding to that level is used as the first score.

[0104] Step B24: Determine the second score corresponding to the level of the assessment data provided by the same data provider in the second preset classification.

[0105] Step B24 here is implemented in the same way as step B22 above, and will not be repeated here.

[0106] Step B25: Add the first score and the second score to obtain the total count for the corresponding data provider.

[0107] That is, the total count of data providers = the score of the number of times threat intelligence is cited + the score of the number of times the data is analyzed and cited.

[0108] As can be seen from the above, in the process of adding the number of times threat intelligence from the same data provider is cited and the number of times the analysis data is used, the number of times the threat intelligence is cited is mapped to a specific level in the first preset classification, and the number of times the analysis data is used is mapped to a specific level in the second preset classification. The scores of the corresponding levels are then added as the first score of the threat intelligence or the second score of the analysis data. This reduces the magnitude of the values ​​involved in the addition calculation, thereby improving the calculation efficiency and the efficiency of identifying the target data provider, or even the effective threat intelligence.

[0109] In practical applications, the aforementioned analytical data may include at least one of the following: log data, alarm data, and clue data.

[0110] Log data is generated based on various network behaviors. When a network asset is subjected to network behavior, the behavior is recorded in the logs of the device it resides in, and the data provider can obtain this log through probes. In security incident analysis, analysts can then use the data recorded in these logs to assess the security incident.

[0111] Alert data is generated by matching log data with alert rules. Alert rules can be pre-defined behavioral characteristics that may involve cybersecurity threats by the data provider. After obtaining the logs, the data in the logs can be matched with the alert rules. If the match is successful, it means that the behavior involved in the logs poses a security threat, and the corresponding data in the logs is the alert data, which is a suspected security event.

[0112] Clue data is generated based on the clustering of behaviors of unknown objects. After generating various alert data, some alert data may have some correlation, such as the same IP source or targeting the same object. Aggregating these correlated alert data together forms clue data. Compared to threat intelligence, the perpetrator of the behavior in clue data may not be currently known, so it is unknown whether the perpetrator is benevolent or malevolent. In contrast, the perpetrator of the behavior in threat intelligence is currently known, and the perpetrator has already created a known security threat in the network, which is a cluster of security events involving unknown objects. Threat intelligence, on the other hand, consists of high-value security events involving known objects.

[0113] As can be seen from the above, evaluating data providers through dimensions such as log data, alert data, threat intelligence (IOC and threat radar), and clue data ensures that these dimensions do not overlap and are not omitted from the data used in security incident analysis. This allows for a simpler and more accurate evaluation of data providers, and consequently, a simpler and more accurate identification of target data providers and effective threat intelligence.

[0114] Finally, a complete example will be used to further illustrate the method for determining effective threat intelligence provided in the embodiments of this application.

[0115] In general, it involves scoring the logs, alerts, IOCs, threat radar, and clues reported by each vendor, and evaluating each vendor based on both their contributions and efforts.

[0116] First, before entering logs, alerts, IOCs, threat radars, and leads provided by various vendors into the database, they are tagged according to type, labeled as Y data from vendor X.

[0117] During the analysis of security incidents, a data point is considered to have been referenced once if the analysis personnel perform a correlation analysis on the logs, alerts, IOCs, threat radar, or clues in the database.

[0118] In the scoring process for logs, alerts, IOCs, threat radar, and leads, if the corresponding data is correlated and analyzed, and a security incident is successfully identified (i.e., confirmed by human operations), then the score for that data corresponds to a merit score. If the corresponding data is correlated and analyzed, but a security incident is not successfully identified, or the analysis cannot confirm it as a security incident, or more intelligence information is needed for further analysis, then the score for that data corresponds to a labor score.

[0119] It should be noted that security incidents are categorized by their importance, ranging from extremely serious to serious, relatively serious, and general. If an incident is classified as general, the corresponding data will be counted as one citation. If it is classified as relatively serious, the corresponding data will be counted as two citations. If it is classified as serious, the corresponding data will be counted as four citations. If it is classified as extremely serious, the corresponding data will be counted as eight citations.

[0120] The following is a score table showing the number of times logs, alerts, IOCs, threat radar, and clues are cited, based on merit.

[0121] Table 1

[0122]

[0123] The following is a score table showing the number of times logs, alerts, IOCs, threat radar, and clues are cited, based on effort.

[0124] Table 2

[0125]

[0126] Each vendor provides logs, alerts, IOCs, threat radar, and clues. Some of these logs, alerts, IOCs, threat radar, and clues indicate a security incident during an analysis, while others indicate a non-security incident. Therefore, it's necessary to determine the score based on the number of citations in the merit and effort dimension tables.

[0127] The scoring method for each manufacturer is as follows: Manufacturer A's intelligence performance (per unit of time) = (per unit of time) merit performance value + (per unit of time) effort performance value.

[0128] Merit Performance Score = (Score corresponding to the number of times logs are referenced in the merit dimension table) + (Score corresponding to the number of times alarms are referenced in the merit dimension table) + (Score corresponding to the number of times IOCs are referenced in the merit dimension table) + (Score corresponding to the number of times threat radars are referenced in the merit dimension table) + (Score corresponding to the number of times clues are referenced in the merit dimension table) +

[0129] Hard work performance score = (Score corresponding to the number of times logs are referenced in the Hard Work Dimension Table) + (Score corresponding to the number of times alarms are referenced in the Hard Work Dimension Table) + (Score corresponding to the number of times IOCs are referenced in the Hard Work Dimension Table) + (Score corresponding to the number of times threat radars are referenced in the Hard Work Dimension Table) + (Score corresponding to the number of times clues are referenced in the Hard Work Dimension Table) + .

[0130] It should be noted that the unit of time mentioned above can be in days.

[0131] The typical assessment cycle is weekly, monthly, quarterly, semi-annually, or annually. Now, let's assume a project has purchased intelligence data from five vendors, and the assessment is conducted monthly.

[0132] January performance review results:

[0133]

[0134] February performance review results:

[0135]

[0136] March performance review results:

[0137]

[0138] April performance review results:

[0139]

[0140] May performance review results:

[0141]

[0142] June performance review results:

[0143]

[0144]

[0145] Performance rankings for January to June:

[0146]

[0147] As can be seen, the performance ranking from highest to lowest for January to June is: Vendor C, Vendor B, Vendor D, Vendor A, and Vendor E. This indicates that Vendor C provided the most valuable intelligence. In subsequent analysis and assessment, operations personnel can prioritize using the intelligence data provided by Vendor C to improve analysis efficiency. Furthermore, in the intelligence data purchase for the following year, priority should be given to continuing to purchase intelligence data from Vendor C, and even other types of intelligence data from Vendor C. The intelligence data from Vendor E, which ranked lowest, can be considered for elimination and should no longer be purchased.

[0148] Threat intelligence performance evaluation can be conducted continuously over time, thereby constantly identifying the vendors with the highest intelligence value and improving security protection in a dynamic competitive ranking.

[0149] Based on the same inventive concept, embodiments of this application also provide a method for evaluating data providers. Figure 2 This is a flowchart illustrating the evaluation method for the data provider in this application embodiment. See [link / reference] Figure 2 As shown, the method may include:

[0150] S21: Obtain the number of times threat intelligence provided by different data providers is cited in historical security incident analysis;

[0151] S22: Data providers of threat intelligence that have been cited more than a preset number of times are identified as qualified providers.

[0152] Among them, threat intelligence provided by qualified providers is considered valid threat intelligence.

[0153] Furthermore, the number of times threat intelligence from different data providers is cited in historical security incident analysis includes:

[0154] The number of times threat intelligence provided by different data providers was effectively cited and invalidated in historical security incident analysis is obtained. The effective citations are used to characterize the identification of security incidents in historical security incident analysis, and the invalid citations are used to characterize the identification of non-security incidents in historical security incident analysis.

[0155] A first weight is assigned to the number of valid citations of each threat intelligence, and a second weight is assigned to the number of invalid citations of each threat intelligence, wherein the first weight is greater than the second weight;

[0156] In each threat intelligence, the number of valid citations after adding the first weight and the number of invalid citations after adding the second weight are added together to obtain the total number of times multiple threat intelligences are cited.

[0157] Furthermore, before identifying data providers of threat intelligence that have been cited more than a preset number of times as qualified providers, the method further includes:

[0158] The number of times that analytical data provided by different data providers is used in historical security incident analysis is obtained, wherein the analytical data refers to data other than threat intelligence used in security incident analysis;

[0159] The qualified providers of threat intelligence data that are cited more than a preset number of times include:

[0160] The qualified providers are those whose threat intelligence is cited more than a first preset number of times and whose analysis data is used more than a second preset number of times.

[0161] Furthermore, before selecting data providers of threat intelligence cited more than a first preset number of times and assessment data used more than a second preset number of times as qualified providers, the method further includes:

[0162] The total count for the corresponding data provider is obtained by adding the number of times the threat intelligence provided by the same data provider is cited and the number of times the analysis data is used.

[0163] The qualified providers of threat intelligence that has been cited more than a first preset number of times and analysis data that has been used more than a second preset number of times include:

[0164] Data providers whose total count is greater than the preset count are considered qualified providers.

[0165] Furthermore, before adding the number of times threat intelligence provided by the same data provider is cited and the number of times the assessment data is used, the method also includes:

[0166] Determine the weights of threat intelligence and assessment data, with the higher the importance of threat intelligence and assessment data in security incident analysis, the greater their corresponding weights.

[0167] The summation of the number of times threat intelligence and analysis data provided by the same data provider are cited includes:

[0168] The number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used are weighted according to the respective weights of threat intelligence and analysis data.

[0169] Furthermore, the sum of the number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used to obtain the total count for the corresponding data provider includes:

[0170] According to the first preset classification, the number of times threat intelligence provided by the same data provider is cited is divided into corresponding levels. In the first preset classification, multiple adjacent citation counts correspond to one level. The more citation counts, the higher the level and the higher the score corresponding to the level.

[0171] According to the second preset classification, the number of times the analysis data provided by the same data provider is used is divided into corresponding levels. In the second preset classification, multiple adjacent usage counts correspond to one level, so that the more times it is cited, the higher the level and the higher the score corresponding to the level.

[0172] Determine the first score corresponding to the level of threat intelligence provided by the same data provider in the first preset classification;

[0173] Determine the second score in the second preset classification for the level of the assessment data provided by the same data provider;

[0174] Add the first score and the second score to obtain the total count for the corresponding data provider.

[0175] Furthermore, the analysis data includes at least one of log data, alarm data, and clue data, wherein the log data is generated based on multiple network behaviors, the alarm data is generated based on the matching of log data and alarm rules, and the clue data is generated based on the clustering of the behaviors of unknown objects.

[0176] It should be noted that the description of the above data provider evaluation method embodiment is the same as the description of the first two steps in the above effective threat intelligence determination method embodiment, and has the same beneficial effects as the effective threat intelligence determination method embodiment. For technical details not disclosed in the data provider evaluation method embodiment of this application, please refer to the description of the effective threat intelligence determination method embodiment of this application for understanding.

[0177] Based on the same inventive concept, as an implementation of the above-mentioned method for determining effective threat intelligence, this application also provides an apparatus for determining effective threat intelligence. Figure 3 This is a schematic diagram of the structure of the effective threat intelligence determination device in the embodiments of this application. See also Figure 3 As shown, the device may include:

[0178] Receiver module 31 is used to receive pending threat intelligence;

[0179] The first determining module 32 is used to determine the target provider of the pending threat intelligence;

[0180] The judgment module 33 is used to determine whether the target provider is a qualified provider. If so, it proceeds to the second determination module, whereby the qualified provider is determined based on the number of times the threat intelligence it provides is cited in historical security event analysis.

[0181] The second determining module 34 is used to determine the pending threat intelligence as valid threat intelligence.

[0182] Furthermore, the device also includes: an acquisition module for acquiring the number of times threat intelligence provided by different data providers is cited in historical security event analysis; and a third determination module for determining data providers of threat intelligence that are cited more than a preset number of times as qualified providers.

[0183] Furthermore, the acquisition module is used to acquire the number of times threat intelligence provided by different data providers is effectively cited and the number of times it is invalidally cited in historical security event analysis. The effective citations are used to characterize the assistance in identifying security events in historical security event analysis, and the invalid citations are used to characterize the assistance in identifying non-security events in historical security event analysis. A first weight is added to the number of times each threat intelligence is effectively cited, and a second weight is added to the number of times each threat intelligence is invalidally cited, wherein the first weight is greater than the second weight. In each threat intelligence, the number of times the first weight is added and the number of times the second weight is added are summed to obtain the number of times multiple threat intelligences are cited.

[0184] Furthermore, the device also includes: an acquisition module, used to acquire the number of times that assessment data provided by different data providers is used in historical security incident analysis, wherein the assessment data is data other than threat intelligence used in security incident analysis;

[0185] The third determining module is used to identify the data providers of threat intelligence that has been cited more than a first preset number of times and the analysis data that has been used more than a second preset number of times as qualified providers.

[0186] Furthermore, the device also includes: a calculation module, used to add the number of times the threat intelligence provided by the same data provider is cited and the number of times the analysis data is used to obtain the total count of the corresponding data provider;

[0187] The third determining module is used to identify data providers whose total count is greater than a preset count as qualified providers.

[0188] Furthermore, the device also includes: a configuration module, used to determine the weights corresponding to threat intelligence and assessment data respectively, wherein the higher the importance of threat intelligence and assessment data in security incident analysis, the greater the corresponding weight;

[0189] The calculation module is used to weight the number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used, according to the weights corresponding to threat intelligence and analysis data respectively.

[0190] Further, the calculation module is used to classify the number of times threat intelligence provided by the same data provider is cited into corresponding levels according to a first preset level, wherein multiple adjacent citations in the first preset level correspond to one level, and the more citations, the higher the level and the higher the score corresponding to the level; according to a second preset level, the number of times analysis data provided by the same data provider is used into corresponding levels, wherein multiple adjacent usages in the second preset level correspond to one level, such that the more citations, the higher the level and the higher the score corresponding to the level; determine the first score corresponding to the level of threat intelligence provided by the same data provider in the first preset level; determine the second score corresponding to the level of analysis data provided by the same data provider in the second preset level; and add the first score and the second score to obtain the total count for the corresponding data provider.

[0191] Furthermore, the analysis data includes at least one of log data, alarm data, and clue data, wherein the log data is generated based on multiple network behaviors, the alarm data is generated based on the matching of log data and alarm rules, and the clue data is generated based on the clustering of the behaviors of unknown objects.

[0192] It should be noted that the description of the above embodiments of the effective threat intelligence determination device is similar to the description of the above embodiments of the effective threat intelligence determination method, and has similar beneficial effects. For technical details not disclosed in the embodiments of the effective threat intelligence determination device of this application, please refer to the description of the embodiments of the effective threat intelligence determination method of this application for understanding.

[0193] Based on the same inventive concept, as an implementation of the above-mentioned method for evaluating data providers, this application also provides a device for evaluating data providers. Figure 4 This is a schematic diagram of the evaluation device of the data provider in the embodiments of this application. See also Figure 4 As shown, the device may include:

[0194] The acquisition module 41 is used to acquire the number of times threat intelligence provided by different data providers is cited in historical security incident analysis;

[0195] The third determination module 42 is used to determine the data provider of threat intelligence that has been cited more than a preset number of times as a qualified provider.

[0196] Furthermore, the acquisition module is used to acquire the number of times threat intelligence provided by different data providers is effectively cited and the number of times it is invalidally cited in historical security event analysis. The effective citations are used to characterize the assistance in identifying security events in historical security event analysis, and the invalid citations are used to characterize the assistance in identifying non-security events in historical security event analysis. A first weight is added to the number of times each threat intelligence is effectively cited, and a second weight is added to the number of times each threat intelligence is invalidally cited, wherein the first weight is greater than the second weight. In each threat intelligence, the number of times the first weight is added and the number of times the second weight is added are summed to obtain the number of times multiple threat intelligences are cited.

[0197] Furthermore, the device also includes: an acquisition module, used to acquire the number of times that assessment data provided by different data providers is used in historical security incident analysis, wherein the assessment data is data other than threat intelligence used in security incident analysis;

[0198] The third determining module is used to identify the data providers of threat intelligence that has been cited more than a first preset number of times and the analysis data that has been used more than a second preset number of times as qualified providers.

[0199] Furthermore, the device also includes: a calculation module, used to add the number of times the threat intelligence provided by the same data provider is cited and the number of times the analysis data is used to obtain the total count of the corresponding data provider;

[0200] The third determining module is used to identify data providers whose total count is greater than a preset count as qualified providers.

[0201] Furthermore, the device also includes: a configuration module, used to determine the weights corresponding to threat intelligence and assessment data respectively, wherein the higher the importance of threat intelligence and assessment data in security incident analysis, the greater the corresponding weight;

[0202] The calculation module is used to weight the number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used, according to the weights corresponding to threat intelligence and analysis data respectively.

[0203] Further, the calculation module is used to classify the number of times threat intelligence provided by the same data provider is cited into corresponding levels according to a first preset level, wherein multiple adjacent citations in the first preset level correspond to one level, and the more citations, the higher the level and the higher the score corresponding to the level; according to a second preset level, the number of times analysis data provided by the same data provider is used into corresponding levels, wherein multiple adjacent usages in the second preset level correspond to one level, such that the more citations, the higher the level and the higher the score corresponding to the level; determine the first score corresponding to the level of threat intelligence provided by the same data provider in the first preset level; determine the second score corresponding to the level of analysis data provided by the same data provider in the second preset level; and add the first score and the second score to obtain the total count for the corresponding data provider.

[0204] Furthermore, the analysis data includes at least one of log data, alarm data, and clue data, wherein the log data is generated based on multiple network behaviors, the alarm data is generated based on the matching of log data and alarm rules, and the clue data is generated based on the clustering of the behaviors of unknown objects.

[0205] It should be noted that the description of the evaluation device embodiment of the data provider above is similar to the description of the evaluation method embodiment of the data provider above, and has similar beneficial effects as the evaluation method embodiment of the data provider. For technical details not disclosed in the evaluation device embodiment of the data provider in this application, please refer to the description of the evaluation method embodiment of the data provider in this application for understanding.

[0206] Based on the same inventive concept, embodiments of this application also provide an electronic device. Figure 5 This is a schematic diagram of the electronic device in an embodiment of this application. See also... Figure 5As shown, the electronic device may include: a processor 501, a memory 502, and a bus 503; wherein the processor 501 and the memory 502 communicate with each other through the bus 503; the processor 501 is used to call program instructions in the memory 502 to execute the methods in one or more of the above embodiments.

[0207] It should be noted that the descriptions of the above electronic device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the electronic device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0208] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium, which may include: a stored program; wherein, when the program is running, it controls the device where the storage medium is located to execute the methods in one or more of the above embodiments.

[0209] It should be noted that the descriptions of the storage medium embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0210] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining effective threat intelligence, characterized in that, The method includes: Receive pending threat intelligence; Identify the target provider of the proposed threat intelligence; Determine whether the target provider is a qualified provider. The qualified provider is determined based on the number of times the threat intelligence it provides is cited in historical security incident analysis. If so, the pending threat intelligence will be determined as valid threat intelligence; The method further includes, prior to receiving the pending threat intelligence: Obtain the number of times threat intelligence from different data providers is cited in historical security incident analysis; Data providers whose threat intelligence is cited more than a preset number of times are identified as qualified providers; The frequency with which threat intelligence from different data providers is cited in historical security incident analysis includes: The number of times threat intelligence provided by different data providers was effectively cited and invalidated in historical security incident analysis is obtained. The effective citations are used to characterize the identification of security incidents in historical security incident analysis, and the invalid citations are used to characterize the identification of non-security incidents in historical security incident analysis. A first weight is assigned to the number of valid citations of each threat intelligence, and a second weight is assigned to the number of invalid citations of each threat intelligence, wherein the first weight is greater than the second weight; In each threat intelligence, the number of valid citations after adding the first weight and the number of invalid citations after adding the second weight are added together to obtain the total number of times multiple threat intelligences are cited.

2. The method according to claim 1, characterized in that, Before data providers of threat intelligence that have been cited more than a preset number of times are designated as qualified providers, the method further includes: The number of times that analytical data provided by different data providers is used in historical security incident analysis is obtained, wherein the analytical data refers to data other than threat intelligence used in security incident analysis; The qualified providers of threat intelligence data that are cited more than a preset number of times include: The qualified providers are those whose threat intelligence is cited more than a first preset number of times and whose analysis data is used more than a second preset number of times.

3. The method according to claim 2, characterized in that, Before data providers of threat intelligence cited more than a first preset number of times and analytical data used more than a second preset number of times are designated as qualified providers, the method further includes: The total count for the corresponding data provider is obtained by adding the number of times the threat intelligence provided by the same data provider is cited and the number of times the analysis data is used. The qualified providers of threat intelligence that has been cited more than a first preset number of times and analysis data that has been used more than a second preset number of times include: Data providers whose total count is greater than the preset count are considered qualified providers.

4. The method according to claim 3, characterized in that, Before adding the number of times threat intelligence provided by the same data provider is cited and the number of times the assessment data is used, the method further includes: Determine the weights of threat intelligence and assessment data, with the higher the importance of threat intelligence and assessment data in security incident analysis, the greater their corresponding weights. The summation of the number of times threat intelligence and analysis data provided by the same data provider are cited includes: The number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used are weighted according to the respective weights of threat intelligence and analysis data.

5. The method according to claim 3, characterized in that, The method of adding the number of times threat intelligence provided by the same data provider is cited and the number of times the analysis data is used to obtain the total count for the corresponding data provider includes: According to the first preset classification, the number of times threat intelligence provided by the same data provider is cited is divided into corresponding levels. In the first preset classification, multiple adjacent citation counts correspond to one level. The more citation counts, the higher the level and the higher the score corresponding to the level. According to the second preset classification, the number of times the analysis data provided by the same data provider is used is divided into corresponding levels. In the second preset classification, multiple adjacent usage counts correspond to one level, so that the more times it is cited, the higher the level and the higher the score corresponding to the level. Determine the first score corresponding to the level of threat intelligence provided by the same data provider in the first preset classification; Determine the second score in the second preset classification for the level of the assessment data provided by the same data provider; Add the first score and the second score to obtain the total count for the corresponding data provider.

6. The method according to any one of claims 2 to 5, characterized in that, The analysis data includes at least one of log data, alarm data, and clue data, wherein the log data is generated based on multiple network behaviors, the alarm data is generated based on the matching of log data and alarm rules, and the clue data is generated based on the clustering of the behaviors of unknown objects.

7. A method for evaluating data providers, characterized in that, The method includes: Obtain the number of times threat intelligence from different data providers is cited in historical security incident analysis; The data providers of threat intelligence that are cited more than a preset number of times are identified as qualified providers, wherein the threat intelligence provided by the qualified providers is valid threat intelligence; The frequency with which threat intelligence from different data providers is cited in historical security incident analysis includes: The number of times threat intelligence provided by different data providers was effectively cited and invalidated in historical security incident analysis is obtained. The effective citations are used to characterize the identification of security incidents in historical security incident analysis, and the invalid citations are used to characterize the identification of non-security incidents in historical security incident analysis. A first weight is assigned to the number of valid citations of each threat intelligence, and a second weight is assigned to the number of invalid citations of each threat intelligence, wherein the first weight is greater than the second weight; In each threat intelligence, the number of valid citations after adding the first weight and the number of invalid citations after adding the second weight are added together to obtain the total number of times multiple threat intelligences are cited.

8. A device for determining effective threat intelligence, characterized in that, The device includes: The receiving module is used to receive pending threat intelligence. The first determining module is used to determine the target provider of the pending threat intelligence; The judgment module is used to determine whether the target provider is a qualified provider. If so, the process proceeds to the second determination module, whereby the qualified provider is determined based on the number of times the threat intelligence it provides is cited in historical security event analysis. The second determining module is used to determine the pending threat intelligence as valid threat intelligence; The device further includes: an acquisition module for acquiring the number of times threat intelligence provided by different data providers is cited in historical security event analysis; and a third determination module for determining data providers of threat intelligence whose cited number of times is greater than a preset number as qualified providers. The acquisition module is used to acquire the number of times threat intelligence provided by different data providers is effectively cited and the number of times it is invalidally cited in historical security event analysis. The effective citations are used to characterize the assistance in identifying security events in historical security event analysis, and the invalid citations are used to characterize the assistance in identifying non-security events in historical security event analysis. A first weight is added to the number of times each threat intelligence is effectively cited, and a second weight is added to the number of times each threat intelligence is invalidally cited, wherein the first weight is greater than the second weight. For each threat intelligence, the number of times the first weight is added and the number of times the second weight is added are summed to obtain the number of times multiple threat intelligences are cited.

9. An evaluation device for a data provider, characterized in that, The device includes: The acquisition module is used to obtain the number of times threat intelligence provided by different data providers is cited in historical security incident analysis; The third determination module is used to determine the data provider of threat intelligence that has been cited more than a preset number of times as a qualified provider, wherein the threat intelligence provided by the qualified provider is valid threat intelligence; The acquisition module is used to acquire the number of times threat intelligence provided by different data providers is effectively cited and the number of times it is invalidally cited in historical security event analysis. The effective citations are used to characterize the assistance in identifying security events in historical security event analysis, and the invalid citations are used to characterize the assistance in identifying non-security events in historical security event analysis. A first weight is added to the number of times each threat intelligence is effectively cited, and a second weight is added to the number of times each threat intelligence is invalidally cited, wherein the first weight is greater than the second weight. For each threat intelligence, the number of times the first weight is added and the number of times the second weight is added are summed to obtain the number of times multiple threat intelligences are cited.

10. An electronic device, characterized in that, The electronic device includes: a processor, a memory, and a bus; wherein the processor and the memory communicate with each other via the bus; the processor is used to call program instructions in the memory to execute the method as described in any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that, The storage medium includes: a stored program; wherein, when the program is executed, it controls the device where the storage medium is located to perform the method as described in any one of claims 1 to 7.

Images